DEV Community: openHacking

Python + ChatGPT API Development | Based on gpt-3.5-turbo model

openHacking — Thu, 02 Mar 2023 06:58:01 +0000

Background

ChatGPT is a cloud-based artificial intelligence chatbot that uses OpenAI's GPT-3.5-turbo model to implement natural language processing (NLP) and language generation tasks. The new GPT-3.5-turbo model is an upgrade and improvement based on the GPT-3 model, with higher accuracy and expressiveness, it can automatically identify the semantics and context of text, and generate more accurate and Natural reply.

The OpenAI team released the latest intelligent chatbot ChatGPT API and voice-to-text Whisper API. ChatGPT API provides a convenient and easy-to-use way for developers to use this powerful NLP model easily. The latest release of the ChatGPT API uses the model family gpt-3.5-turbo which is the same model used in the ChatGPT product.

chatopenai.pro

ChatGPT API

Introduction

I wrote an article about Develop an Intelligent Chat Program Using Python and ChatGPT API before, introducing the use of text-davinci-002 (or text-davinci-003) model as an example of Python development, this time ChatGPT officially updated the gpt-3.5-turbo model, there will be some adjustments on the interface, providing more API parameters And options, so that developers can customize and optimize the performance of the API according to their own needs.

If the new model is used, the price is $0.002 per 1000 tokens, which is 10 times cheaper than the current GPT-3.5 model. It has high availability and scalability, and can also handle high-traffic application scenarios. The official test engineer migrated from text-davinci-003 to gpt-3.5-turbo, only need a few adjustments to complete the development, and provides very friendly development documents and sample codes to help developers get started quickly.

Usage

If you just want to quickly verify that the ChatGPT API is available, you can try using curl to send the request.

curl https://api.openai.com/v1/chat/completions
   -H "Authorization: Bearer $OPENAI_API_KEY"
   -H "Content-Type: application/json"
   -d'{
   "model": "gpt-3.5-turbo",
   "messages": [{"role": "user", "content": "What is the OpenAI mission?"}]
}'

If you need to go deep into the development of ChatGPT API, I will show you how to use Python + ChatGPT API to quickly build software based on ChatGPT.

Here is a code sample for basic usage:

# Pay attention to check that your OpenAI version must be v0.27.0 or above
import openai

# First, you need to set up your API key
openai.api_key = "YOUR_API_KEY"

# Then, you can call the "gpt-3.5-turbo" model
model_engine = "gpt-3.5-turbo"

# set your input text
input_text = "Where is the 2014 World Cup held?"

# Send an API request and get a response, note that the interface and parameters have changed compared to the old model
response = openai.ChatCompletion.create(
   model=model_engine,
   messages=[{"role": "user", "content": input_text }]
)

# response will get a json message with a structure like this
# {
#  'id': 'chatcmpl-6p9XYPYSTTRi0xEviKjjilqrWU2Ve',
#  'object': 'chat.completion',
#  'created': 1677649420,
#  'model': 'gpt-3.5-turbo',
#  'usage': {'prompt_tokens': 56, 'completion_tokens': 31, 'total_tokens': 87},
#  'choices': [
#    {
#     'message': {
#       'role': 'assistant',
#       'content': 'The 2014 FIFA World Cup was held in Brazil.'},
#     'finish_reason': 'stop',
#     'index': 0
#    }
#   ]
# }

# Parse the response and output the result
output_text = response['choices'][0]['message']['content']
print("ChatGPT API reply:", output_text)

In the code above, we first set up our API key, then specified the GPT-3.5-turbo model to use, set the input text and sent the API request, and finally parsed the response and output the ChatGPT reply . There are a few caveats

New interface: Originally used text-davinci-002 model, we used openai.Completion.create interface, now the new gpt-3.5-turbo should use openai.ChatCompletion.create interface.

New parameters: The first parameter sets the model model name, the second parameter is a conversation list

Why provide a conversation list? Because the API call is a single interface request, the previous chat information will not be automatically recorded, and there is no context. To let ChatGPT understand your context in a single request, you need to provide such a complete list of conversations, such as this one dialogue

import openai

openai.ChatCompletion.create(
   model="gpt-3.5-turbo",
   messages=[
         {"role": "system", "content": "You are a helpful assistant."},
         {"role": "user", "content": "Who won the world series in 2020?"},
         {"role": "assistant", "content": "The Los Angeles Dodgers won the World Series in 2020."},
         {"role": "user", "content": "Where was it played?"}
     ]
)

Each dialog message needs to provide role and content. There are three roles: system, user or assistant.

system: The system message is equivalent to an administrator, who can set the behavior and characteristics of the assistant. In the example above, the assistant is indicated You are a helpful assistant.
user: The user message is ourselves, which can be asked by the user, or directly let the developer build some prompts in advance. Some reference ChatGPT Prompts
assistant: The assistant message is the reply provided by the ChatGPT API before, and it is stored here. You can also modify this reply or make up a dialogue yourself to make the whole dialogue more smooth.

If you don't need a dialog, just provide a single user message, as demonstrated in the Python code before.

For more development documents, please refer to Chat completions

Whisper API

OpenAI also released the Whisper API that supports speech-to-text. It offers a large-v2 model, which is easy to use, priced at $0.006/minute, can be accessed on demand, and a highly optimized service stack can ensure faster performance.

Whisper API is available through transcriptions (transcribes in source language) or translations (transcribes into English) endpoints, and accepts a variety of formats (m4a, mp3, mp4, mpeg, mpga, wav, webm):

A simple curl example

curl https://api.openai.com/v1/audio/transcriptions \
   -H "Authorization: Bearer $OPENAI_API_KEY" \
   -H "Content-Type: multipart/form-data" \
   -F model="whisper-1" \
   -F file="@/path/to/file/openai.mp3"

Interface response format

{
   "text": "Imagine the wildest idea that you've ever had, and you're curious about how it might scale to something that's a 100, a 1,000 times bigger..."
}

Use in Python

import openai

file = open("/path/to/file/openai.mp3", "rb")
transcription = openai.Audio.transcribe("whisper-1", f)

print(transcription)

For more development documents, please refer to Speech to text

Conclusion

The above is just a simple example showing how to use Python to send requests with ChatGPT API, you can use ChatGPT API to complete more complex tasks according to your needs.

Reference

Develop an Intelligent Chat Program Using Python and ChatGPT API

openHacking — Tue, 28 Feb 2023 07:26:03 +0000

Introduction

In this blog, I will introduce how to develop a smart chat program using Python and ChatGPT API.
ChatGPT is a natural language processing tool based on artificial intelligence technology, which can generate natural and fluent language and conduct conversations. Using the ChatGPT API, we can easily integrate this artificial intelligence technology into our Python applications.

Original:Develop an Intelligent Chat Program Using Python and ChatGPT API

Solution

The following are the steps to implement a smart chat program:

Step 1: Install the necessary Python libraries

Before we begin, we need to install some necessary Python libraries. Run the following command in a terminal:

pip install openai
pip install python-dotenv
pip install Flask

These libraries are:

openai: Provides access and control to the ChatGPT API;
python-dotenv: used to save sensitive information such as API keys in environment variables;
Flask: Used to implement web applications.

Step 2: Obtain an API key

Before using the ChatGPT API, we need to obtain an API key. You can obtain an API key by visiting https://platform.openai.com/account/api-keys. Here you need to provide some information about your application such as application name, application description and application URL etc. Once your application is approved, you will receive an API key that authorizes your application to access the ChatGPT API.

Step 3: Create the Flask Application

Now, we will create a web application using the Flask framework.

First create a blank directory, then create a .env file, and fill in the API key you just applied for

OPENAI_API_KEY = '[YOUR_API_KEY]'

Create a file called app.py in your code editor and write the following code in it:

from flask import Flask, request, jsonify
from dotenv import load_dotenv
import openai
import os

# load environment variables
load_dotenv()

# Set the OpenAI API key
openai.api_key = os.getenv("OPENAI_API_KEY")

# Initialize the Flask application
app = Flask(__name__)

# Define the POST route
@app.route('/chat', methods=['POST'])
def chat():
     # Get the message of the POST request
     message = request.json['message']

     # Use the ChatGPT API to generate a response message
     response = openai. Completion. create(
         engine="text-davinci-002",
         prompt=message,
         max_tokens=1024,
         n=1,
         stop=None,
         temperature=0.5
     )

     # Send the response message back to the client
     return jsonify(response.choices[0].text.strip())

Here, we first loaded the .env file where we saved our API key. Then, we initialized a Flask application and defined a route called chat. In this route, we use the POST request to get the message, and then use the ChatGPT API to generate the response message. Finally, we send the response message back to the client as a JSON object.

Step 4: Run the application

Now that we have finished writing our Flask application, we can start the application in the terminal with the following command:

python -m flask --app app.py run

This will start the web application at http://127.0.0.1:5000/. We can now test our application by sending a POST request to http://127.0.0.1:5000/chat using any HTTP client such as Postman or cURL.
When sending a POST request, we need to include the message as part of the JSON object in the request body. For example, if we want ChatGPT to answer "Hello", we can use the following command:

curl --header "Content-Type: application/json" \
   --request POST \
   --data '{"message":"Hello"}' \
   http://127.0.0.1:5000/chat

or use Postman

This will return a JSON object containing the response message generated by ChatGPT.

Conclusion

In this blog, we created a smart chat program using Python and the ChatGPT API. We have created a web application using Flask framework and using openai library and ChatGPT API to generate response messages. Finally, we test our application by sending a POST request. This example shows how to use the ChatGPT API to implement smart chat functionality, and how to integrate ChatGPT into a Python application.

Reference

Open Source IM System Tinode Deployment Tutorial WSL Environment

openHacking — Thu, 09 Feb 2023 04:34:21 +0000

Introduction

Our requirement is to deploy an IM system locally, choose tinode. In order to facilitate the startup of the backend, we use the WSL environment and install the database with docker to start the IM application.

Solution

WSL starts front-end and back-end services

cmd input WSL --update to ensure that WSL is relatively new
Enter WSL with cmd, make sure docker is installed, docker ps -a to see if it starts.
if prompted
```
wsl Cannot connect to the Docker daemon at unix:///var/run/docker.sock. Is the docker daemon running?
```
Need to start docker: sudo service docker start

docker install mysql

docker run --network="host" --name im-mysql -e MYSQL_ROOT_PASSWORD=123456 -d mysql:latest

Download https://github.com/tinode/chat/releases/ linux package
Put it in the WSL directory and unzip it
Execute ./init-db -data=data.json
Execute ./tinode
If you want to start on the server, use
```
nohup ./tinode >im.out 2>&1 &
```
Check whether the startup is successful ps aux|grep tinode
Open http://localhost:6060/
Login test, user name bob password bob123, user alice password alice123

The local deployment has been completed. If you want to modify the front-end code, the front-end needs to pull it from another repository

Front-end development

First make sure that the back-end has started normally, and then look at the front-end part alone

WSL ensures that git is installed, and extracts the front-end code from an empty directory
```
git clone https://github.com/tinode/webapp.git
```
WSL ensures that nodejs is installed (refer to nodejs-on-wsl ), installation dependencies
```
cd webapp
npm i
```
Package after modifying the code His project does not have a development mode, and it can only be repackaged after each modification
```
npm run build
```
Open index-dev.html with the VSCODE Live Server plugin But you need to change 127.0.0.1 to localhost
```
http://localhost:8080/index-dev.html`
```
Every time you open it, it is best to clear the browser cache and refresh

How to clear the browser cache: Open the browser console, right-click the refresh button on the left side of the address bar, click Empty cache and hard refresh
Login test

Reference

3 Ways to Implement Network Requests in Pyodide

openHacking — Sun, 17 Jul 2022 01:14:44 +0000

Question

I mentioned an Online Python Tool before, and the core technology is to use a tool called Pyodide library, which can make Python run on web pages, but I found in the learning process that not all Python built-in libraries or extension libraries can run, such as requests is not supported.

According to the discussion under this issue, requests relies on Lib/http.client.py which relies on Lib/sockets.py which relies on socketmodule.c which needs <sys/socket.h>. Emscripten provides <sys/socket.h> support but if we used this, then the http requests would only work if we used them with a custom server that accepts web socket connections (or proxied all requests through such a server that could forward the requests as true http requests). So that wouldn't be ideal.

So if your local Python program uses the requests method, when switching to our online Python tool to run, you need to update the usage of network requests. Here we briefly introduce the three methods of network request.

Solution

1. http.open_url

Fetches a given URL synchronously.

import pyodide
print(pyodide.http.open_url('https://mocki.io/v1/d4867d8b-b5d5-4a48-a4ab-79131b5809b8').read())

Online Demo: Python Playground - http.open_url

2. http.pyfetch

Fetch the url and return the response.

import pyodide

async def get_data():
    response = await pyodide.http.pyfetch('https://mocki.io/v1/d4867d8b-b5d5-4a48-a4ab-79131b5809b8')
    data = await response.json()
    print(data)

get_data()

Online Demo: Python Playground - http.pyfetch

This method is recommended for obtaining image data

# request image
response = await pyodide.http.pyfetch('https://gcore.jsdelivr.net/gh/openHacking/static-files@main/img/16576149784751657614977527.png')
# Return the response body as a bytes object
image_data = await response.bytes()

Detailed reference case: Python Image to ASCII Online

3. fetch in js module

Pyodide wraps the js API, and uses the native js fetch API to implement network requests

import json
from js import fetch, JSON

async def get_data():
    response = await fetch('https://mocki.io/v1/d4867d8b-b5d5-4a48-a4ab-79131b5809b8',{'method':'GET'})
    data = await response.json()
    print(JSON.stringify(data))

await get_data()

Online Demo: Python Playground - js fetch

Conclusion

The above are the three network request methods commonly used in Pyodide that I have summarized. There may be many shortcomings. If you have a better method, please share it.

Reference

Python Image to ASCII Online

openHacking — Thu, 14 Jul 2022 07:47:38 +0000

Introduction

Recently I found a fun way to play Python, using Python to convert pictures into ASCII image, most of which are local programs, because I built an online version of Python before, so I tried to use the online version of Python to realize image-to-ASCII.

Online Python editor: https://lwebapp.com/en/python-playground

Analysis

The basic idea of image-to-ASCII painting is

Use PIL to get the RGB color value of each pixel of the image
Then map the RGB color value to a string
Finally, just splicing the strings together, printing to the terminal or saving as text, you can see the ASCII image.

We are here for the convenience of demonstration, and use the print method to print characters uniformly.

Code

Since we want to present on a web page, we can't load the image locally, we choose to load the image directly from base64 encoding, or load it from a remote URL. Here we will introduce how to load images.

1. Use base64 images

Use regular expressions to remove the header information of the picture base64 encoded string, and then use the base64 method to decode, and BytesIO cooperates with PIL to open it as a PIL picture instance.

Here I recommend my Online Image to Base64 tool, which supports directly pasting images and converting them into base64 strings, which is very convenient

import base64
import re
from io import BytesIO
from PIL import Image

# random string
char = list('M3NB6Q#OC?7>!:–;. ')

# color value map string
def get_char(r, g, b, alpha=256):
    if alpha == 0:
        return ' '

    grey = (2126 * r + 7152 * g + 722 * b) / 10000

    char_idx = int((grey / (alpha + 1.0)) * len(char))
    return char[char_idx]

# image base64 string
img_base64 = '''data:image/png;base64,iVBORw0KGgoAAAANSUhEUgAAAMgAAADICAYAAACtWK6eAAAAAXNSR0IArs4c6QAAHR5JREFUeF7tnQvUPlVVxh9FES1CU9SCLmhKoHkBwbxLamgm2sULmFqCrZVkllqaWmpGeb+irWVgmgmplYK3zPsNTQJTC8VL1ApKRVOkC6FY6wdn/r7f+837zpmZvc85M3P2WrPeD/4z5+yzzzxzbns/+2qqUi1QLbDRAlertqkWqBbYbIEKkPp2VAtssUAFiP/rsa+km0i6kST+5vrulb/X/xuNLg3Xf678zf9b/e8vSfqn8O/+rVhoDRUgNh0PAA4KQFj/vYFNFRtL+UoAygUbfp2rn3fxFSDD+vfWkn5C0tHhGlZKmqfeIYnrPZI+kabK+dRSARLXlz8i6U6S7hIAcWDcY8XddWEAywclfVjS54vTsDCFKkA2d8i9JN1H0p0lHVFYv1mpc7akD0l6u6R3WhU6p3IqQHb25uGSjgnXbebU0RFt+XtJZ4brnIj7F3FLBchVi+sGFKwrqly1XmnAwuJ/sbJUgOwl6bgVYOy92Ddge8MvXwHKaZKuWJqdlgaQ/ST9UrhutbTOHtneT0r6k3BdMrKsyTy+FIAcEEDxyHBeMZkOKlBRplyvDEC5qED9TFWaO0AOlgQoGDX2N7VcLeziABLAcv5czTFXgNxW0gkBGNeea+cV0q7/CUA5RdLHC9HJTI05AuS3JT05+DuZGaoW1GkB/MT+QNIfdt45oRvmBBAO9QAHp91V8lmAU3pAwuHj5GUOALlxGDEeM/nemFcDXhpGlC9OuVlTBwiLb6ZT+EpVKc8C+Hox7WJ7eJIyVYDgTfsUSQ+cpNWXp/QbJJ00RW/iKQLkFyU9W9INl/eeTbrFX5b0REmvmlIrpgQQtmsBRl1rTOkN260raxOAwvZw8TIVgBCLATj4nZLgy/TpcJ0Xflm0Xrblon37bLnYlDhE0qHhl7+n5ktGLAog4bdomQJAGDEAR+kHfpwD4AVLfEUDii8k6v2broCF+BW8kol7L1kYQQAJI0qxUjJAWGMADNYcJcq3AiDeK+kDks4qTMk7SrqrpKMCYK5RmH6NOqxJAAprlOKkVIAcKekVktitKklgEXmrpAYUXy1JuS26XH8FLPcN5BIlqU6s/C9L+lhJSqFLiQD5KUn49XxfIcb6twAKgMHFyDFlYSQBJM31/YU05t+D/9zbCtHnSjVKA8ixkl4t6ZqZjcTI0ACC3//KrI9X9d+1BhZGmpzyTUmPkHR6TiVW6y4JIMeHkSOnbf5Z0qnh4ou2JGHEpg+4fjhzw/HEph+ySykA+dXMuxmfWQHG17L3Sl4FrrcClB/NqAq7lydnrL+YKdZvhd2qHLaAyYMvFUE//51DgYLrvE4INmNEycXwwu7Wc3LaKPcI8nRJT8tgABbez5L0MknfzlD/lKq8uqQTJT1JUo4F/TMk8Z5kkZwAwROXL3dq+eMADrZsq8RbAP5hQPKo+EfM7iRsOotHcC6A3DtDQM1HAzDOMOu2ZRZ0/wCUH0/cfALi/jpxnVm2eTnZxSUjlbBFy3SKa+pnGKls1lUPZymMJlxsFacSXGg4pE0mqUcQnA3xVUolfxM6cXZkAqkM2FEP5Bh8eH4yoT74miVzckwJEAig8VnCUzWFPE/Sb6aoqNah50p6QiI74AmNjxnE2+6SCiCwGP6FpJu5t0gi8xLAeE2CumoV37HAw6QrgUImLW/5nKSflwTbo6ukAAheuUx1UjgeUg/gcDeca69Mt3A+hIAkxZQLB0fqcfUCTgEQtudSuKzXKVU5wEo15cJVnuMCN/EGCO4CL3HT/jsFQ+AAe0aVciwA2wxEDd7ya55uSp4AYceKrEXekYCPl/QC716o5Q+ywOMkPX/Qk/EPEZlINjCXnS0vgAAKwOEdQ/7wuhiPf5My3cni/U+d6wYcgMScCMILIEyrvNlH7iCJ0/Eq5VuAU/ePOKtJbDvTLVPxAAgLcm+/GXKPTyXc1bTDJlwYwVjkdPcUFuymvFvWAGErl61WT1K3fSXBIFJlehaAaeVSR7XZ8mXr1ywfvDVAXu9MB8pBY83t7fiGJSgaHmUO+rwEmtMHWRVuCRBv93XcC6DWrzJ9C5CiArcjLzFzj7cCCGx/vLxeLOt8EfgyVJmPBSAeZ8bhIcwyAOHo1AtWAPHctXI9CPLonVpmtAU8D5JNdrUsAEIgixeXUXUfiX7XJnujp1sKHGujMl1ZAIS5pEfaM3bDjp5st1fF+1jgHU4Ojkz7WbsOlrEAISeghw8ULuts11Wv3MFdO6kH8QLmg+jhKo9P2ODEomMAQjQZo4cHi3h1IZnU+22irJdLCmdmjCKDokrHAATKnEebmGZnIXXd4WDUiRTptR55eaAu6m2GoQA5OCDS2lO3rjt6d+HsHvBYj+DEyIzn/L7WGgoQ8nbAiGgpsI+w2B80FFoqUsvKagFeZBbX1mwpMDTC1NhLhgDkgPAS79+rpu6bf0fS73ffVu9YgAWeKumZxu28OIwiF/UpdwhAPJTHbZ3Ro/JW9em9+d4L7xajiDU5Xe+PcF+A7BdGj4OM++YBkirjobFRJ14cDI5vMm7DBWEUuSS23L4A+XVJL4wtPPI+uHJJv1WlWmDdAqThs+YC/g1JL4o1dR+A7CXpXEkc6lgJLOtMrSqRtJVF51UOhNlMtSxZ5Tl8PkzSFTGm6gMQj4Oc6ogY00vLvsfDoTH6ILoPQHA3h83OSkhec3jNz2FlztmWQ36Sc4yT+MDyibt9p8QChEU5acr27iwx/oYiUmzFq1vvzGgB6xR9l0sivRyL9q0SC5DH9lnYdFUawMboUdOeRRir3iLSwTGKWOZMZMPpxV22jQXIuyWRm8FK4M/F56pKtUCsBWCPx1fLSshRc4+uwmIAwpf+77oK6vHvpFpmF2Hp2WR7mKzeKonsu+yiWqaovl0YmTYaOAYgJFH8XcMu6n2aaVj3tqJYX7GdSL5wXKTZgl4K9xacVbSd0AXyw9N25umlibUXx+91JZGNAQjOg1ZpgHnhfix0QgnGh7/r5yQdI4m8ietC0P+fSWKKmTw/nrOBaC9TjF+QBOnGutDeMyX9pXeKgR7t5OP1KUkA2kLYScU5cvAIAt8pLuhWAkfrI6wKG1kOGw9csW4zp4dF3d+OrDf347cP7T42UhF2eljMdi5oI8sbe9urJXGOYSVErsIj3SpdIwhuJaz2reTBjlQvfXR8V8wCbUOBo0I4+yjpcO+YEGlG0Xs66NS3SCigXtf3oS3343aC+8kggLC1xoLaQpjX3lwScR855RuSoC8dI1mT2w9U/Old8+2IcqEN/Z6I+zxvIU7ks4buJyz82YjqDRD2nD9t2NISnBJJIXx3ozZNiczOkqTtfZJI5Z1TrJ0YDwlnc7vatG2KRbw5cedWktulHYCeYNWYUA7xCqWvSVhzWKeJOMXBy7ZP11i7wp8oibj1XgD5c0msGSwEb13i2HMFRLFbw7rDWli4H2ddqHF5p0mKXZD3qZr1COuSHEJAFfHlePtaCGuah/QFCGsGttUsxIQGcoQiBGOxlesh93VklhyrL8yCbx1byIbn2QLmS55LLOluOftpdanfNMU60njqwBwYD8ocwslrp1PaCMVKWFttUt96rr5eD1vkeEbkEDzLLQnNmYp+bL0hmwAyZjtwvQ6mVRA9uOaz3tJD1p6g61WRNcmawMLqhYOogGxcXpLTI5tDXggYmG5ZSOv2/SaAjDknWFcW9sW7WbRgYBlM7wCJp5SYEi5FyrOTE+Si3NZv7x/LvbtSeOs5zyaA/J/h2/QsSYxIuYSpHe4knkIYMi4QJQkuPd7cxrihWAbR9bUfnLtP6vvQlvt34aENIOwMfMGwUnaQcC3OJR5MfettuWOCLK597UcW4LP6PtTz/txMmIRgWO6k3XSdH6ENIJZbopyae5Bb9+lHMu6SeddTci5WN7XLe3OCeskoS+q9nILntRUL466t6zaAQLPC7oeFsMX40xYFjSiD9AzeU7xrFegejvv+/46wW8yjTHFY3OaUt0hiq91CoJ9iV3KPtAHE8oWCv9cyCmyIEbwTRuIWTpatEoXsSm1u/Fa6lpBYlehUeHctZBfg2wBieYJ+P0kgPLdAOMFJvofgkvNHHgUblPkrm1woDMrmJNsyRnyoSsxQ3jz04bXndp2otwEE3yIOCi2ErLeWC/6hOjENOGnow1ue4wASf6xcZzxdTeKsAD+s2JiXrvJW//0pTtnF+ujAvSysyWprIRwUcmC4R9oAYnW4RMgmc/MShEUcL8otjZWJYsYwrrNvcdaMNNT/D+HDkDt0obEFay0LSqpdh77rACFOgngJC/mEYaiuhT5EoRGNZiWlBBDFtMfy4Jf6iAolOrQUIXT21kbKEO9C3MuVsg4QKqEyC2Et4+FFOkY3i6Ah6i8hcKivHSwCxaizxGAxvKpbvXH7Gil81Pm4twKE+FwO1iwEJhTrJCgWeo0NHiohYGioHcYGjJUaJAZTDgwlFrIjt/r6CIJLhpXXbU4P3i5DoRsA7rsmyR0o1NWumH8fEjjGmoMX0NJ7NkbX2HssPXshaYfJpnUE4cSZk2cL4fzhQxYFOZXBwp0F7PERgTfEPhB/YOnW4NSsqGLxloBZvytGhkC3UwOjSSkL8rYGspP4kaiWd9+0I3/I+ghi6Rp+hDEjY3fTht/B1JJYdQLECJxhjUEQDXv9nOPkinkY3qK4J3FH4RyBMyLaziYNgXK0namkJeVTnEbD7rLkTyBPJlO21hHEMg4Eb1KG5irVAt4WgPgOUFsIh74c/rYCxNLN5GaGBzgWDa9lzNcC+0giF7qFvH6Vi2F9imUZXPQDki600LiWUS0QYQEAAlDGyo7zrXWAWLqGE4bKyWSVaoEUFmCK1cYx3LfuHXy96wCxjL5jwYevfpVqgRQWgOTQwnnyXyX94KY1iGX03TUz8mCl6JBaR1kWYJuX7d6xQtazPQFY6yNIBchY89bnc1kgCUDqFCtX99Z6x1ogyRSrLtLHdlN9PpcFkizS6zZvru6t9Y61QJJt3npQOLab6vM5LJDsoLC6muTo3lrnWAskczVZqrPi2A6qz+e1QDJnxSW5u+ft0lq7pQWSubsvJWDKsnNqWfktkCxgagkht/m7s2pgbYFkIbdzJ22w7phaXhkWSEbaMGfanzK6smrhYYFktD8oP0fiOI9OqWWWY4FkxHE0eY7Uo+V0ZdXE2gLJqUfnSF5t3Sm1vHIskJy82tLdpIT0B+V0ZdXEwwLJ0x9YJtDJnUvbo0NqmWVZ4IwIfq9YjaMS6FimYCPklp2xKtUCXhaAw8wqzV9UCjbrJJ53SpBMMtb4pGumff8oqWSmwNj2pLyPMNRbhCSXpZBxkDz1w4ZGiEriSX2WaaDxECYVdC4hyeQjQ6IV2AMbeZskyJxJDWDFaJ+rjV713kYSX9WjJEHq3AjBSSRGeqUhVe2QNpACmrRpVhKVBprKyLt3tFGtuVIF/4wkvJNJFbxNvh4o/V9k1N65FENyoKdJum5Hg0jxfbKkN2ZouCWHAmXtyufYlmGKdkLg+wKjBn8r8BV91ai8mGKGuO2/RhJJdqpclRwHlvM+8pgAlD7PjLn3+pK+KOkaYwpZefZxkl64XtYmgFj6ZFHnzyb8wpASeGji0NaviFEHTKWYMbMHziRI/Z1CmCH8lWFFTCf3JM5pyt0EEP4dAq0DjRQg1h26fW+BMJsOgvZ0qJwm6aFDH574c6+VdNyINvDO8IH61IgyYh8lHQWjloVAkdv6zmwDCMliyJ1hIeSZgGKf6ZankHkV+vqxkgrQY/W0fN7qhXuqU0bh1bYyrSI1BTuSFkIOlBPaCtoGEBI1vsqi9lDGAyRxqOMpY1OMrepWYi4+L9tZ5W5EvxQp6u4v6U2GxiCStjXB6zaAkOP8c4ZKkPqLk0ovsXTVb3RcAkgswdHYbUemWIcOf4UkPD6sZGOqjm0AoXISq5MpykLIXHRzxwM6DiQ9Ur6xz/5kCwMUWIal391q8+5sfIC3WjYHlp8NmcAsTHq2pCM3FdQFELZ62fK1kgdLIkGJhxwi6TyPgsNUEztwZjIH4WyDLU2mFh5yqCSoQD2ETLuvMywYO7DF2ypdALmXcZ469tdZ23gIp+SMUl7CaTtTLsu5r5eu28plLcgBINuaXkKeR6uUaOs6slawPK+Ch+GdQwHCcx83NCaHhWzFehjPkl1v24vDiTtAmdpowqgBMDgh95ZrS7rMoRI+gmwhc0hoITuS5bQV2DWC8AwvAznFrQQGCout2DZ9LLmFt7UXw1IXvkhTEHzRODPwHDUaO+B2YnU+sW5btpCfaWhwcr/z0dgoMQA53DidMymVD5P0NcOGNkVx1kLI8H4OZbcV+QFJL3NcV41tBvP1EyXddWxBkc9fIun24Ywi8pHo264n6VxJpK62kttJOmcsQHiexIZdTn99lCYK7Hl9HuhxL18YvjQpBc9gUkeQX6UEgUgNL+ZVD9wUeu3IMW5c4RMkPdewTJwsiX3aKjEjCAU8VpKlt+tnJDEyke7KQ/DFwuUhteDLww4LF94DKYVTZXYJufClSy24+OCL5SHXCV96ixyEjX6sxV7cpWwsQA6SxEu9d1eBPf7d2/vzLEl36KGP5a3kqnhzOAtgh8Rry5OtbXYaOQO6nyQWxzmE9GcEL3nJEO/sbbpcHhJ+XtClcCxAKOcNkhi6rYSFLqPIt60KbCnHKi3XWBXxGyIuhjk00Yyc1/SNaOSAjPMFovpYw7E9yZort/DhBKhecvUwelhuMDAVfmCMwn0AQnwA5xiWgocvu0Ge8iVJN/SsYGDZbFYQlfcNScRVc/E3gqsGrjNc/E0oqOXidKDKux77sqQbWRW2oRxmGjhSWgrnKMT/dEofgOwVvoC36iw1/gYO9u6SYL7+TcPAmvjWzftOPLNJ9e0prKs+aOhWgq6fDCPwFTGK9wEI5bGw2RV1FVPRlnu8nRibqvli/9BIXevjV1ngXxKNaNZOieiOy1D0hlNfgHC+wMk6i3ZLSeEKj75vb4s7tmzIAsoi4vA+Cdpp7dKOyizKbyuJ85oo6QsQCrU+zaTMj4aplndAFXWxtZciujGqAyZ2E2sBtvy9hYAoplZkjrKU3l4cQwByQBhF9rfUXFJv5UfUDw3RE0c8v8RHny0Jmp0U4vERJmsBo8dFfRowBCCUj7Hg3bUUtj1ZsDOFSyH44BAsVKXbAikDx3iJGT3Y1raU5wz5KA4FCPvvvMjWB1OpObRwNcBhzfOQy7KTU5fFYSuOqrgapRJLrqtGZw5uAR7nUb1kKECoBCe9R/eqLe5mfLTw1UoleAcAkjrl2mlxZgmAg1PnVIKvFT5X1vLy4LTZu9wxAAGReLNaEQevKh99kNO7xZsfwI+IqQSn1EsWTvuZfg7lFhtqO4+DaHSBQB1v5kFT9zEAoXJ4d4lrthZOv3Gl4FAnpbCNzeiF708ql/mU7dtWF1ufxHLwFY/eBjVSnsNnptcep/LwCQzm7x0LEOzDKMLi2lpSr0dW9cdrFJAQS7EEYboMOPCryiEe6w7awWJ/VCyMBUA4NCIewkNSr0fW28A+PL5AY9gGPexiVSYskvjCcQ6VS7zWHbSHeBgOhweLBUCo3IqVr60hsCV6TOP6GI2PAGGrlt7Mfeq3vhdvVsKFR708Bkox/TnJoJy2IkzYMa0AcuMwnEE25yGPN2SbH6MfLtcPCdfU/LrwnyJBK1cJ+VCg2nn+mM7Y8uznw7Qf9vdRYgUQlCDE05PEIMfO1ibj4obeAMUyFHlUZ254mNDSBhi41JcgXjtWTdsY7QmBHi2WAEEZSOGiAlEGak6EYM75cpvahLfeLSwG+SXNW04hPdr7w+YJv7so/XMqF/yriED0EgL7IKswEWuA8LKw++QZoMQLmDIZT19DE/7KqMIF1Sokap5CTA1UnIwUXJY5+6z1hs/KM78hAVwcD5h9FKwBgkGhszQZ3rb0DlMcDoCmIBykAhQIkvnlIlyAl6W5CCttE8KR+Rg0F+7agIELYnF+p2QH7yke03zLjATyAAgd7bmr1bxIGxm5p4CaNR3hfPreFcZAAPEfTtxhOcxjnSmgrQ0mu1brBXsBBCdG2DyYbngKh0AcBlUp1wIcInOY7ClMK2F3wSnRVLwAgpKAA5BYe/yuG4AFGQuzKuVZgA0bLzb/prWAAnC4rL08AUIDPBgp2l6DFOwo5b1+ZWs0i773BghdyILdKw/F6iuS2y2l7Nc1rXae7iOrLWFBzsLcTVIAhC1ftn5T0GFSD964qb2A3TpoYgXjlQs42Gr1FrZyqYetXTdJARCUJ8UViWfI7+AtuMoDkihiMG9lFlQ+p+OAw8Nlfd2M5JeBCYcUga6SCiA0As9KQOJNNtYYrE65XF+dHYWnmlJRKSSAgMPLg3xHw1IChIqPlYSLdSphygUTx6BoslRKTrgeokphiEkxpWrMROjB6alslhogtOt4SaekamAgiaYTuVLwbiVsWraq4K3iw8NlzT6yrVEnSDo1ZatzAIT2WdPZx9gMJ0dAckbMzfWejRaA8RBgWJO6dZncO11Ga/25AIIy8GrBnJFa4AIGKKkT3KRup3V9EEkDjEdZFxxRHowz8Foll5wAobEQt21NouhkETxgAQmx2J75SZzUT1osjpTE5gMOb8/ktoalJK3bVX9ugKCQd6DVtreJyDrmtAR6eaWDS/o2G1ZG2jMCj1gzWiav6aOiWeBTn0pX7y0BIOhz78zx0bB5ABQuj+y7Q/snx3N4FgMKLsucgH3bAg8ATPJZpRSAYISjQsBPToOQQ6QBCodRSxIOcRtg5M5mRbDZe0swfkkAwR54AL9L0j6ZjUM8Bllbm6tvPsHM6kdXzxYt2YCbiwCunHKZpHt6eeYOaVhpAKENR0h6bYjAG9Im62dY0K+CZepnKZxhrIIix8K7rY+IkHyopLOtO3BMeSUChPbg9EbC0BQOjn3sx9YwYCEAiMvVUa6PYh334jBKcBkX4GDLtiTB8RDWmuKcTEsFCJ1Hp3JOksJVfsjLwkhCegAuAsMgTChJmMcTSERqBy5GjhIFl3XOOYr82JQMkKYzOUEFKN6RiWNfHtYp7wuUO+Rn5yLNcwohTTS5yrmgHrp7YheQIW0kEhBgeKcBH6LbnmemABCUZfEOSLxj3EcZs+Vhcms0YDkv/H2hpK+vXCxMtwkbFtdduQ4MQDh0BRTkOJmSEB4LOFzCZC0NMRWA0GZGEEDCiDInASCrgKFtq4DIvaNnbWtGDMBhTrBgrSjlTQkgTftZkwAUT3I6D1svvUzWGADDlLfK26hTBAg2YXcL1ndPmlNv2y+pfFhnYHE3YzxMZbypAqSxD35cUOh7scqn6oe51gPLOqkrvJk23ew3dYBgGFIvAJK5rU3cOj1Rwaw1AMfoFASJ9G2tZg4AaRqGcxs5Ez3SweXso6nVDdMlOQFzJ+cxsducANIYBJAwonhk3zUx+kwLgUSbEWNwwswS7TJHgGBnyASIX2aNUvoBY4nvRR+d2K5ljQHPwOzIMeYKkKaDDw5BPwBl/z69Xu/ttMDFARgEm53fefdEb5g7QJpuOSCMJkSokZujynALkKMEUDBqXDS8mGk8uRSANL2xXwAKIwoew1XiLYCnLaDguiT+sWnfuTSANL21V8h9fowkrqn5MqV66/AlOzNcEP5dkariUupZKkBW7c+UqwFK6RlrU703uO43wGBKtVipANnZ9YevgCUXk0eulxGGlwYU5+RSorR6K0A29wjBRhBuE4V3WGkdZ6TPuSEyEiJogr6qrFmgAiTulYD+hulXE7aaIo1DnGb97oKppQkXZhoF3VGVLRaoABn2epDv5B4r17BS0jz1bknN5Z5PI02T0tVSAWJja0gQWOy3/d7ApoqNpXwl8AyzmIZUYv3Xufp5F18B4t+/+wbgQK8D7xSA4XfThUbwcm26AAT/Bh0RgLjUvwnLraECZLl9X1seYYEKkAgj1VuWa4EKkOX2fW15hAX+H2d/zQWBav7aAAAAAElFTkSuQmCC'''

# Regularly delete header information, that is, Data URI scheme
image_data = re.sub('^data:image/.+;base64,', '', img_base64)
# open image
img = Image.open(BytesIO(base64.b64decode(image_data)))

img_widht = img.size[0]
img_height = img.size[1]

# set zoom ratio
scale_width = 0.3 # 0.75
scale_height = 0.1 # 0.5

# zoom image
img = img.resize((int(img_widht*scale_width),
                 int(img_height*scale_height)), Image.NEAREST)

# output ASCII image
text = ''
for i in range(int(img_height*scale_height)):
    for j in range(int(img_widht*scale_width)):
        text += get_char(*img.getpixel((j, i)))
    text += '\n'

print(text)

Save the above code locally or copy it to Online Python Editor and run it to see the effect (Pillow needs to be installed before running)

2. Load image from URL

If it is a local application, we can use requests, a built-in Python library, to load network images. Since this online Python tool uses pyodide and does not support requests, we need to use the official API pyodide.http.pyfetch to implement the network ask

response = await pyodide.http.pyfetch('https://gcore.jsdelivr.net/gh/openHacking/static-files@main/img/16576149784751657614977527.png')

After getting the image data, you can also open it with PIL. The detailed code is as follows

from io import BytesIO
from PIL import Image
import pyodide

char = list('M3NB6Q#OC?7>!:–;. ')


def get_char(r, g, b, alpha=256):
    if alpha == 0:
        return ' '

    grey = (2126 * r + 7152 * g + 722 * b) / 10000

    char_idx = int((grey / (alpha + 1.0)) * len(char))
    return char[char_idx]


def write_file(out_file_name, content):
    with open(out_file_name, 'w') as f:
        f.write(content)

# Python API provided by pyodide for network requests, reference: https://pyodide.org/en/stable/usage/api/python-api.html#pyodide.http.pyfetch
response = await pyodide.http.pyfetch('https://gcore.jsdelivr.net/gh/openHacking/static-files@main/img/16576149784751657614977527.png')

image_data = await response.bytes()
img = Image.open(BytesIO(image_data))

img_widht = img.size[0]
img_height = img.size[1]

scale_width = 0.3 # 0.75
scale_height = 0.1 # 0.5

img = img.resize((int(img_widht*scale_width),
                 int(img_height*scale_height)), Image.NEAREST)

text = ''
for i in range(int(img_height*scale_height)):
    for j in range(int(img_widht*scale_width)):
        text += get_char(*img.getpixel((j, i)))
    text += '\n'

print(text)

Online Demo: Python Image to ASCII - URL Image

Conclusion

The above is my summary of the method of converting image to ASCII in Python, which can be run on web pages and shared with your friends. There may be some shortcomings, welcome to explore. There will be more interesting Python usages in the future, welcome to follow my updates.

Reference

Install Bun for Windows: A Modern JavaScript Runtime Like Node or Deno

openHacking — Mon, 11 Jul 2022 07:12:35 +0000

Introduction

Recently, a project called Bun in the front-end tool chain has become popular. Bun is a new JavaScript runtime with built-in bundler, transpiler, task runner and npm client.

Bun is a modern JavaScript runtime like Node or Deno. Unlike Nodejs, Bun extends jsCore, not V8. Bun natively implements hundreds of Node.js and Web APIs, including ~90% of Node-API functions (native modules), fs, path, Buffer and more.

The goal of Bun is to run most of the worlds JavaScript outside of browsers, bringing performance and complexity enhancements to your future infrastructure, as well as developer productivity through better, simpler tooling.

I plan to try installing Bun on Windows to experience it.

Environment Preparation

1.Install WSL

The official website provides a one-click installation script, but it can only run on Linux, macOS, and WSL. The Windows desktop environment cannot be installed, so if you want to install on Windows, you need to install the Linux subsystem, which is WSL.

For specific steps, please refer to Microsoft's official documentation Install Linux on Windows with WSL

It is recommended to use Windows Terminal to quickly open the WSL terminal: Microsoft Store: Windows Terminal

2.Make sure the BIOS virtual machine function is turned on

Enter the BIOS to set the virtualization technology Intel Virtual Technology, disabled by Disable to Enable enabled.

3.Enable Hyper-v

PowerShell or CMD.exe run in administrator mode

bcdedit /set hypervisorlaunchtype auto

For the details of virtual machine and Hyper-v settings, you can also refer to this blog post: "VMware Workstation Unrecoverable Error: (vcpu-1)" Troubleshooting for Vmware Workstation16

Install Bun

1.Make sure that the WSL system (mine is Ubuntu) has unzip installed, otherwise an error will be reported when installing Bun

error: unzip is required to install Bun (see: https://github.com/Jarred-Sumner/bun#unzip-is-required

Use Terminal to open WSL, Ubuntu system install unzip with the following command

sudo apt-get install unzip

2.Enter under WSL

 curl https://bun.sh/install | bash

The following prompt appears, the installation is successful

Bun was installed successfully to /root/.bun/bin/bun

Manually add the directory to your $HOME/.bashrc (or similar)

   BUN_INSTALL="/root/.bun"
   PATH="$BUN_INSTALL/bin:$PATH"

3.Manually add environment variables as prompted

Open the .bashrc file with vim

vim /root/.bashrc

Add the following environment variable settings to the end of the .bashrc file and save it

BUN_INSTALL="/root/.bun"
PATH="$BUN_INSTALL/bin:$PATH"

Exit and re-enter WSL to check if the installation is successful

bun -h

4.Test Module

Use the WSL extension for vscode to manage projects in a WSL system

Official detailed tutorial Get started using VSCode with WSL

We create a new http.js file and write the code

// http.js
export default {
  port: 3000,
  fetch(request) {
    return new Response("Welcome to Bun!");
  },
};

start http service

bun run http.js

Visit http://localhost:3000 to see the Welcome to Bun! message, indicating that the operation is successful.

The official website also provides more cases to try: Bun Examples

Conclusion

After experience, the use of this tool needs to be improved, especially for Windows users, it is not easy to install, it is best to provide a Windows installation package. It is currently only a beta version, which can be used for learning. There may still be a long way to go before it can be used in production. After all, Node.js has been well known by front-end development, and the ecology of Deno is still being established. There is still a long way to go.

Reference

Share a Useful Python Online Editor

openHacking — Fri, 13 May 2022 09:31:10 +0000

Original: https://lwebapp.com/en/python-online

Introduction

Some friends may have heard of PyScript, and know that Python can be packaged into wasm and run on the browser side. To do some functions that require Python to do, can be done directly in the browser without interacting with the server. Open the developer imagination.

What we want to recommend here is an online tool that also supports the execution of Python code, an online Python code editor

Python online editor address: https://lwebapp.com/en/python-playground

How to use Python Playground tool?

Copy the Python code from other places, paste it into the editing area on the left side of the Python Playground tool, or directly enter your Python code
Click the Run button at the top to execute the code
The right area will immediately display the running results of the Python code, click the Clear button to clear the running results
You can copy or download your Python code

What is Python?

Python was designed in the early 1990s by Guido van Rossum of the Netherlands Society for Mathematics and Computer Science as an alternative to a language called ABC. Python provides efficient high-level data structures, as well as simple and efficient object-oriented programming. Python syntax and dynamic typing, as well as the nature of an interpreted language, make it a programming language for scripting and rapid application development on most platforms. development of the project.

Why learn Python?

Python syntax is simple and easy to learn
Python lets you build more functionality with less code
Python is cross-platform and open source
Python can do web backend, web crawler, office automation, data analysis, artificial intelligence, automated testing, desktop applications, etc.
Rich third-party libraries to meet almost all your development needs
Active development community, detailed development documents, guides, tutorials

Why use Python Playground?

Open the browser to write Python code, just use it and leave without installing a local Python environment
Run Python code on the browser side, no need to interact with the server, and the response is fast
Help beginners learn Python syntax quickly
Support Python 3.10
Support to quickly copy or download your code

Conclusion

The above is the my introduction to the Python online editor launched by the lwebapp.com website. This tool is still under iteration, and your valuable comments are welcome.

Reference

2Captcha Review 2022 | Principles & Usage & Getting Started

openHacking — Thu, 14 Apr 2022 12:01:05 +0000

Original: https://lwebapp.com/en/post/2captcha-review

Introduction

You are no stranger to web page captcha. Almost all web pages or apps with a login function have a captcha function.

The captcha does not necessarily appear every time you log in, but when you fail to log in multiple times, you will be asked to enter the captcha question answer, mainly to prevent you from maliciously logging into other people's accounts. Or some websites are more strict, and a captcha window will pop up every time. This captcha is usually used to prevent robots from logging in. Just like the previous 12306 train ticket website, the captcha will pop up every time you buy a ticket, and it is difficult to verify. It does block the script, but it also blocks our normal users.

Question

As a normal user, I don’t really want to make a captcha that is too difficult. Every manual click is very annoying. Especially when some developers are doing automated testing and script login, the problem of captcha recognition is more prominent. Many free captcha recognition libraries have been found on the Internet, and it is difficult to find a way to easily bypass the captcha.

I have studied captcha identification before, and found a captcha identification service provider 2Captcha, and tried their service, the effect is very good.

Next, I will talk about my experience with 2Captcha.

2Captcha Introduction

According to the introduction on the official website, 2Captcha is an automatic captcha recognition service provider that supports many types of captcha, such as recognizing distorted text, selecting eligible images, reCAPTCHA click I am not a robot, etc.

2Captcha aims to connect customers who need to recognize many captchas in real time and those who make money by recognizing captcha.

2Captcha official website: 2captcha.com

Why use 2Captcha

Just like the application scenarios mentioned above, there are two groups of people who will have this demand.

If you want to do automated testing and use scripts to log in to the website, you should use the 2Captcha captcha identification service at this time. As long as you request the interface provided by the official, you can get the parameters for the successful verification of the captcha, and quickly bypass the captcha.
Or if you want to make money online, 2Captcha provides a background for manual verification of captcha. You only need to click on each captcha to reach a certain number, and then you can make money. It is especially suitable for students, full-time mothers, and idle employees. They can use their spare time to make pocket money. The official provides a variety of payment methods.

So 2Captcha connects these two types of customers and implements a closed-loop solution to the problem of captcha identification.

2Captcha development experience

I am engaged in development work, so by reading the API documentation of 2Captcha, the captcha recognition function is implemented in the automated script.

2Captcha's API Documentation

Simply put, according to the interface provided by 2Captcha, send your image or verification code parameters to their server, get the ID of the task, and then check the verification result in a loop.

2Captcha's interface documentation is very detailed, and provides complete operation steps for each type of captcha. Basically, following the tutorial is the expected result, including input parameters, return parameters, error codes, etc.

All supported captcha types and rates are listed here

CAPTCHA TYPE	RATE PER 1000 PEOPLE	DESCRIPTION
Normal Captcha, Text Captcha	$0.5-$1	The rate is flexible and depends on current load of the service. You can see current rate and limit max rate in your account settings.
reCAPTCHA V2	$2.99	The rate applies to Google's reCAPTCHA V2 solved via token.
reCAPTCHA V3	$1.45, $2.99	score <= 0.3, score > 0.3
reCAPTCHA Enterprise	$2.99	The rate applies to Google's reCAPTCHA Enterprise.
Geetest	$2.99	The rate applies to Geetest captcha.
Grid Method, Coordinates	$1.20	The rate applies to any captcha where you need to click images.
RotateCaptcha	$0.50	The rate applies to any captcha where you need to rotate images.
Arkose Labs FunCaptcha Token Method	$2.99	The rate applies to Arkose Labs FunCaptcha solved via token.
KeyCaptcha	$2.99	The rate applies to KeyCaptcha.
hCaptcha	$2.99	The rate applies to hCaptcha.
Capy	$2.99	The rate applies to Capy.
TikTok	$2.99	The rate applies to TikTok captcha.

The official website also provides complete code examples in multiple development languages, including Go, Python, PHP, Java, C#, C++, etc.

In order to realize the automatic login function of bilibili.com, in most cases, I just need to read the official API documentation and simply implement a feasible case with Node.js

For specific technical tutorials, please refer to this How to Bypass Captcha Automatic Login with Nodejs Playwright 2Captcha

So if you are a developer, please read the official website API documentation carefully.

2Captcha Advantages

Some friends will also have questions, why not use a free captcha recognition library? What are the advantages of 2Captcha?

If you are doing the captcha recognition yourself, the general solution is to do a lot of image recognition training based on machine learning such as TensorFlow library to improve the accuracy, or directly use pytesseract for OCR recognition. The limitation of this type of scheme is that it can only recognize simple pictures and text, and a little more complex dynamic pictures and reCAPTCHA clicks can't be helped.

In the final analysis, the idea of machine recognition can only solve simple scenarios. To truly bypass the captcha perfectly, it still depends on manual work. Therefore, 2Captcha is based on this idea. On the one hand, the developer sends a request to identify the captcha, and on the other hand, it manually clicks for you, providing a two-way service to solve the complex captcha problem.

According to my development experience, in addition to the very simple interface development, the recognition success rate of 2Captcha is still very high. If you have any questions, you can also provide feedback according to the contact information provided on the official website.

Conclusion

The above is a summary of the experience of using the 2Captcha captcha verification interface based on my own needs for automatic captcha identification. Overall, it is highly recommended to try 2Captcha to improve work efficiency and save time. Try 2Captcha ➜

Reference

How to Bypass Captcha Automatic Login with Nodejs Playwright 2Captcha

openHacking — Wed, 13 Apr 2022 06:08:07 +0000

Original: https://lwebapp.com/en/post/regular-expression-to-match-multiple-lines-of-text

Question

In our daily work, in order to improve work efficiency, we may write scripts to automate tasks. Because some websites require users to log in, the automatic login function of the script is essential.

However, when we log in to the website, we often see verification codes. The purpose of verification codes is to prevent machine logins and automate script operations. Is there a way for scripts to automatically identify verification codes to achieve login?

Next, I will use bilibili.com as an example to explain to you how to solve the most critical verification code problem in the automatic login script.

Explore

First of all, you need to experience the login method of this website and understand its verification code type.

Open https://www.bilibili.com/, open the console, click login, then a small login box will pop up in the middle, usually after entering the account and password, the verification code box will pop up, we guess the verification code interface has been requested at this time.

Since the English of the verification code is captcha, we search for captcha in the network panel

An interface related to verification code was found



https://passport.bilibili.com/x/passport-login/captcha

Click on the interface to see the results, and there is some useful information, we found that the captcha type is geetest.



{
  "code": 0,
  "message": "0",
  "ttl": 1,
  "data": {
    "type": "geetest",
    "token": "b416c387953540608bb5da384b4e372b",
    "geetest": {
      "challenge": "aeb4653fb336f5dcd63baecb0d51a1f3",
      "gt": "ac597a4506fee079629df5d8b66dd4fe"
    },
    "tencent": {
      "appid": ""
    }
  }
}

Through searching, I found that the verification code service used by bilibili.com is provided by geetest, which is used by many websites. The feature of geetest verification code is to move puzzles and select words or numbers in order.

So next, let's find a way to recognize the geetest verification code.

I learned about the verification code solutions provided on the market, and the most effective ones are basically OCR service providers. After comparison, I found that the service of 2Captcha is very good, with fast decoding speed, stable server connection, multi-language API support, and reasonable price, I decided to try 2Captcha.

2Captcha official website

Next, we will show the use of Nodejs + Playwright + 2Captcha to solve the login verification code problem at bilibili.com.

If you want to use other languages and frameworks, such as Python + Selenium, you can also refer to this tutorial, the idea of solving the problem is the same.

Solution

How to identify the verification code

First read the official document 2Captcha API Geetest, the solution is very detailed, simply put

By intercepting the website interface, get the two verification code parameters gt and challenge, request http://2captcha.com/in.php, and get the verification code ID
Request http://2captcha.com/res.php after a period of time, and get the challenge, validate, seccode of successful verification

How to apply verification results

After getting the most critical validate, simulate the user to fill in the account and password to log in, intercept the return parameters of the verification code request interface, replace them with the parameters of successful verification, and then trigger the login interface.

Next, we analyze the detailed steps

Environment

Let's build the script execution environment first.

We use Node.js + Playwright for scripting.

Make sure that Nodejs has been installed locally on your computer
Create a new empty project and install Playwright



mkdir bypass-captcha
cd bypass-captcha
npm init
npm i -D playwright

We adopt Playwright's library mode, detailed documentation: Playwright

Create a new script file captcha.js in the project root directory, fill in the following content, run node captcha.js on the command line to simply test whether the project can be started normally



const { chromium } = require("playwright");

(async () => {
  const browser = await chromium.launch({
    headless: false,
  });
  const page = await browser.newPage();
  await page.goto("https://www.bilibili.com/");

  await browser.close();
})();

Under normal circumstances, a Google browser interface will pop up, displaying the home page of bilibili.com, and then the browser will automatically close.

Request `in.php` interface

First, sort out the parameters required to request the http://2captcha.com/in.php interface. You can see the list of parameters. We will pay attention to the parameters that must be passed.

Parameter	Type	Required	Description
key	String	Yes	your API key
method	String	Yes	geetest - defines that you're sending a Geetest captcha
gt	String	Yes	Value of gt parameter you found on target website
challenge	String	Yes	Value of challenge parameter you found on target website
api_server	String	No	Value of api_server parameter you found on target website
pageurl	String	Yes	Full URL of the page where you see Geetest captcha
header_acao	IntegerDefault: 0	No	0 - disabled1 - enabled.If enabled in.php will include Access-Control-Allow-Origin:* header in the response. Used for cross-domain AJAX requests in web applications. Also supported by res.php.
pingback	String	No	URL for pingback (callback) response that will be sent when captcha is solved.URL should be registered on the server. More info here.
json	IntegerDefault: 0	No	0 - server will send the response as plain text1 - tells the server to send the response as JSON
soft_id	Integer	No	ID of software developer. Developers who integrated their software with 2captcha get reward: 10% of spendings of their software users.
proxy	String	No	Format: login:password@123.123.123.123:3128 You can find more info about proxies here.
proxytype	String	No	Type of your proxy: HTTP, HTTPS, SOCKS4, SOCKS5.
userAgent	String	No	Your userAgent that will be passed to our worker and used to solve the captcha.

key needs to be registered on the 2Captcha official website, and there is an API key in the account settings of the dashboard. Need to recharge a certain amount
method is a fixed value geetest
gt and challenge have been seen before in the interface of the website login page. However, there is a note here, gt is only one value per website, the gt value of bilibili.com is ac597a4506fee079629df5d8b66dd4fe, but challenge is a dynamic value, each API request will get a new challenge value . Once the captcha is loaded on the page, the challenge value becomes invalid. So you need to listen to the request https://passport.bilibili.com/x/passport-login/captcha, when the website login page loads, and re-identify the new challenge value each time. The following will explain how to listen.
pageurl is the address of the login page https://www.bilibili.com/

So we can get a request interface like this



http://2captcha.com/in.php?key=1abc234de56fab7c89012d34e56fa7b8&method=geetest&gt=ac597a4506fee079629df5d8b66dd4fe&challenge=12345678abc90123d45678ef90123a456b&pageurl=https://www.bilibilicom/

Next, solve the problem of getting a new challenge value every time you enter the home page

The process of simulating user click login

Start Google Chrome first and open the home page of bilibili.com
Click the login button at the top, a login box will pop up
At this time, the verification code interface has been sent, and you can intercept the values of gt and challenge by listening to the response returned by the verification code interface.



const { chromium } = require("playwright");

(async () => {
  // Select the Chrome browser, set headless: false to see the browser interface
  const browser = await chromium.launch({
    headless: false,
  });

  const page = await browser.newPage();

  // open bilibili.com
  await page.goto("https://www.bilibili.com/");

  const [response] = await Promise.all([
    // request verification code interface
    page.waitForResponse(
      (response) =>
        response.url().includes("/x/passport-login/captcha") &&
        response.status() === 200
    ),
    // Click the login button at the top
    page.click(".header-login-entry"),
  ]);

  // Get the interface response information
  const responseJson = await response.body();

  // Parse out gt and challenge
  const json = JSON.parse(responseJson);
  const gt = json.data.geetest.gt;
  const challenge = json.data.geetest.challenge;

  console.log("get gt", gt, "challenge", challenge);

  // Pause for 5 seconds to prevent the browser from closing too fast to see the effect
  sleep(5000);

  // close the browser
  await browser.close();
})();

/**
 * Simulate the sleep function, delay for a number of milliseconds
 */
function sleep(delay) {
  var start = new Date().getTime();
  while (new Date().getTime() < start + delay);
}

Use the request library to request the in.php interface separately

Install request first



npm i request

Now it is time to request the http://2captcha.com/in.php interface



// request in.php interface
const inData = {
  key: API_KEY,
  method: METHOD,
  gt: gt,
  challenge: challenge,
  pageurl: PAGE_URL,
  json: 1,
};

request.post(
  "http://2captcha.com/in.php",
  { json: inData },
  function (error, response, body) {
    if (!error && response.statusCode == 200) {
      console.log("response", body);
    }
  }
);

Under normal circumstances, the verification code ID will be returned at this time, such as {"status":1,"request":"2122988149"}, just take the request field.

If the interface returns the code ERROR_ZERO_BALANCE, it means that your account balance is insufficient and you need to recharge. I have recharged the minimum amount here for demonstration, and you can experience it according to your own needs.

Extended Learning

In order to improve security, we refer to the API Key in the environment variable file.

Create a new environment variable file .env in the root directory and write the value of API Key



# .env file
API_KEY="d34y92u74en96yu6530t5p2i2oe3oqy9"

Then install the dotenv library to get the environment variables



npm i dotenv

Use it in js



require("dotenv").config();

In this way, the variables in .env can be obtained through process.env.API_KEY. Usually .env files are not uploaded to the code repository to ensure the security of personal information.

If you don’t want to write the information to the file while ensuring security, you can also directly enter the Node.js environment variable in the console, such as



API_KEY=d34y92u74en96yu6530t5p2i2oe3oqy9 node captcha.js

Request `res.php` interface

Before requesting the interface, we also sort out the required parameters


GET parameter	Type	Required	Description
key	String	Yes	your API key
action	String	Yes	get - get the asnwer for your captcha
id	Integer	Yes	ID of captcha returned by in.php.
json	IntegerDefault: 1	No	Server will alsways return the response as JSON for Geetest captcha.

key is API_KEY, which is also used in the previous interface
action is fixed value get
id is the captcha ID just returned by in.php

20 seconds after the last request, request the http://2captcha.com/res.php interface to get the verification result



request.get(
  `http://2captcha.com/res.php?key=${API_KEY}&action=get&id=${ID}&json=1`,
  function (error, response, body) {
    if (!error && response.statusCode == 200) {
      const data = JSON.parse(body);
      if (data.status == 1) {
        console.log(data.request);
      }
    }
  }
);

The interface will return three values challenge, validate and seccode, each parameter is a string



{
  "geetest_challenge": "aeb4653fb336f5dcd63baecb0d51a1f3",
  "geetest_validate": "9f36e8f3a928a7d382dad8f6c1b10429",
  "geetest_seccode": "9f36e8f3a928a7d382dad8f6c1b10429|jordan"
}

Among them, challenge is the parameter we intercepted earlier, validate is the verification result identifier, and the content of seccode is basically the same as that of validate, with only one more word. We need to store validate for later use.

Sometimes the verification code cannot be verified here. You can try several times, or contact 2Captcha official website to troubleshoot the problem

At this point, the information of the verification code verification result has been obtained, and the next step is to log in with the verification result.

Login

Let’s first study the login process after a normal user clicks on the verification code to verify the success

We found three interfaces

https://api.geetest.com/ajax.php: verification code interface, used to generate verification code and verify whether the verification code is passed. The validate field in the data returned by the validation interface is the geetest_validate obtained by the 2Captcha service.
https://passport.bilibili.com/x/passport-login/web/key?_=1649087831803: password encryption interface, used to obtain hash and public key
https://passport.bilibili.com/x/passport-login/web/login: login interface, input parameters include account, password, token, challenge, validate and seccode, etc.

We analyze these interfaces, two login schemes are available.

The first solution is to request the encryption interface and the login interface in theNode.js environment to obtain the user's cookie information, and the user can log in directly with the cookie information. The difficulty of this scheme is that it needs to deal with password encryption separately, which is not very friendly to beginner.
The second solution is to use Playwright to simulate the user to fill in the account and password to log in, randomly click the verification code to trigger the login, intercept the response parameter of the verification code interface, replace it with the successful verification code, and then trigger the login interface.

We take the second solution.

But I also encountered difficulties, in the Node.js environment, the verification code image could not be loaded. Then, I found the verification code interface https://api.geetest.com/ajax.php is also responsible for pulling the verification code image and verifying verification code. We directly intercept the request when pulling the verification code image, and replace the verification result to trigger the login, without waiting for the image verification code to come out. This detail is critical.

Conclusion

The above is some research on common automatic login functions in automated testing tasks. Combine the strengths of Node.js, Playwright, and 2Captcha, the verification code recognition is realized. I have uploaded the complete code to GitHub.

https://github.com/openHacking/bypass-captcha

There may be many places to be optimized, and you are welcome to point out.

Disclaimer: This script is only used as a test and learning case, and the risk is self-assessed.

Reference

pip Install Third-party Library Error

openHacking — Mon, 28 Mar 2022 03:49:13 +0000

Original: https://lwebapp.com/en/post/pip-install-error-guide

Question

When we use Python to develop, we usually use some useful third-party libraries. It is recommended to use pip to install, such as installing pandas

python -m pip install pandas

Under normal circumstances, there is no problem, but some small partners will encounter installation failures such as pip installation of third-party libraries and errors, pip install stuck and other installation failures.

For example, the following is the code that pip installs a third-party library to report an error

Retrying (Retry(total=1, connect=None, read=None, redirect=None, status=None))...

I will summarize here the solutions for the failure of Python to install third-party libraries.

Solution

Option 1

The Python environment factor reports an error, check whether Python and pip under your computer are installed normally

# Print the current Python version
python --version

If the Python version information is printed out normally after running the above command, it means that it is normal. If an error is reported, it may be that the environment variables are not configured when installing Python under Windows. You can configure the Python environment variables, or directly reinstall Python to the C drive, so that there will be no problem with environment variables.

# print pip version
python -m pip --version

If the pip version information is printed out normally after running the above command, it means that it is normal. If an error is reported, you can use this command to install and upgrade

python -m ensurepip --upgrade

If you can't run pip yet, you can also install pip manually

Open https://bootstrap.pypa.io/get-pip.py
Right click on the page - save as - save anywhere
Execute python get-pip.py on the command line of the directory where the get-pip.py file is located

Another advantage of manually installing pip is that you can directly use global pip to install dependent packages, such as

pip install pandas

Option 2

Although pip is installed normally, sometimes it will prompt you that the version of pip is too low

WARNING: You are using pip version 22.0.3; however, version 22.0.4 is available.
You should consider upgrading via the '/usr/local/bin/python -m pip install --upgrade pip' command.

Run the following command to upgrade pip

python -m pip install --upgrade pip

Option 3

The third-party package name or version number is entered incorrectly. For example, I install a package called padas in the command line terminal

python -m pip install padas

The following error will appear

ERROR: Could not find a version that satisfies the requirement padas (from versions: none)
ERROR: No matching distribution found for padas

The error tells me that I typed padas, that the package and its version number cannot be found. At this time, you need to check whether the package name is entered correctly, the correct one should be pandas. (Of course, after the article is published, some developers may have released this package, which is only for demonstration)

There is also a problem that the version number may not exist. For example, I install a higher version of pandas

python -m pip install pandas==6.5

The following error will appear

ERROR: Could not find a version that satisfies the requirement pandas==6.5 (from versions: 0.1, 0.2, 0.3.0, 0.4.0, 0.4.1, 0.4.2, 0.4.3, 0.5.0, 0.6.0 , 0.6.1, 0.7.0, 0.7.1, 0.7.2, 0.7.3, 0.8.0, 0.8.1, 0.9.0, 0.9.1, 0.10.0, 0.10.1, 0.11.0, 0.12 .0, 0.13.0, 0.13.1, 0.14.0, 0.14.1, 0.15.0, 0.15.1, 0.15.2, 0.16.0, 0.16.1, 0.16.2, 0.17.0, 0.17.1 , 0.18.0, 0.18.1, 0.19.0, 0.19.1, 0.19.2, 0.20.0, 0.20.1, 0.20.2, 0.20.3, 0.21.0, 0.21.1, 0.22.0, 0.23 .0, 0.23.1, 0.23.2, 0.23.3, 0.23.4, 0.24.0, 0.24.1, 0.24.2, 0.25.0, 0.25.1, 0.25.2, 0.25.3, 1.0.0 , 1.0.1, 1.0.2, 1.0.3, 1.0.4, 1.0.5, 1.1.0, 1.1.1, 1.1.2, 1.1.3, 1.1.4, 1.1.5, 1.2.0, 1.2 .1, 1.2.2, 1.2.3, 1.2.4, 1.2.5, 1.3.0, 1.3.1, 1.3.2, 1.3.3, 1.3.4, 1.3.5, 1.4.0rc0, 1.4.0 , 1.4.1)
ERROR: No matching distribution found for pandas==6.5

Obviously, this version number cannot be found, and it tells you all the version numbers that can be installed. We only need to select a version number that we need, or install the latest version by default without specifying a version number.

Option 4

The network quality and communication speed are different in different place, and the pip installation of dependencies will also encounter network timeout problems, such as the following errors

raise ReadTimeoutError(self._pool, None, 'Read timed out.')
pip._vendor.requests.packages.urllib3.exceptions.ReadTimeoutError: HTTPSConnectionPool(host='pypi.org', port=443): Read timed out.

It indicates that the network has timed out, and there is a problem connecting to pypi.org. This is the website hosting the python dependencies, and all pip packages are published on it.

We can set a longer timeout, because the network in most places is not completely unconnectable, but the speed is a bit slow. Here the --default-timeout is set to 200s

python -m pip --default-timeout=200 install pandas

Set a little more timeout, go have a cup of tea and wait for a while, and it may be downloaded.

Option 5

If you haven't downloaded it after drinking tea, then consider changing the mirror source. For example, we switch to the mirror source of Tsinghua University.

# --index-url can be abbreviated as -i
python -m pip install --index-url https://pypi.tuna.tsinghua.edu.cn/simple/ pandas

In addition to the official source and Tsinghua mirror source

Tsinghua: https://pypi.tuna.tsinghua.edu.cn/simple/
Official: https://pypi.org/

There are other mirror sources you can try

Aliyun: https://mirrors.aliyun.com/pypi/simple/
Douban: https://pypi.douban.com/simple/
Beijing Foreign Studies University https://mirrors.bfsu.edu.cn/pypi/web/simple/

Option 6

In addition to switching mirror sources, pip officially provides a variety of ways to install dependent packages

Download the dependency package in advance, and then install it directly from the local path

For example, we can download the pandas distribution package file from the pip official website, and then execute the installation command in the directory where the local dependency package is located.

Install from Source Distribution

python -m pip install pandas-1.4.1.tar.gz

or install from Built Distributions

# Buildpacks for Windows platforms only
python -m pip install pandas-1.4.1-cp310-cp310-win_amd64.whl

List of all pandas distributions: pandas download files

To install from any VCS (version control systems), use the following command format

python -m pip install -e "vcs+protocol://repo_url/#egg=pkg&subdirectory=pkg_dir"

vcs: version control system name
protocol: protocol
repo_url: repository url
egg: package name
subdirectory: If the package is not in the project root directory, specify the subdirectory name

For example, install the pandas library directly from github

python -m pip install git+https://github.com/pandas-dev/pandas.git#egg=pandas

This method will pull the latest code from github for local construction, which takes a long time, and is usually the development version.

pip supports a lot of version control systems and protocols, see VCS Support

Installation from github also supports manual local installation, clone the github repository of the dependent package directly, and execute it in the project directory

python setup.py install

In this way, the library built by this project is directly installed. In essence, it is similar to the remote method of using github. It needs to be compiled locally. It is usually used in the local development stage, or if you want to try the latest features of the project.

However, if you have network problems through pip install, this way of installing through github usually also has certain network problems.

Option 7

I have tried several of the above solutions, but none of them can perfectly meet my needs.

Although a long timeout is set, sometimes the network is very slow, and no matter how long the timeout is, it is a waste of time.
Compared with the official website, the mirror source has a slightly delayed synchronization time. The official pypi.org dependency package is the most stable. The same problem is also common in the npm package management of front-end development. The mirror source sometimes has unpredictable errors, and it is often fixed by switching to the official source (refer to NPM Install Error Guide).
If you directly download the source package to build, because many Python libraries are written by foreigners, the website is not in your country, the download speed is very slow when visiting other countries' websites, downloading from github is just as slow (refer to GitHub Clone Is Very Slow).

We can consider some more scientific ways to surf the Internet to speed up access to the official website. After the acceleration, directly use pip install to install any third-party library. Basically, it can be completed in a few seconds. There is no need to set a timeout period, no need to switch mirror sources, and no need to worry about the delay of the installation package version. If you want to download the source package from the pip official website or installing Python third-party libraries remotely using github is very fast.

In addition, there are the following benefits

When you access some Python third-party library documents, you can also speed up access, such as pandas official website https://pandas.pydata.org/
When you go to github to learn the source code of open source projects, and use git clone to pull github open source projects, the speed is significantly improved
The technical documents searched by Google are often more accurate, and the top of the search results are the highly praised answers on stackoverflow, which are very useful
Visit some other great tech sites

Recommend a more scientific way on this Official Website ➜

If you don't understand, please refer to my Hard-earned Experience ➜

Reference

LeetCode Notes: Find All Duplicates in an Array

openHacking — Sat, 12 Mar 2022 05:01:43 +0000

Question

Given an integer array nums of length n where all the integers of nums are in the range [1, n] and each integer appears once or twice, return an array of all the integers that appears twice.

You must write an algorithm that runs in O(n) time and uses only constant extra space.

Example 1:

Input: nums = [4,3,2,7,8,2,3,1]

Output: [2,3]

Example 2:

Input: nums = [1,1,2]

Output: [1]

Example 3:

Input: nums = [1]

Output: []

Constraints:

n == nums.length
1 <= n <= 105
1 <= nums[i] <= n
Each element in nums appears once or twice.

Solution One

Analysis:

Use the unique feature of Set to continuously add the numbers in nums to an empty Set, and then use the set.add method to determine whether there are repeated numbers by getting the length of set to increase.

Code:

/**
 * @param {number[]} nums
 * @return {number[]}
 */
var findDuplicates = function (nums) {
  const set = new Set(); // unique value test
  const result = []; // result array

  nums.forEach((n) => {
    const preSize = set.size;

    // Use the set.add method to determine whether there are duplicate numbers by getting the length of the set to increase
    set.add(n);

    // find duplicate numbers
    if (preSize === set.size) {
      result.push(n);
    }
  });

  return result;
};

Solution Two

Analysis:

Traverse the entire array, treat each number as the array position information, and then reverse the number corresponding to each position to a negative number, which is equivalent to making a mark, indicating that the position corresponding to this number is already occupied by a number, and we will meet again next time If this number is found to be negative, it means that it has already appeared.

For example [4,3,2,7,8,2,3,1], when reaching the first 2, the number 3 whose position is 1 is flipped to -3, go By the next 2, the number -3 in position 1 can be found, which has been flipped, indicating that the number 2 occurs twice.

Code:

/**
  * @param {number[]} nums
  * @return {number[]}
  */
var findDuplicates = function(nums) {
     let result = [];
     for (let i = 0; i < nums.length; i++) {
         let num = Math.abs(nums[i]);
         if (nums[num - 1] > 0) {
             /**
              The purpose of flipping the number to a negative number is to make a mark to indicate that the position corresponding to the number is already occupied by a number. If the number is found to be a negative number next time, it means that it has already appeared.

              For example [4,3,2,7,8,2,3,1]

              When you go to the first 2, the number in position 1 is 3, flip the 3 to -3, and when you go to the next 2, when you flip 3, you find that it has been flipped.
              */
             nums[num - 1] *= -1;
         } else {
             result.push(num);
         }
     }
     return result;

};

Check more LeetCode Notes:https://lwebapp.com/en/tag/leetcode

Reference

How to Implement an Event Bus in JavaScript

openHacking — Thu, 10 Mar 2022 10:07:42 +0000

Introduction

The Event Bus is usually used as a communication mechanism between multiple modules, which is equivalent to an event management center. One module sends messages, and other modules receive messages, which achieves the function of communication.

For example, data passing between Vue components can be communicated using an Event Bus, or it can be used as plugin and core communication in the microkernel plugin system.

Principle

Event Bus essentially adopts a publish-subscribe design pattern. For example, multiple modules A, B, and C subscribe to an event EventX, and then a certain module X publishes this event on the event bus, then the event bus will be responsible for notifying all subscriptions. A, B, C, they can all receive this notification message, and can also pass parameters.

// relation chart
                          module X
                            ⬇ Release EventX
╔════════════════════════════════════════════════════════════════════╗
║                         Event Bus                                  ║
║                                                                    ║
║         【EventX】       【EventY】       【EventZ】   ...           ║
╚════════════════════════════════════════════════════════════════════╝
  ⬆Subscribe to EventX   ⬆Subscribe to EventX   ⬆Subscribe to EventX
Module A                Module B                Module C

Analysis

How to implement a simple version of Event Bus using JavaScript

First construct an EventBus class, initialize an empty object to store all events
When accepting a subscription, use the event name as the key value, and use the callback function that needs to be executed after accepting the published message as the value. Since an event may have multiple subscribers, the callback function here should be stored as a list
When publishing an event message, get all the callback functions corresponding to the specified event name from the event list, and trigger and execute them in sequence

The following is the detailed implementation of the code, which can be copied to the Google Chrome console to run the detection effect directly.

Code

class EventBus {
  constructor() {
    // initialize event list
    this.eventObject = {};
  }
  // publish event
  publish(eventName) {
    // Get all the callback functions of the current event
    const callbackList = this.eventObject[eventName];

    if (!callbackList) return console.warn(eventName + " not found!");

    // execute each callback function
    for (let callback of callbackList) {
      callback();
    }
  }
  // Subscribe to events
  subscribe(eventName, callback) {
    // initialize this event
    if (!this.eventObject[eventName]) {
      this.eventObject[eventName] = [];
    }

    // store the callback function of the subscriber
    this.eventObject[eventName].push(callback);
  }
}

// test
const eventBus = new EventBus();

// Subscribe to event eventX
eventBus.subscribe("eventX", () => {
  console.log("Module A");
});
eventBus.subscribe("eventX", () => {
  console.log("Module B");
});
eventBus.subscribe("eventX", () => {
  console.log("Module C");
});

// publish event eventX
eventBus.publish("eventX");

// output
> Module A
> Module B
> Module C

We have implemented the most basic publish and subscribe functions above. In practical applications, there may be more advanced requirements.

Advanced

1. How to pass parameters when sending a message

The publisher passes a parameter into EventBus, and then passes the parameter when the callback function is executed, so that each subscriber can receive the parameter.

Code

class EventBus {
  constructor() {
    // initialize event list
    this.eventObject = {};
  }
  // publish event
  publish(eventName, ...args) {
    // Get all the callback functions of the current event
    const callbackList = this.eventObject[eventName];

    if (!callbackList) return console.warn(eventName + " not found!");

    // execute each callback function
    for (let callback of callbackList) {
      // pass parameters when executing
      callback(...args);
    }
  }
  // Subscribe to events
  subscribe(eventName, callback) {
    // initialize this event
    if (!this.eventObject[eventName]) {
      this.eventObject[eventName] = [];
    }

    // store the callback function of the subscriber
    this.eventObject[eventName].push(callback);
  }
}

// test
const eventBus = new EventBus();

// Subscribe to event eventX
eventBus.subscribe("eventX", (obj, num) => {
  console.log("Module A", obj, num);
});
eventBus.subscribe("eventX", (obj, num) => {
  console.log("Module B", obj, num);
});
eventBus.subscribe("eventX", (obj, num) => {
  console.log("Module C", obj, num);
});

// publish event eventX
eventBus.publish("eventX", { msg: "EventX published!" }, 1);

// output
> Module A {msg: 'EventX published!'} 1
> Module B {msg: 'EventX published!'} 1
> Module C {msg: 'EventX published!'} 1

2. How to unsubscribe after subscription

Sometimes subscribers only want to subscribe to messages in a certain period of time, which involves the ability to unsubscribe. We will revamp the code.

First of all, to achieve the specified subscriber unsubscribe, each time an event is subscribed, a unique unsubscribe function is generated. The user directly calls this function, and we delete the currently subscribed callback function.

// Every time you subscribe to an event, a unique unsubscribe function is generated
const unSubscribe = () => {
  // clear the callback function of this subscriber
  delete this.eventObject[eventName][id];
};

Secondly, the subscribed callback function list is stored in an object structure, and a unique id is set for each callback function. When canceling the callback function, the efficiency of deletion can be improved. If you still use an array, you need to use split to delete, which is less efficient than delete of objects.

Code

class EventBus {
  constructor() {
    // initialize event list
    this.eventObject = {};
    // id of the callback function list
    this.callbackId = 0;
  }
  // publish event
  publish(eventName, ...args) {
    // Get all the callback functions of the current event
    const callbackObject = this.eventObject[eventName];

    if (!callbackObject) return console.warn(eventName + " not found!");

    // execute each callback function
    for (let id in callbackObject) {
      // pass parameters when executing
      callbackObject[id](...args);
    }
  }
  // Subscribe to events
  subscribe(eventName, callback) {
    // initialize this event
    if (!this.eventObject[eventName]) {
      // Use object storage to improve the efficiency of deletion when logging out the callback function
      this.eventObject[eventName] = {};
    }

    const id = this.callbackId++;

    // store the callback function of the subscriber
    // callbackId needs to be incremented after use for the next callback function
    this.eventObject[eventName][id] = callback;

    // Every time you subscribe to an event, a unique unsubscribe function is generated
    const unSubscribe = () => {
      // clear the callback function of this subscriber
      delete this.eventObject[eventName][id];

      // If this event has no subscribers, also clear the entire event object
      if (Object.keys(this.eventObject[eventName]).length === 0) {
        delete this.eventObject[eventName];
      }
    };

    return { unSubscribe };
  }
}

// test
const eventBus = new EventBus();

// Subscribe to event eventX
eventBus.subscribe("eventX", (obj, num) => {
  console.log("Module A", obj, num);
});
eventBus.subscribe("eventX", (obj, num) => {
  console.log("Module B", obj, num);
});
const subscriberC = eventBus.subscribe("eventX", (obj, num) => {
  console.log("Module C", obj, num);
});

// publish event eventX
eventBus.publish("eventX", { msg: "EventX published!" }, 1);

// Module C unsubscribes
subscriberC.unSubscribe();

// Publish the event eventX again, module C will no longer receive the message
eventBus.publish("eventX", { msg: "EventX published again!" }, 2);

// output
> Module A {msg: 'EventX published!'} 1
> Module B {msg: 'EventX published!'} 1
> Module C {msg: 'EventX published!'} 1
> Module A {msg: 'EventX published again!'} 2
> Module B {msg: 'EventX published again!'} 2

3. How to subscribe only once

If an event occurs only once, it usually only needs to be subscribed once, and there is no need to receive messages after receiving messages.

First, we provide an interface of subscribeOnce, the internal implementation is almost the same as subscribe, there is only one difference, add a character d before callbackId to indicate that this is a subscription that needs to be deleted.

// Callback function marked as subscribe only once
const id = "d" + this.callbackId++;

Then, after executing the callback function, judge whether the id of the current callback function is marked, and decide whether we need to delete the callback function.

// The callback function that is only subscribed once needs to be deleted
if (id[0] === "d") {
  delete callbackObject[id];
}

Code

class EventBus {
  constructor() {
    // initialize event list
    this.eventObject = {};
    // id of the callback function list
    this.callbackId = 0;
  }
  // publish event
  publish(eventName, ...args) {
    // Get all the callback functions of the current event
    const callbackObject = this.eventObject[eventName];

    if (!callbackObject) return console.warn(eventName + " not found!");

    // execute each callback function
    for (let id in callbackObject) {
      // pass parameters when executing
      callbackObject[id](...args);

      // The callback function that is only subscribed once needs to be deleted
      if (id[0] === "d") {
        delete callbackObject[id];
      }
    }
  }
  // Subscribe to events
  subscribe(eventName, callback) {
    // initialize this event
    if (!this.eventObject[eventName]) {
      // Use object storage to improve the efficiency of deletion when logging out the callback function
      this.eventObject[eventName] = {};
    }

    const id = this.callbackId++;

    // store the callback function of the subscriber
    // callbackId needs to be incremented after use for the next callback function
    this.eventObject[eventName][id] = callback;

    // Every time you subscribe to an event, a unique unsubscribe function is generated
    const unSubscribe = () => {
      // clear the callback function of this subscriber
      delete this.eventObject[eventName][id];

      // If this event has no subscribers, also clear the entire event object
      if (Object.keys(this.eventObject[eventName]).length === 0) {
        delete this.eventObject[eventName];
      }
    };

    return { unSubscribe };
  }

  // only subscribe once
  subscribeOnce(eventName, callback) {
    // initialize this event
    if (!this.eventObject[eventName]) {
      // Use object storage to improve the efficiency of deletion when logging out the callback function
      this.eventObject[eventName] = {};
    }

    // Callback function marked as subscribe only once
    const id = "d" + this.callbackId++;

    // store the callback function of the subscriber
    // callbackId needs to be incremented after use for the next callback function
    this.eventObject[eventName][id] = callback;

    // Every time you subscribe to an event, a unique unsubscribe function is generated
    const unSubscribe = () => {
      // clear the callback function of this subscriber
      delete this.eventObject[eventName][id];

      // If this event has no subscribers, also clear the entire event object
      if (Object.keys(this.eventObject[eventName]).length === 0) {
        delete this.eventObject[eventName];
      }
    };

    return { unSubscribe };
  }
}

// test
const eventBus = new EventBus();

// Subscribe to event eventX
eventBus.subscribe("eventX", (obj, num) => {
  console.log("Module A", obj, num);
});
eventBus.subscribeOnce("eventX", (obj, num) => {
  console.log("Module B", obj, num);
});
eventBus.subscribe("eventX", (obj, num) => {
  console.log("Module C", obj, num);
});

// publish event eventX
eventBus.publish("eventX", { msg: "EventX published!" }, 1);

// Publish the event eventX again, module B only subscribes once, and will not receive any more messages
eventBus.publish("eventX", { msg: "EventX published again!" }, 2);

// output
> Module A {msg: 'EventX published!'} 1
> Module C {msg: 'EventX published!'} 1
> Module B {msg: 'EventX published!'} 1
> Module A {msg: 'EventX published again!'} 2
> Module C {msg: 'EventX published again!'} 2

4. How to clear an event or all events

We also hope to clear all subscriptions of the specified event through a clear operation, which is usually used when some components or modules are uninstalled.

// clear event
clear(eventName){

    // If no event name is provided, all events are cleared by default
    if(!eventName){
        this.eventObject = {}
        return
    }

    // clear the specified event
    delete this.eventObject[eventName]
}

Similar to the logic of unsubscribing, except that it is handled uniformly here.

Code

class EventBus {
  constructor() {
    // initialize event list
    this.eventObject = {};
    // id of the callback function list
    this.callbackId = 0;
  }
  // publish event
  publish(eventName, ...args) {
    // Get all the callback functions of the current event
    const callbackObject = this.eventObject[eventName];

    if (!callbackObject) return console.warn(eventName + " not found!");

    // execute each callback function
    for (let id in callbackObject) {
      // pass parameters when executing
      callbackObject[id](...args);

      // The callback function that is only subscribed once needs to be deleted
      if (id[0] === "d") {
        delete callbackObject[id];
      }
    }
  }
  // Subscribe to events
  subscribe(eventName, callback) {
    // initialize this event
    if (!this.eventObject[eventName]) {
      // Use object storage to improve the efficiency of deletion when logging out the callback function
      this.eventObject[eventName] = {};
    }

    const id = this.callbackId++;

    // store the callback function of the subscriber
    // callbackId needs to be incremented after use for the next callback function
    this.eventObject[eventName][id] = callback;

    // Every time you subscribe to an event, a unique unsubscribe function is generated
    const unSubscribe = () => {
      // clear the callback function of this subscriber
      delete this.eventObject[eventName][id];

      // If this event has no subscribers, also clear the entire event object
      if (Object.keys(this.eventObject[eventName]).length === 0) {
        delete this.eventObject[eventName];
      }
    };

    return { unSubscribe };
  }

  // only subscribe once
  subscribeOnce(eventName, callback) {
    // initialize this event
    if (!this.eventObject[eventName]) {
      // Use object storage to improve the efficiency of deletion when logging out the callback function
      this.eventObject[eventName] = {};
    }

    // Callback function marked as subscribe only once
    const id = "d" + this.callbackId++;

    // store the callback function of the subscriber
    // callbackId needs to be incremented after use for the next callback function
    this.eventObject[eventName][id] = callback;

    // Every time you subscribe to an event, a unique unsubscribe function is generated
    const unSubscribe = () => {
      // clear the callback function of this subscriber
      delete this.eventObject[eventName][id];

      // If this event has no subscribers, also clear the entire event object
      if (Object.keys(this.eventObject[eventName]).length === 0) {
        delete this.eventObject[eventName];
      }
    };

    return { unSubscribe };
  }

  // clear event
  clear(eventName) {
    // If no event name is provided, all events are cleared by default
    if (!eventName) {
      this.eventObject = {};
      return;
    }

    // clear the specified event
    delete this.eventObject[eventName];
  }
}

// test
const eventBus = new EventBus();

// Subscribe to event eventX
eventBus.subscribe("eventX", (obj, num) => {
  console.log("Module A", obj, num);
});
eventBus.subscribe("eventX", (obj, num) => {
  console.log("Module B", obj, num);
});
eventBus.subscribe("eventX", (obj, num) => {
  console.log("Module C", obj, num);
});

// publish event eventX
eventBus.publish("eventX", { msg: "EventX published!" }, 1);

// clear
eventBus.clear("eventX");

// Publish the event eventX again, since it has been cleared, all modules will no longer receive the message
eventBus.publish("eventX", { msg: "EventX published again!" }, 2);

// output
> Module A {msg: 'EventX published!'} 1
> Module B {msg: 'EventX published!'} 1
> Module C {msg: 'EventX published!'} 1
> eventX not found!

5. TypeScript version

TypeScript is now widely adopted, especially for large front-end projects, we briefly revamp it to a TypeScript version

You can copy the following code to TypeScript Playground to run

Code

interface ICallbackList {
  [id: string]: Function;
}

interface IEventObject {
  [eventName: string]: ICallbackList;
}

interface ISubscribe {
  unSubscribe: () => void;
}

interface IEventBus {
  publish<T extends any[]>(eventName: string, ...args: T): void;
  subscribe(eventName: string, callback: Function): ISubscribe;
  subscribeOnce(eventName: string, callback: Function): ISubscribe;
  clear(eventName: string): void;
}

class EventBus implements IEventBus {
  private _eventObject: IEventObject;
  private _callbackId: number;
  constructor() {
    // initialize event list
    this._eventObject = {};
    // id of the callback function list
    this._callbackId = 0;
  }
  // publish event
  publish<T extends any[]>(eventName: string, ...args: T): void {
    // Get all the callback functions of the current event
    const callbackObject = this._eventObject[eventName];

    if (!callbackObject) return console.warn(eventName + " not found!");

    // execute each callback function
    for (let id in callbackObject) {
      // pass parameters when executing
      callbackObject[id](...args);

      // The callback function that is only subscribed once needs to be deleted
      if (id[0] === "d") {
        delete callbackObject[id];
      }
    }
  }
  // Subscribe to events
  subscribe(eventName: string, callback: Function): ISubscribe {
    // initialize this event
    if (!this._eventObject[eventName]) {
      // Use object storage to improve the efficiency of deletion when logging out the callback function
      this._eventObject[eventName] = {};
    }

    const id = this._callbackId++;

    // store the callback function of the subscriber
    // callbackId needs to be incremented after use for the next callback function
    this._eventObject[eventName][id] = callback;

    // Every time you subscribe to an event, a unique unsubscribe function is generated
    const unSubscribe = () => {
      // clear the callback function of this subscriber
      delete this._eventObject[eventName][id];

      // If this event has no subscribers, also clear the entire event object
      if (Object.keys(this._eventObject[eventName]).length === 0) {
        delete this._eventObject[eventName];
      }
    };

    return { unSubscribe };
  }

  // only subscribe once
  subscribeOnce(eventName: string, callback: Function): ISubscribe {
    // initialize this event
    if (!this._eventObject[eventName]) {
      // Use object storage to improve the efficiency of deletion when logging out the callback function
      this._eventObject[eventName] = {};
    }

    // Callback function marked as subscribe only once
    const id = "d" + this._callbackId++;

    // store the callback function of the subscriber
    // callbackId needs to be incremented after use for the next callback function
    this._eventObject[eventName][id] = callback;

    // Every time you subscribe to an event, a unique unsubscribe function is generated
    const unSubscribe = () => {
      // clear the callback function of this subscriber
      delete this._eventObject[eventName][id];

      // If this event has no subscribers, also clear the entire event object
      if (Object.keys(this._eventObject[eventName]).length === 0) {
        delete this._eventObject[eventName];
      }
    };

    return { unSubscribe };
  }

  // clear event
  clear(eventName: string): void {
    // If no event name is provided, all events are cleared by default
    if (!eventName) {
      this._eventObject = {};
      return;
    }

    // clear the specified event
    delete this._eventObject[eventName];
  }
}

// test
interface IObj {
  msg: string;
}

type PublishType = [IObj, number];

const eventBus = new EventBus();

// Subscribe to event eventX
eventBus.subscribe("eventX", (obj: IObj, num: number, s: string) => {
  console.log("Module A", obj, num);
});
eventBus.subscribe("eventX", (obj: IObj, num: number) => {
  console.log("Module B", obj, num);
});
eventBus.subscribe("eventX", (obj: IObj, num: number) => {
  console.log("Module C", obj, num);
});

// publish event eventX
eventBus.publish("eventX", { msg: "EventX published!" }, 1);

// clear
eventBus.clear("eventX");

// Publish the event eventX again, since it has been cleared, all modules will no longer receive the message
eventBus.publish<PublishType>("eventX", { msg: "EventX published again!" }, 2);

// output
[LOG]: "Module A", {
  "msg": "EventX published!"
},  1
[LOG]: "Module B", {
  "msg": "EventX published!"
},  1
[LOG]: "Module C", {
  "msg": "EventX published!"
},  1
[WRN]: "eventX not found!"

6. Singleton Pattern

In actual use, only one event bus is often needed to meet the requirements. There are two cases here, keep the singleton in the upper instance, and the global singleton.

Keep the singleton in the upper instance

Import the event bus to the upper-layer instance, it only needs to ensure that there is only one EventBus in an upper-layer instance. If there are multiple upper-layer instances, it means that there are multiple event buses, but each upper-layer instance controls its own event bus.
First, a variable is established in the upper-level instance to store the event bus, which is only initialized when it is used for the first time, and the event bus instance is directly obtained when other modules use the event bus.

Code

// upper instance
class LWebApp {
  private _eventBus?: EventBus;

  constructor() {}

  public getEventBus() {
    // first initialization
    if (this._eventBus == undefined) {
      this._eventBus = new EventBus();
    }

    // Subsequent to directly take only one instance each time, keep it as a single instance in the LWebApp instance
    return this._eventBus;
  }
}

// use
const eventBus = new LWebApp().getEventBus();

Global singleton

Sometimes we hope that no matter which module wants to use our event bus, we all want these modules to use the same instance, which is a global singleton. This design makes it easier to manage events in a unified manner.

The writing method is similar to the above, the difference is to convert _eventBus and getEventBus to static properties. There is no need to instantiate the EventBusTool class when using it, just use the static method directly.

Code

// upper instance
class EventBusTool {
  private static _eventBus?: EventBus;

  constructor() {}

  public static getEventBus(): EventBus {
    // first initialization
    if (this._eventBus == undefined) {
      this._eventBus = new EventBus();
    }

    // Subsequent to directly take a unique instance each time, keep the global singleton
    return this._eventBus;
  }
}

// use
const eventBus = EventBusTool.getEventBus();

Conclusion

The above are some of the my understanding of Event Bus, which basically achieves the desired feature. By implementing the publish-subscribe model by yourself, it also deepens the understanding of the classic design pattern. There are still many shortcomings and areas that need to be optimized. Welcome to share your experience.

Check my blog https://lwebapp.com/en/posts

DEV Community: openHacking

Python + ChatGPT API Development | Based on gpt-3.5-turbo model

Background

ChatGPT API

Introduction

Usage

Whisper API

Conclusion

Reference

Develop an Intelligent Chat Program Using Python and ChatGPT API

Introduction

Solution

Step 1: Install the necessary Python libraries

Step 2: Obtain an API key

Step 3: Create the Flask Application

Step 4: Run the application

Conclusion

Reference

Open Source IM System Tinode Deployment Tutorial WSL Environment

Introduction

Solution

WSL starts front-end and back-end services

Front-end development

Reference

3 Ways to Implement Network Requests in Pyodide

Question

Solution

1. http.open_url

2. http.pyfetch

3. fetch in js module

Conclusion

Reference

Python Image to ASCII Online

Introduction

Analysis

Code

1. Use base64 images

2. Load image from URL

Conclusion

Reference

Install Bun for Windows: A Modern JavaScript Runtime Like Node or Deno

Introduction

Environment Preparation

Install Bun

Conclusion

Reference

Share a Useful Python Online Editor

Introduction

How to use Python Playground tool?

What is Python?

Why learn Python?

Why use Python Playground?

Conclusion

Reference

2Captcha Review 2022 | Principles & Usage & Getting Started

Introduction

Question

2Captcha Introduction

Why use 2Captcha

2Captcha development experience

2Captcha Advantages

Conclusion

Reference

How to Bypass Captcha Automatic Login with Nodejs Playwright 2Captcha

Question

Explore

Solution

Environment

Request in.php interface

Request res.php interface

Login

Conclusion

Reference

pip Install Third-party Library Error

Question

Solution

Option 1

Option 2

Option 3

Option 4

Request `in.php` interface

Request `res.php` interface