Base128 Algorithm: Tool for Encoding and Decoding text data

  __________                          ____ ________    ______  
  \______   \_____     ______  ____  /_   |\_____  \  /  __  \ 
   |    |  _/\__  \   /  ___/_/ __ \  |   | /  ____/  >      < 
   |    |   \ / __ \_ \___ \ \  ___/  |   |/       \ /   --   \
   |______  /(____  //____  > \___  > |___|\_______ \\______  /
          \/      \/      \/      \/               \/       \/

Encoding and decoding algorithms are critical in today's digital world.

One of the interesting and reliable solutions is the Base128 algorithm.

This article will introduce you to the principles of operation of this algorithm and present the practical application of the source code.

Base128 is a program written in Go that allows you to encode and decode text data using a custom implementation of Base128 encoding. The algorithm processes the input data by converting it to a sequence of integers (in the range 0-127) and then reconstructing the original text from it.

The program can be run with various flags that determine whether the data is to be encoded or decoded, whether help is displayed, or whether information about the program version is available.

If the user selects the -e flag, the program encodes the input using Base128 encoding. Text data is read from the input file or standard input, and whitespace and newlines are stripped from it. The input is then converted to a sequence of integers using the encodeBase128 function.

Finally, the sequence of numbers is formatted as a single string and displayed on standard output.

If the user selects the -d flag, the program decodes the input, which is read from the input file or from standard input. As with the encoding, whitespace and newlines are removed. The input is then split into single numbers, which are converted to a sequence of integers using the decodeBase128 function. Finally, the sequence of numbers is converted back to text and displayed on standard output.

The program also supports the -h, --version, and --copyright flags, which display the appropriate information. These flags require no input and do no encoding or decoding.

In conclusion, the Base128 program allows you to encode and decode text data using a custom implementation of Base128 encoding. It can be used to transform text data into a sequence of integers and vice versa. The program is simple to use, with a command-line interface and various flags that can be used depending on your needs.

Below is a step-by-step description of this algorithm and the source code written in Go that implements the Base128 algorithm. This code will allow you to encode and decode text data in this format.

• package main: Defines that this is the main program (not a library).
• Imports:
• flag: For handling command line flags (e.g. -e, -d).
• fmt: For formatting and displaying text.
• io: For input/output operations (e.g. reading from stdin).
• os: For interacting with the operating system (e.g. reading files).
• strconv: For converting strings to numbers and vice versa.
• strings: For manipulating strings (e.g. concatenation, splitting).

package main

import (
    "flag"
    "fmt"
    "io/ioutil"
    "os"
    "strconv"
    "strings"
)

2. Constant base

const base = 128

• base = 128: Defines the encoding range (0-127) that corresponds to standard ASCII characters.

3. Function encodeBase128

func encodeBase128(input string) []int {
result := make([]int, 0, len(input))
for _, char := range input {
result = append(result, int(char))
}
return result
}

• Purpose: Converts a string to an array of integers (range 0-127).
• Step by step:
• Creates an empty array result with capacity equal to the length of the input string.
• Iterates over each character in the input string.
• Converts a character to its numeric value (ASCII) and appends it to the result array.
• Returns an array of numbers.

---

4. Function decodeBase128

func decodeBase128(input []int) string {
var builder strings.Builder
for _, code := range input {
_, err := builder.WriteRune(rune(code))
if err != nil {
fmt.Fprintln(os.Stderr, "base128: error writing to builder:", err)
os.Exit(1)
}
}
return builder.String()
}

• Purpose: Converts an integer array back to a string.
• Step by step:
• Creates a strings.Builder object to efficiently build a string.
• Iterates over each number in the input array.
• Converts the number to a character (rune) and adds it to builder.
• If an error occurs, the program exits with an error message.
• Returns the built string.

5. The main function

The main function is the main function of the program, which handles flags, reads input data, and calls the appropriate functions to encode or decode.

5.1. Definition of flags

encodeFlag := flag.Bool("e", false, "Converts the input's base128 encoding into an output text file.")
decodeFlag := flag.Bool("d", false, "Recovers the original input file by decoding the information that was previously encoded using base128.")
helpFlag := flag.Bool("h", false, "Print instructions for calling and a list of available alternatives.")
versionFlag := flag.Bool("version", false, "Print the program's version.")
copyrightFlag := flag.Bool("copyright", false, "Print copyright information.")
flag.Parse()

• Flags:
  • -e: Encodes the input.
  • -d: Decodes the input.
  • -h: Display help.
  • --version: Displays the program version.
• --copyright: Displays copyright information.
• flag.Parse(): Parses flags from the command line.

5.2. Handling -h, --version, --copyright

if *helpFlag {
// Display help
os.Exit(0)
}
if *versionFlag {
// Display version
os.Exit(0)
}
if *copyrightFlag {
// Display copyright information
os.Exit(0)
}

• If any of these flags are set, the program displays an appropriate message and exits.

---

5.3. Check for flag conflict

if *encodeFlag && *decodeFlag {
fmt.Fprintln(os.Stderr, "base128: cannot specify both -e and -d flags. Please use -e for ENCODE or -d for DECODE or -h for HELP")
os.Exit(1)
}

• Checks if the user is trying to encode and decode data at the same time. If so, the program exits with an error.

---

5.4. Checking for no action

if !*encodeFlag && !*decodeFlag {
fmt.Fprintln(os.Stderr, "base128: no action specified. Please use -e for ENCODE or -d for DECODE or -h for HELP")
os.Exit(1)
}

• Checks if the user has not selected any action (encode or decode). If not, the program exits with an error.

---

5.5. Reading input data

args := flag.Args()
var input string
if len(args) == 0 || args[0] == "-" {
bytes, err := io.ReadAll(os.Stdin)
if err != nil {
fmt.Fprintln(os.Stderr, "base128: error reading from stdin:", err)
os.Exit(1)
}
input = string(bytes)
} else {
bytes, err := os.ReadFile(args[0])
if err != nil {
fmt.Fprintln(os.Stderr, "base128: error reading file:", err)
os.Exit(1)
}
input = string(bytes)
}

• Step by step:
• Checks if the user has provided a file as an argument.
• If not, reads data from standard input (stdin).
• If a file is given, reads data from the file.
• If an error occurs, the program exits with an error message.

---

5.6. Data processing

if *encodeFlag {
 numbers := encodeBase128(input)
 output := strings.Trim(strings.Join(strings.Fields(fmt.Sprint(numbers)), " "), "[]")
 fmt.Println(output)
} else if *decodeFlag {
 numbersStr := strings.Fields(input)
 numbers := make([]int, len(numbersStr))
 for i, numberStr := range numbersStr {
 number, err := strconv.Atoi(numberStr)
 if err != nil {
 fmt.Fprintln(os.Stderr, "base128: error decoding number:", err)
 os.Exit(1)
 }
 numbers[i] = number
 }
 output := decodeBase128(numbers)
fmt.Println(output)
}

• If -e flag:
• Encodes input using encodeBase128.
• Formats the resulting array of numbers as a string (e.g. [72 101 108] → "72 101 108").
• Displays the result.
• If -d flag:
• Splits input into single numbers (e.g. "72 101 108" → ["72", "101", "108"]).
• Converts strings to integers.
• Decodes numbers using decodeBase128.
• Displays the result.

Summary

The base128 program:

1. Supports flags for encoding, decoding, displaying help, version, and copyright information.
2. Reads data from a file or standard input.
3. Encodes text to a string of numbers (range 0-127) or decodes a string of numbers back to text.
4. It is intended primarily for text data, not binary data.

---

The Base128 encoding and decoding algorithm is applicable to text data processing. It offers effective tools for converting text data into a numeric representation, which can be useful in certain scenarios.

For general use, the Base128 algorithm can be used in a variety of situations. Application examples are:

Using the Base128 algorithm:

1. Text data transformation:

The Base128 algorithm can be used to transform text data into a sequence of integers, making it less human-readable. However, it is important to note that this is not a form of encryption and does not provide strong security.

2. Communication over the network:

Base128-encoded data can be transmitted over the network in a numeric format. However, since the algorithm is designed for text data, it is not suitable for encoding binary data.

3. Data representation:

The Base128 algorithm can be used to represent text data in a numeric form, which may be useful in certain data processing or storage scenarios.

Limitations of the Base128 algorithm in this implementation:

1. Text-only support:
   
   The program is designed to work with text data and does not support binary data. Attempting to encode or decode binary data may result in errors or incorrect output.

2. No compression or security:
   
   The algorithm does not compress data or provide encryption. It simply converts text to a sequence of integers and vice versa.

3. Limited character range:
   
   The algorithm works with characters in the range 0-127 (ASCII), which means it cannot handle extended character sets or binary data.

Summary:

The Base128 algorithm, as implemented in this program, is an effective tool for encoding and decoding text data. It offers a simple way to transform text into a numeric representation and vice versa. The Go source code presented in this article enables practical application and experimentation with this algorithm.

While the algorithm is not suitable for binary data or secure encryption, it can be useful in scenarios where text data needs to be represented in a numeric format. Thanks to its simplicity of operation and flexibility, the Base128 algorithm is a tool worth exploring for text data processing.

The entire base128 algorithm:

The base128 site: HERE
The GitHub repositories: THERE

Thank you for reading.
If you have any questions, or tips, you can leave them here in the comments. I will answer as soon as possible.

Comments

[Zayd Krunz]

The Base128 algorithm can be used to encode sensitive information such as passwords or credit card numbers. Converting data into numeric form makes it difficult for unauthorized users to read it.

That is just completely untrue. No encoding is ever, ever secure. Other than that, great article.

[feranmiodugbemi]

Very nice and informative content here Łukasz Wójcik, but try to add syntax highlighting to your markdown code