DEV Community: Pawel Suchanecki

Generate ASCII Graphs of Cryptocurrency Prices in Python

Pawel Suchanecki — Sun, 02 Apr 2023 23:05:44 +0000

Have you ever wanted to visualize the historical prices of a cryptocurrency in a fun and interactive way? Well, look no further! In this article, we'll explore how to generate an ASCII graph of cryptocurrency prices using Python.

Disclaimer:

The author of this article is not affiliated with Binance in any way. This code was written to retrieve and visualize data, which the author found to be useful and accessible without requiring an API key. The author is not responsible for any errors or issues that may arise from using this code.

Example

$ python3 ascii-currency-graphs.py --pair BTCEUR --start-date 2023-03-14 --end-date 2023-04-01
2023-03-14T01:00:00 ########################################### 22985.84
2023-03-15T01:00:00 ########################################### 23026.80
2023-03-16T01:00:00 ############################################ 23665.73
2023-03-17T01:00:00 ################################################ 25789.77
2023-03-18T01:00:00 ############################################### 25373.59
2023-03-19T01:00:00 ################################################## 26474.44
2023-03-20T01:00:00 ################################################# 26082.53
2023-03-21T01:00:00 ################################################# 26324.74
2023-03-22T01:00:00 ############################################### 25342.45
2023-03-23T01:00:00 ################################################# 26260.20
2023-03-24T01:00:00 ################################################ 25746.57
2023-03-25T01:00:00 ################################################ 25735.12
2023-03-26T01:00:00 ################################################# 26076.46
2023-03-27T02:00:00 ############################################### 25267.76
2023-03-28T02:00:00 ############################################### 25189.56
2023-03-29T02:00:00 ################################################# 26188.76
2023-03-30T02:00:00 ################################################ 25798.34
2023-03-31T02:00:00 ################################################# 26332.77

Raw Data

Here you can check the data that we get from Binance. The below presented script (The Code) receives it for command:

python3 ascii-currency-graphs.py --pair BTCEUR --start-date 2023-04-01 --end-date 2023-04-05

[
  [
    1680307200000,
    "26330.62000000",
    "26633.95000000",
    "26127.13000000",
    "26315.71000000",
    "494.04749000",
    1680393599999,
    "13008528.45896390",
    50549,
    "256.75654000",
    "6762406.03339990",
    "0"
  ],
  [
    1680393600000,
    "26319.67000000",
    "26403.97000000",
    "25828.00000000",
    "26184.72000000",
    "571.36554000",
    1680479999999,
    "14920527.65324270",
    55327,
    "277.62573000",
    "7250885.00425640",
    "0"
  ],
  [
    1680480000000,
    "26187.25000000",
    "26349.00000000",
    "25221.16000000",
    "25662.56000000",
    "1681.40230000",
    1680566399999,
    "43525785.31802880",
    100106,
    "868.39176000",
    "22493356.97115660",
    "0"
  ],
  [
    1680566400000,
    "25662.15000000",
    "26182.85000000",
    "25488.59000000",
    "25795.05000000",
    "1008.78663000",
    1680652799999,
    "26053821.41279880",
    80241,
    "579.95600000",
    "14980595.35159550",
    "0"
  ]
]

The Code

import argparse
import datetime
import requests


def iso_date(date_string):
    """Convert ISO-formatted date string to timestamp."""
    try:
        date = datetime.datetime.strptime(date_string, '%Y-%m-%d')
        timestamp = int(date.timestamp() * 1000)
        return timestamp
    except ValueError:
        raise argparse.ArgumentTypeError(
            f"Invalid date format: {date_string}. Date should be in the format YYYY-MM-DD."
        )


def get_crypto_prices(start_timestamp, end_timestamp, pair):
    """Retrieve historical prices for the specified trading pair and date range."""
    start_date = datetime.date.fromtimestamp(start_timestamp / 1000)
    end_date = datetime.date.fromtimestamp(end_timestamp / 1000)

    symbol = pair.replace(':', '').replace('_', '').upper()
    url = f"https://api.binance.com/api/v3/klines?symbol={symbol}&interval=1d&startTime={start_timestamp}&endTime={end_timestamp}"
    response = requests.get(url)

    if response.status_code == 200:
        prices = response.json()
        return [(datetime.datetime.fromtimestamp(int(price[0]) / 1000).isoformat(), float(price[4])) for price in prices]
    else:
        print(f"Failed to retrieve prices for {pair} from {start_date} - {end_date}")
        return []


def generate_ascii_graph(data):
    """Generate an ASCII graph of cryptocurrency prices."""
    max_price = max(data, key=lambda x: x[1])[1]

    for date, price in data:
        bar = '#' * int(price / max_price * 50)
        print(f"{date} {bar} {price:.2f}")


def list_cryptocurrencies():
    """List all possible cryptocurrencies supported by the Binance API."""
    url = 'https://api.binance.com/api/v3/exchangeInfo'
    response = requests.get(url)

    if response.status_code == 200:
        data = response.json()
        symbols = data['symbols']
        currencies = set()

        for symbol in symbols:
            base_currency = symbol['baseAsset']
            quote_currency = symbol['quoteAsset']
            currencies.add(base_currency)
            currencies.add(quote_currency)

        print("Available cryptocurrencies on Binance:")
        for currency in sorted(currencies):
            print(currency)
    else:
        print("Failed to retrieve list of cryptocurrencies from Binance.")


if __name__ == '__main__':
    # Parse command-line arguments
    parser = argparse.ArgumentParser(description="Generate an ASCII graph of cryptocurrency prices for a given date range")
    parser.add_argument("--pair", help="Trading pair for the historical data in the format 'BTC:USDT' or 'BTCUSDT'", default='BTCUSDT')
    parser.add_argument("--list", help="List all possible cryptocurrencies supported by the Binance API", action='store_true')
    parser.add_argument("--start-date", type=iso_date, help="Start date (in YYYY-MM-DD format) for the historical data", default=int((datetime.datetime.now() - datetime.timedelta(days=1)).timestamp() * 1000))
    parser.add_argument("--end-date", type=iso_date, help="End date (in YYYY-MM-DD format) for the historical data", default=int(datetime.datetime.now().timestamp() * 1000))
    args = parser.parse_args()

    if args.list:
        list_cryptocurrencies()
    else:
        # Retrieve and process cryptocurrency prices
        prices = get_crypto_prices(args.start_date, args.end_date, args.pair)
        if prices:
            generate_ascii_graph(prices)

What does this code do?

This Python code retrieves historical prices for a specified cryptocurrency trading pair from the Binance API and generates an ASCII graph of the prices for the specified date range. It also allows you to list all possible cryptocurrencies supported by the Binance API. You can easily run this code from the command line and get a cool visualization of the cryptocurrency prices.

How can you add it to your .bashrc or MOTD regenerator?

If you're feeling adventurous, you can add this code to your .bashrc or MOTD regenerator for fun and profit! Just call the script to your .bashrc or MOTD regenerator. Now, every time you open your terminal or log in to your server, you'll be greeted with a visualization of the current cryptocurrency prices!

How does the code work?

Parsing Command-Line Arguments

The code uses Python's built-in argparse module to parse command-line arguments. The parser.add_argument() method is used to define the different arguments that the script accepts. For example, the --pair argument is used to specify the trading pair for the historical data, while the --start-date and --end-date arguments are used to specify the date range for the historical data. The --list argument is used to list all possible cryptocurrencies supported by the Binance API.

Retrieving Historical Data

The get_crypto_prices() function retrieves historical prices for a specified cryptocurrency trading pair and date range from the Binance API. It uses the requests module to make an HTTP GET request to the Binance API, and then parses the JSON response to extract the historical prices. The function returns a list of tuples, where each tuple contains a date and a price.

Generating an ASCII Graph

The generate_ascii_graph() function takes a list of tuples containing dates and prices, and generates an ASCII graph of the prices. It first determines the maximum price in the dataset using a lambda function that calls the max function with a key argument of lambda x: x[1]. This specifies that we want to get the maximum price based on the second element of each tuple in the dataset. It then loops through the data and generates a bar chart for each data point, with the length of the bar proportional to the price relative to the maximum price.

Listing All Cryptocurrencies Supported by Binance

The list_cryptocurrencies() function retrieves a list of all possible cryptocurrencies supported by the Binance API. It makes an HTTP GET request to the Binance API and parses the JSON response to extract the list of symbols. It then extracts the base and quote currencies from each symbol and adds them to a set of unique currencies. Finally, it prints out the set of unique currencies in alphabetical order.

Lambda Explained

One interesting part of this code is the use of a lambda function to get the maximum price in the dataset. A lambda function is a small anonymous function that can have any number of arguments, but can only have one expression. In this code, we use a lambda function to get the maximum price in the dataset by calling the max function with a key argument of lambda x: x[1]. This specifies that we want to get the maximum price based on the second element of each tuple in the dataset. If you're interested in learning more about lambda functions, check out the official Python documentation.

Example Usage

Here are some example usage scenarios for this code (written in 2023 ;-)):

To get the historical prices for BTCUSDT from the past week and generate an ASCII graph:

python ascii-currency-graphs.py --pair BTCUSDT --start-date 2023-03-26 --end-date 2023-04-02

To get the historical prices for ETHBTC from the past year and generate an ASCII graph:

python ascii-currency-graphs.py --pair ETHBTC --start-date 2022-04-02`

To list all possible cryptocurrencies supported by the Binance API:

python ascii-currency-graphs.py --list

Conclusion

In this article, we learned how to generate an ASCII graph of cryptocurrency prices using Python. We explored the different parts of the code and how to use it from the command line. We also learned about lambda functions and how they can be used in Python. Now, go forth and visualize those cryptocurrency prices in style!

Next read

Stay tuned for the next article in this series, where we will build upon this code to generate PNG graphs with financial trendlines. This will give us a better understanding of the historical performance of cryptocurrencies and help us make informed decisions about our investments.

Scalable, Producer-Consumer-based, Remote Log Monitor in Python

Pawel Suchanecki — Fri, 10 Mar 2023 01:18:29 +0000

Intro

In this continuation of the article "How to Implement the Producer-Consumer Concurrency Design Pattern with asyncio Coroutines", we will show how to enable monitoring of multiple log files on remote machines.

We will explain the changes introduced in the code to achieve the goal and deepen the context of the ideas and discuss the used API.

The data

- host: testhost.almalinux.org
   user: someuser
   key_filename: /home/pawel/.ssh/id_ed25519 
   log_file: /var/log/nginx/access.log
   delay: 5
   buffer_lines: 10

- host: refurby
   user: pablo
   key_filename: /home/pawel/.ssh/id_rsa
   log_file: /var/log/syslog
   delay: 60
   buffer_lines: 5

The code

#!/usr/bin/env python3

import asyncio
import datetime
import time
from fabric import Connection
from asyncio.queues import Queue
import yaml


async def producer(queue: Queue, host: dict):
    conn = Connection(host['host'], user=host['user'], connect_kwargs={'key_filename': host['key_filename']})
    with conn.cd('/tmp'):
        tail_cmd = f"sudo tail -n {host['buffer_lines']} {host['log_file']}"
        old_data = ()
        while True:
            channel = conn.run(command=tail_cmd, hide='both')
            data = channel.stdout.splitlines()
            if old_data != data:
                old_d = set(old_data)
                delta_data = [x for x in data if x not in old_d]
                for d in delta_data:
                    await queue.put((host['host'], host['log_file'], str(d)))
            old_data = data.copy()
            data.clear()
            await asyncio.sleep(host['delay'])
    conn.close()


async def consumer(queue: Queue):
    while True:
        line = await queue.get()
        host, log_file, data = line
        dt = datetime.datetime.fromtimestamp(time.time())
        print(f"@{dt} {host}:{log_file}:\n{data}\n-----", end='\n', flush=True)
        queue.task_done()


async def main():
    with open('hosts.yml') as f:
        hosts = yaml.safe_load(f)
        queue = asyncio.Queue()
        producers = [asyncio.create_task(producer(queue, host)) for host in hosts]
        consumer_task = asyncio.create_task(consumer(queue))
        await asyncio.gather(*producers)
        await queue.join()
        consumer_task.cancel()


if __name__ == '__main__':
    asyncio.run(main())

Explanation: Supporting Multiple Hosts (Producers)

The main idea of the changes introduced in the code is to enable monitoring of multiple log files on remote machines using the same producer-consumer concurrency design pattern. To achieve this, we introduce a YAML configuration file hosts.yml that specifies the hosts to be monitored along with their properties such as the log file path, delay between checks, and number of lines to read.

In the main() function, we load the configuration from the YAML file using the yaml module and create an asyncio.Queue() object for inter-coroutine communication. We then create a separate producer coroutine for each host using a list comprehension and add each of them to the producers list. The asyncio.gather() method is used to run all producers and the consumer coroutine concurrently.

Explanation: Consuming the output from multiple Producers

The consumer() coroutine remains the same as in the previous version, except for the modification to handle data from multiple hosts. Each log line is now a tuple containing the host name, the log file path, and the log line text.

In the producer() coroutine, we now pass the host dictionary to the coroutine instead of using global variables. The dictionary contains the properties of the host to be monitored, such as the host name, SSH key filename, log file path, delay between checks, and number of lines to read. As in the original version, we use these properties to establish an SSH connection to the remote machine and execute the tail command on the log file. The resulting data is put into the queue as a tuple containing the host name, log file path, and log line text.

Explanation: How sleeping allows producers to reduce load

The await asyncio.sleep(host['delay']) line is used to introduce a delay between log file checks in the producer() coroutine. The delay property is specified in the host configuration dictionary loaded from the YAML file for each host individually, and the asyncio.sleep() function is used to pause execution of the coroutine for the specified amount of time. This allows the producer to wait for new log entries to appear before checking again, reducing unnecessary network and system load.

Summary

Overall, changes made to the previous version of the code enable the script to monitor multiple log files on remote machines in a scalable and maintainable way. By using a YAML configuration file, adding new hosts or modifying properties of existing hosts is easy and doesn't require changes to the code.

About API

As for the APIs used in the code, we used the yaml module to load the configuration from the YAML file and the [asyncio.queues.Queue()](https://docs.python.org/3/library/asyncio-queue.html) class to create the queue for inter-coroutine communication. The create_task() method is used to create the producer tasks and the gather() method is used to run all tasks concurrently. Finally, the join() and cancel() methods of the queue and consumer task are used to gracefully exit the program.

Outro

We hope this continuation has helped you understand how Consumer-Producer based code is easy to extend to support more producers in a scalable way. As always, feel free to experiment with the code and ask any questions you may have.

p.s. Stay tuned for more articles in the "Producer-Consumer Concurrency Design Pattern" series.

How to Implement the Producer-Consumer Concurrency Design Pattern with asyncio Coroutines

Pawel Suchanecki — Sat, 25 Feb 2023 01:19:01 +0000

The Producer-Consumer concurrency design pattern is a widely used approach in concurrent programming, where a producer generates data and a consumer consumes data. These components run in parallel, and the data is passed between them using a buffer or a queue.

In this article, we demonstrate how to use asyncio coroutines to implement the Producer-Consumer concurrency design pattern, with a practical example of monitoring a log file on a remote machine over SSH connection. We start by implementing the Producer-Consumer pattern using the yield statement, then we modify the code to use a queue for inter-coroutine communication.

Note: in example we require password-less sudo to work on remote host (server) as we monitor nginx log file, which is not publicly accessible.

Using `yield` Statement

from fabric import Connection
import asyncio
import datetime
import time

host = '1337-demo-host.org'
user = 'almalinux'
key_filename = '/Users/user/.ssh/id_rsa'
log_file = '/var/log/nginx/access.log'
delay = 5
buffer_lines = 5

async def producer():
    conn = Connection(host, user=user, connect_kwargs={'key_filename': key_filename})
    with conn.cd('/tmp'):
        tail_cmd = f'sudo tail -n {buffer_lines} {log_file}'
        old_data = ()
        while True:
            channel = conn.run(command=tail_cmd, hide='both')
            data = channel.stdout.splitlines()
            if old_data != data:
                old_d=set(old_data)
                delta_data = [x for x in data if x not in old_d]
                for d in delta_data:
                    yield str(d)
            old_data = data.copy()
            data.clear()
            time.sleep(delay)
    conn.close()

async def consumer():
    p=producer()
    async for line in p:
        dt = datetime.datetime.fromtimestamp(time.time())
        print (f"TS: {dt}\n{line}", end = '\n', flush = True)

async def main():
    await asyncio.gather(consumer())

if __name__ == '__main__':
    asyncio.run(main())

To make the script asynchronous, we use the asyncio library. The asyncio module provides a high-level concurrency framework based on coroutines, which allows us to write asynchronous code that looks synchronous. Coroutines are implemented using the async and await keywords.

In our script, the producer coroutine generates data by executing the tail command on the remote machine over SSH connection. The consumer coroutine consumes data by printing the new log lines along with the timestamp. The yield statement is used in the producer coroutine to generate new log lines, which are consumed by the consumer coroutine.

The asyncio.gather() method is used to run both coroutines concurrently, allowing the consumer coroutine to consume the data produced by the producer coroutine in a non-blocking way. The await keyword is used to wait for the coroutines to complete.

About Fabric

We used Fabric to establish an SSH connection to a remote machine, execute remote command and retrieve the output of it. Fabric is a Python library that simplifies SSH connections and remote execution of commands on multiple machines, providing a higher-level API for common tasks.
More on Fabric can be found at the project's website.

Using asyncio.Queue()

In this version, we replace the yield statement with an asyncio.Queue() object for inter-coroutine communication.

As previously, the producer coroutine generates data by executing the tail command on the remote machine over SSH connection but this time, it puts new log lines into the queue. The consumer coroutine reads data from the queue and prints the new log lines along with the timestamp.

The await queue.put() method is used to put new log lines into the queue, and the await queue.get() method is used to read new log lines from the queue. The queue.task_done() method is used to indicate that the consumer has finished processing a log line.

By using a queue for inter-coroutine communication, the script can handle a large number of log lines without the need to copy the data. The use of asyncio.Queue() object for communication is a best practice in asynchronous programming that allows the coroutines to run in parallel.

from fabric import Connection
import asyncio
import datetime
import time
from asyncio.queues import Queue

host = '1337-demo-host2.org'
user = 'almalinux'
key_filename = '/Users/user/.ssh/id_rsa'
log_file = '/var/log/nginx/access.log'
delay = 5
buffer_lines = 5

async def producer(queue: Queue):
    conn = Connection(host, user=user, connect_kwargs={'key_filename': key_filename})
    with conn.cd('/tmp'):
        tail_cmd = f'sudo tail -n {buffer_lines} {log_file}'
        old_data = ()
        while True:
            channel = conn.run(command=tail_cmd, hide='both')
            data = channel.stdout.splitlines()
            if old_data != data:
                old_d=set(old_data)
                delta_data = [x for x in data if x not in old_d]
                for d in delta_data:
                    await queue.put(d)
            old_data = data.copy()
            data.clear()
            await asyncio.sleep(delay)
    conn.close()

async def consumer(queue: Queue):
    while True:
        line = await queue.get()
        dt = datetime.datetime.fromtimestamp(time.time())
        print (f"TS: {dt}\n{line}", end = '\n', flush = True)
        queue.task_done()

async def main():
    queue = asyncio.Queue()
    await asyncio.gather(producer(queue), consumer(queue))

if __name__ == '__main__':
    asyncio.run(main())

Overall, this second version demonstrates how to use a queue for inter-coroutine communication in the Producer-Consumer concurrency design pattern with asyncio coroutines to monitor log file on remote machine. The use case here is trivial but this technique can be applied to a wide range of concurrent programming problems, making it a useful tool for developing scalable and high-performance systems.

Example:

# python3 producer-consumer-tail-logfile-over-ssh-with-queue.py
TS: 2023-02-25 01:12:23.705987
XXX.XX.XX.XX - - [24/Feb/2023:23:50:35 +0000] "GET / HTTP/1.1" 200 4727 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.2 Safari/605.1.15" "-"
TS: 2023-02-25 01:12:23.706113
XXX.XX.XX.XX - - [24/Feb/2023:23:50:36 +0000] "GET / HTTP/1.1" 200 4727 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.2 Safari/605.1.15" "-"
TS: 2023-02-25 01:12:23.706146
XXX.XX.XX.XX - - [24/Feb/2023:23:50:58 +0000] "GET //asdf HTTP/1.1" 404 3797 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.2 Safari/605.1.15" "-"
TS: 2023-02-25 01:12:23.706171
XXX.XX.XX.XX - - [25/Feb/2023:00:03:59 +0000] "GET //asdf HTTP/1.1" 404 3797 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.2 Safari/605.1.15" "-"
TS: 2023-02-25 01:12:23.706193
XXX.XX.XX.XX - - [25/Feb/2023:00:04:02 +0000] "GET //asdf HTTP/1.1" 404 3797 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.2 Safari/605.1.15" "-"
TS: 2023-02-25 01:12:33.965717
XXX.XX.XX.XX - - [25/Feb/2023:00:12:29 +0000] "GET //asdf HTTP/1.1" 404 3797 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/605.1.15 (KHTML, like Gecko) Version/16.2 Safari/605.1.15" "-"
TS: 2023-02-25 01:18:53.170400
XXX.XX.XX.XX - - [25/Feb/2023:00:18:52 +0000] "RAW / HTTP/1.1" 404 146 "-" "Mozilla/5.0 zgrab/0.x" "-"
TS: 2023-02-25 01:30:35.124575
XXX.XX.XX.XX - - [25/Feb/2023:00:30:31 +0000] "GET /.env HTTP/1.1" 404 146 "-" "Mozilla/5.0 (Macintosh; U; Intel Mac OS X 10_6_8; en-us) AppleWebKit/534.50 (KHTML, like Gecko) Version/5.1 Safari/534.50" "-"
TS: 2023-02-25 01:36:13.271236
XXX.XX.XX.XX - - [25/Feb/2023:00:36:12 +0000] "GET / HTTP/1.1" 200 4727 "-" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10.15; rv:106.0) Gecko/20100101 Firefox/106.0" "-"
TS: 2023-02-25 01:37:40.304185
XXX.XX.XX.XX - - [25/Feb/2023:00:37:38 +0000] "GET /0bef HTTP/1.0" 400 248 "-" "-" "-"

UPDATE:

`delta_data = [x for x in data if x not in old_d]` ?

This is a Python list comprehension that creates a new list called delta_data based on two input lists, data and old_d. The resulting list contains elements that are present in data, but not in old_d. With this we only put new lines to the Queue (queue.put()).

Break down of the code

delta_data: the new list we are creating.
data: the input list from which we will extract elements.
old_d: the reference list that we will compare against.
x: a ("temporary") variable that will hold each element of data as we iterate through it.
for x in data: a loop that iterates through each element of data.
if x not in old_d: a condition that checks whether the element x is not in old_d.
[x for x in data if x not in old_d]: the list comprehension that creates a new list delta_data by filtering data based on the above condition.

Summary

Overall, this code is useful for finding the elements that have been added to a list (data) since the last time it was checked, by comparing it to a reference list (old_d).

The resulting delta_data list contains only the new lines from monitored log file - this is what we need not to repeat messages. This also makes the code put nothing to Queue when no new data was observed (since last check).

OpenSSH 9.2: stricter default sandbox policy in OpenSSH client on OpenBSD

Pawel Suchanecki — Thu, 16 Feb 2023 14:51:30 +0000

The Release Notes for OpenSSH 9.2 (published on 2023-02-02) mentioned “stricter default sandbox policy” by “disabling the ~C command-line that was previously enabled by default.”

Turning off the command-line allows platforms that support sandboxing of the ssh client (currently only OpenBSD) to use a stricter default sandbox policy.

TL;DR

In summary, the OpenSSH client sandboxing feature on OpenBSD uses a combination of techniques such as pledge, unveil, chroot jail, and privilege separation to restrict the access of the OpenSSH client process to the system resources, directories and privileged operations. This helps to reduce the risk of security vulnerabilities in the OpenSSH client process from causing damage to the system. Disabling command-line escape makes it more secure. This article describes the context.

What is OpenBSD client sandboxing?

OpenSSH client sandboxing is a security feature implemented in OpenSSH on the OpenBSD operating system. The goal of this feature is to isolate the OpenSSH client process from the rest of the system to limit the damage that could be caused by potential security vulnerabilities.

How does it work?

OpenSSH client sandboxing is achieved through a combination of different techniques. First, the OpenSSH client process is run in a restricted environment using the "ssh-agent" program. This environment is created using the "pledge" and "unveil" system calls, which limit the system resources that the OpenSSH client process can access. For example, the pledge() system call restricts the system calls that the OpenSSH client process can make, while the unveil() system call limits the files and directories that the OpenSSH client process can access.

Second, the OpenSSH client process is executed in a chroot jail, which restricts the directories that the process can access. This ensures that even if the OpenSSH client process is compromised, the attacker will not be able to access other parts of the system.

Finally, the OpenSSH client sandboxing feature uses privilege separation to separate the privileged and non-privileged parts of the OpenSSH client process. This ensures that if an attacker gains access to the non-privileged part of the OpenSSH client process, they will not be able to execute privileged operations.

OpenSSH client pledge() example

The OpenSSH client on OpenBSD uses pledge() to restrict the operations that can be performed by the process. The pledge() call is made in the ssh_init() function in the file ssh.c.

Here's an excerpt from the OpenSSH client code in ssh.c that demonstrates how pledge() is used to set the restricted-service operating mode:

#ifdef HAVE_PLEDGE
   if (pledge("stdio rpath wpath cpath unix inet dns recvfd tty proc exec id", NULL) == -1) {
        error("pledge(): %s", strerror(errno));
        cleanup_exit(255);
    }
#endif

This call to pledge() sets the following promises:

stdio: permits use of standard I/O functions
rpath: permits read-only access to the file system for specific functions
wpath: permits write access to the file system for specific functions
cpath: permits creation of files and directories
unix: permits access to Unix domain sockets
inet: permits access to Internet sockets
dns: permits access to DNS functions
recvfd: permits receipt of file descriptors
tty: permits access to the tty
proc: permits access to process-related functions
exec: permits execution of other programs
id: permits access to identity-related functions

The second argument to pledge() is set to NULL, which means that the current promises are not changed.

By calling pledge() with these promises, the OpenSSH client is restricted to only the operations needed to perform SSH connections, and other operations that could be exploited by attackers are disallowed.

OpenBSD httpd web server `pledge()` example

Here's another example of OpenBSD code that uses pledge() to create a restricted-service operating mode. This code is from the OpenBSD httpd web server:

int main(int argc, char **argv)
{
    /* ... */

#ifdef __OpenBSD__
    /* enter the pledge */
    if (pledge("stdio rpath wpath cpath inet dns", NULL) == -1)
        err(1, "pledge");
#endif

    /* ... */
}

This call to pledge() sets the following promises:

stdio: permits use of standard I/O functions
rpath: permits read-only access to the file system for specific functions
wpath: permits write access to the file system for specific functions
cpath: permits creation of files and directories
inet: permits access to Internet sockets
dns: permits access to DNS functions

Note that this call to pledge() is conditional on the OpenBSD operating system being detected, using the OpenBSD preprocessor macro. This is because pledge() is an OpenBSD-specific system call and is not available on other operating systems.

Again, by calling pledge() with these promises, the httpd web server is restricted to only the operations needed to serve web pages, and other operations that could be exploited by attackers are disallowed.

DEV Community: Pawel Suchanecki

Generate ASCII Graphs of Cryptocurrency Prices in Python

Have you ever wanted to visualize the historical prices of a cryptocurrency in a fun and interactive way? Well, look no further! In this article, we'll explore how to generate an ASCII graph of cryptocurrency prices using Python.

Disclaimer:

Example

Raw Data

The Code

What does this code do?

How can you add it to your .bashrc or MOTD regenerator?

How does the code work?

Parsing Command-Line Arguments

Retrieving Historical Data

Generating an ASCII Graph

Listing All Cryptocurrencies Supported by Binance

Lambda Explained

Example Usage

Conclusion

Next read

Scalable, Producer-Consumer-based, Remote Log Monitor in Python

Intro

The data

The code

Explanation: Supporting Multiple Hosts (Producers)

Explanation: Consuming the output from multiple Producers

Explanation: How sleeping allows producers to reduce load

Summary

About API

Outro

How to Implement the Producer-Consumer Concurrency Design Pattern with asyncio Coroutines

Using yield Statement

About Fabric

Using asyncio.Queue()

UPDATE:

delta_data = [x for x in data if x not in old_d] ?

Break down of the code

Summary

OpenSSH 9.2: stricter default sandbox policy in OpenSSH client on OpenBSD

TL;DR

What is OpenBSD client sandboxing?

How does it work?

OpenSSH client pledge() example

OpenBSD httpd web server pledge() example

Using `yield` Statement

`delta_data = [x for x in data if x not in old_d]` ?

OpenBSD httpd web server `pledge()` example