DEV Community: Adwaith Rajesh

Calling Assembly function from Python (Part - maybe 0)

Adwaith Rajesh — Sun, 15 Jan 2023 18:01:42 +0000

Hi all, welcome back, it's been a long time since I've written a blog post, and it feels so good to be back doing it again.

So, today I wanna talk about a way of calling an ASM function from python, and this is part 0 (maybe),
why maybe? well the thing is that the way we are discussing today is more about ASM which is written as inline assembly and not
ASM that is written in a file. So I might (might) do that soon (I just haven't figured out a 'beautiful' way of doing that. I wish I was good at programming 😅.)

Coming back to today's topic, I will try my level best to explain every line of code that I write and make sure that you understand everything that's written here.

Let's begin 🤓

Let's have an objective, what are we building today?

We are going a build a function that adds two numbers but with a twist (yep, that's it). The twist is that
instead of the usual a + b we will do a + b + 1700. Why 1700, it's the first number that came to my mind.

Here is a list of all the files that we have to 'fill out'.

add.c
add.h
builder.py

`add.c`

The add.c file contains the function that python calls and the function that contains the ASM code that adds two numbers with a twist of course.

So, here is the code, and I will explain what's happening.

__attribute__((naked)) int add_asm(int a, int b) {
    __asm__(
        ".intel_syntax noprefix\n"

        "xor rax, rax\n"
        "add rax, rdi\n"
        "add rax, rsi\n"
        "add rax, 1700\n"  // add_asm twist
        "ret\n"

        ".att_syntax prefix\n");
}

int add_c(int a, int b) {
    return add_asm(a, b);
}

Let's "decode" what's happening.

The add_c function is just a simple wrapper function that calls the add_asm function. The add_c function is what we will be
calling from python.

__attribute__((naked)), you might have wondered what this is, well it's our way of instructing the C compiler in my case GCC
not to add any prologue or epilogue to the final compiled assembly. Or in other words, the ASM that I write must be the only ASM code that should exist when this function is compiled.

__asm__ allows us to write inline assembly in C.

Now let's understand the assembly code that's written. The first thing that we do is to tell the compiler that we will
be writing assembly in Intel Syntax (because I don't like AT&T syntax). In the end, we revert to its default.

Now, when it comes to adding we do the following steps.

xor rax, rax        ; set rax to 0
add rax, rdi        ; add rdi (parameter a) to rax
add rax, rsi        ; add rsi (parameter b) to rax
add rax, 1700       ; add 1700 to rax
ret                 ; return the control back to the caller

You might be wondering how I know where the parameters are stored, well it's just a convention that everyone follows. According to this convention, the return value of a function must be stored in
the rax register.

`add.h`

For simplicity's sake, I'm just gonna say that the add.h header file just contains the function declaration that we want to
call from python.

#ifndef AD_ADD_H
#define AD_ADD_H

int add_c(int, int);
#endif

The AD_ADD_H can be anything you want, the way I like to define is to have the first two letters of my first name followed by
the name of the header file.

`builder.py`

This file will contain all the instructions on how to build the python module that we will finally import the function from.

from cffi import FFI


CDEF = '''
int add_c(int, int);
'''

ffibuilder = FFI()
ffibuilder.cdef(CDEF)
ffibuilder.set_source(
    '_add_lib',
    '''
    #include "add.h"
    ''',
    include_dirs=['.'],
    sources=['add.c']
)
ffibuilder.compile(verbose=True)  # compile with verbose output

We will be using cffi to interface with the C code. It will build a Python wrapper for the C function based on the info that we provide.

You can install the package with the following command.

pip3 install cffi

Now, let's debunk the code.

First, we will create an instance of FFI() class, we will then pass the C decelerations that we will be using from python.

Then we will pass on the information required to build the python module.

The first parameter to set_source is the name of the module that it will build. So that finally when we import the module we can do this.

import _add_lib

The next parameter, you can think of it more like the extra code that we have to add to build the python module. In our case, and in most use cases, it's just the header files that we created.

The include_dirs argument takes a list of the file paths that cffi will look for the extra header files that we've added.

The sources argument contains a list of all the C files where it will search for the source of the function that we've declared.

Let's run it.

To compile run the following command.

python3 builder.py

We can test it using the python REPL.

>>> from _add_lib.lib import add_c
>>>
>>> add_c(10, 10)
1720
>>> add_c(0, 0)
1700
>>> add_c(0, -700)
1000
>>>

Yay, that works.

The End.

Well, that's it, if you have any doubts or questions feel free to ask me through the comments, DM, or mail.
If you want more complicated examples like using strings and whatnot, just comment down below and I might make a dev post on it.

Byeee... Happy Hacking 🤪

Loafang! An alternative to GrapQl?

Adwaith Rajesh — Sun, 21 Nov 2021 15:27:41 +0000

Full disclaimer, I made loafang as a way to kill my time and is still in development.
The current implementation that we will be discussing is written in python. And the example shown will be in python 3.9.
So, let's begin.

What does it aim to solve.

Well, technically it's more about readability and being more verbose in terms of how the query is written and also removing the need to use an external library to send a request from the client's perspective. The way loafang achieves this is by making use of JSON syntax as it's one of those file types that is supported by a lot of languages out of the box.

And it's way too much fun to do something like this. So.....

A sample query

{
    "GET:get-user": {
        "user --name ad": ["username", "email"],
        "posts --user-id ad34": ["latest"]
    }
}

Let's talk about the parts of the query.

"GET:get-user": {
        "user --name ad": ["username", "email"],
        "posts --user-id ad34": ["latest"]
    }

This is an execution block in terms of loafang terminology. You can have multiple execution blocks in the same loafang query (obviously)

GET:get-user

This is the block header, It specifies three things,

A Request method, in this case, it's GET (The schema is case sensitive),
A unique id that represents the block. (get-user)
A block property key. (It's optional)

These three things are arranged in the following order.
METHOD:id:property-key

"user --name ad": ["username", "email"]

This is a query that needs to be executed from the server. And it has three parts.

The head, here its user (this can be a database name or something like that).
The args (--name)
contents, here ["username", "email"]

Now, the contents type must be a list for GET and DELETE queries.
And, must be a dict for POST, PUT, and PATCH, as these three methods require you to update or add things.

Let's write some code.

Let's make a simple example use case for loafang with fastapi and tinydb

We will make a simple example where you can ADD, UPDATE, and GET data from the DB.
I won't let you remove your data from MY database.

Stuff we need

pip3 install loafang fastapi uvicorn tinydb

# app.py

from __future__ import annotations
from typing import Any, Union
from argparse import Namespace  # type hinting purposes only

from fastapi import FastAPI, HTTPException
from loafang import parse, Methods, MethodsError, QueryBuilder
from tinydb import TinyDB, Query

# what our query looks like
query = QueryBuilder("database")
query.add_argument("uname", type=str)

class RequestMethods(Methods):

    # class to handle/create a response for queries in a block
    def __init__(self) -> None:
        # you can have different query parser for different request methods
        self.get_query_parser = query
        self.post_query_parser = query
        self.put_query_parser = query

    def get(self, args: Namespace, contents: list[str]) -> Any:

        # our database only has fields for name, username and email.
        if not all(i in ["name", "email"] for i in contents):
            raise MethodsError(400, "Bad contents arguments")

        User = Query()
        db = TinyDB(args.database)

        db_ret = db.search(User.username == args.uname)
        if not db_ret:
            raise MethodsError(404, "User does not exists")

        else:
            user = db_ret[0]
            return {i: user[i] for i in contents}

    def post(self, args: Namespace, contents: dict[str, Any]) -> Any:

        if not all(i in contents for i in ["name", "email"]):
            raise MethodsError(400, "Bad contents arguments")

        User = Query()
        db = TinyDB(args.database)

        if db.search(User.username == args.uname):
            raise MethodsError(403, "User already exists")

        else:
            db.insert({"username": args.uname, **contents})
            return {"msg": f"user {args.uname} added."}

    def put(self, args: Namespace, contents: dict[str, Any]) -> Any:
        if not all(i in contents for i in ["name", "email"]):
            raise MethodsError(400, "Bad contents arguments")

        User = Query()
        db = TinyDB(args.database)

        user = db.search(User.username == args.uname)

        if not user:
            raise MethodsError(404, "User already exists")

        db.update(contents, User.username == args.uname)
        return {"msg": f"User updated successfully. {args.uname}"}


request_methods = RequestMethods()

# a vague representation of the loafang query
QueryType = Union[dict[str, Any], list[str]]


# initialize the server
app = FastAPI()

@app.get("/")
def read_root():
    return {"ping": "pong"}

@app.post("/loaf")
def loaf(data: QueryType):
    # the parsers gives you three things
    # The parsed data with all the output (None, if something goes wrong)
    # err code (only if something goes wrong, else None)
    # err message (only if something goes wrong else None)
    parsed_data, err, msg = parse(request_methods, data)

    return parsed_data if parsed_data else HTTPException(status_code=err, detail=msg)

Launch the server

uvicorn app:app

Now let's send a sample query to the loaf endpoint and see what happens.

The query

{
    "GET:get-user:pe": {
        "db.json --uname adwaith": ["name", "email"]
    },

    "POST:add-user": {
        "db.json --uname adwaith": {
            "name": "ad",
            "email": "adwaithrajesh3180@gmail.com"
        }
    },

    "PUT:update-user": {
        "db.json --uname adwaith | update-ad": {
            "name": "ad2",
            "email": "new-email@example.com"
        },

        "after": "get-user"
    }
}

The response

{
  "add-user": {
    "db.json --uname adwaith": {
      "msg": "user adwaith added."
    }
  },
  "update-user": {
    "update-ad": {
      "msg": "User updated successfully. adwaith"
    },
    "after": {
      "db.json --uname adwaith": {
        "name": "ad2",
        "email": "new-email@example.com"
      }
    }
  }
}

Let's analyze what the hell just happened here,

In the first execution block, you can see that there is a pe property key (currently that is the only one). It tells the parser to skip the block or "prevents the execution" of the block, the block can now be called from other blocks.

The second block is a POST block and has one query in it that adds a user with the name adwaith to the DB 'db.json'.

The third is a PUT block that has one query that updates a user with the name adwaith. But there is something special with that block. It has an after key. It tells the parser that after it executes the current block it needs to execute the block specified in the after.

The after key can only point to a block that has pe as its property key.
A block with pe as its property cannot have an after key

You might also notice that there is a pipe | in between the query. It specifies that anything after the pipe is an alias to the query. As you can see in the result, the key to a query's values is the query itself. So it might be hard at times to retrieve data using the query as the key.

So, that a brief intro to loafang, feel free to ask questions, use the package and if you have any errors, pls report it or ask me on discord.

GitHub: https://github.com/Adwaith-Rajesh/loafang

Full Docs: https://adwaith-rajesh.github.io/loafang/

Have a nice day.

Happy Coding.

Prevent accidental package installation to system Python.

Adwaith Rajesh — Thu, 04 Nov 2021 04:07:14 +0000

Ever happened to you, that one day you decided to start afresh Python project, and started to install all the packages, like pandas, NumPy, jupyter notebook, and then you realized that you've installed all of them in the system python rather than in a virtual environment. Yeah, you are not alone buddy.

Well, worry no more. Introducing pip-venv-sure. (I don't know what I was thinking when I named it.)

Installation and Setup

pip3 insall pip-venv-sure
alias pip3=pip3-venv-sure

Recommended to place the alias command in your .bashrc.

Now if you try to install a package without an activated venv this happens.

But if you have an activated venv it works as normal.

But, there are times when you've to install packages to the system Python in that case you can prevent pip-venv-sure from checking for a venv using the --allow-no-venv flag.

pip3 install any-template --allow-no-venv

Some other alternatives.

If you don't like to install the said package, there is a way around it.

You can set an environment variable like this

export PIP_REQUIRE_VIRTUALENV=true

The problem with it Is you've to unset or set this to false when you want to install packages globally.

Well, that's all for today, and Thanks for reading.

strip.pysondb a new JSON DB for python

Adwaith Rajesh — Mon, 09 Aug 2021 09:33:51 +0000

Today we will be talking about strip.pysondb, as the name suggest it's stripped down version of an existing JSON DB called pysondb.

We will discuss some issues with pysondb and how the strip.pysondb fixes those issues.

But first, some of the core features of strip.pysondb are.

No third party packages needed
No CLI
No ImageUtils
Use of python naming conventions in functions and classes.
Just the code required to perform CRUD operation on JSON DB.
Users can generate custom id's for their

Even though its called a stripped-down version of the original. There are some huge differences in the strip.pysondb works.

The data is not saved to file instantly, instead a methods needs to be called in order to do that.
The user is supposed to pass the keys / column name for the DB during initialization of the DB class.
Some of the new methods that are not derived from PysonDB are,
- pop
- commit
- load
- values
- set_id_generator
- id_exists

Let's talk about speed

Here is a sample code that continuously add 1000 values to the DB, and calculates the time it takes to add all the 1000 values

pysonDB

  pip3 install pysondb

  # test.py
  from time import time

  from pysondb import db

  a = db.getDb("test.json")

  t1 = time()
  for i in range(1000):
      a.add({"name": f"name{i}", "age": i})
  t2 = time()

  print(t2 - t1)

Output

  $ py test.py
  7.166365623474121

Yep, it takes 8 seconds to add 1000 values. If the DB already has a 1000 values then it takes 20 seconds.

strip.pysondb

  pip3 install strip.pysondb

  # test2.py
  from time import time

  from pysondb import DB

  db = DB(keys = ["name", "age"])

  t1 = time()
  for i in range(1000):
      db.add({"name": f"name{i}", "age": i})

  db.commit("test2.json")
  t2 = time()

  print(t2 - t1)

Output

  $ py test2.py
  0.009546756744384

As you can see it only takes 0.01 seconds to add 1000 values. It only takes 0.9 seconds to 100_000 values to the DB.

Click here to see the GItHub Repo and for more info on the usage of strip.pysondb Star it if you like it.

So next time you want to use a DB in your projects choose wisely and research all the available options.

Thank you for reading.

Onstro-DB a fast DB

Adwaith Rajesh — Mon, 19 Jul 2021 09:37:11 +0000

Installation

pip install onstro-db

Let's see how fast it is.

In this example, we can make a DB with 4 columns and in which 2 columns have a default value. Which are added during Runtime.
We will be adding 10,000 values to the DB and check how fast it is.

The test code

from onstrodb import OnstroDb
import time

test_schema = {
    "name": {"type": "str", "required": True},
    "age": {"type": "int", "required": True},
    "place": {"type": "str", "default": "canada"},
    "alive": {"type": "bool", "default": True}
}

db = OnstroDb(db_name="test", in_memory=True, schema=test_schema)


# generate the data to insert
data = []
for i in range(10_000):
    print(i, end="\r", flush=True)
    d = {"name": f"ad{i}", "age": i}
    data.append(d)


# insert the data into the DB
start = time.time()
db.add(data)
stop = time.time()

print("Elapsed Time: ", stop - start)

Here we first create all the data we want to insert and we time the insertion process.

Here are the results.

Elapsed Time:  1.077016830444336

Onstro-Db supports full CRUD operation, comes with a simple CLI, has a strict schema, and also prevents data duplication by default.

As you can see this thing is fast. To know more visit the Github repo here.

Star and follow the repo for future updates.

Open all your job related apps with one click.

Adwaith Rajesh — Mon, 21 Jun 2021 10:06:20 +0000

Don't let the word tkinter fool you, this is what the apps main page looks like.

This is the group edit page.

what does it do.

Allows the user give a list of all the apps (currently 5 apps are supported) related to a task and then opens all the apps with a single click. It can also run one bat script. If you are fancy and want to do something with it.

Installation

Click here to read more about installation. (Windows executable is available)

An easy and fun way to permanently store python objects.

Adwaith Rajesh — Tue, 15 Jun 2021 10:41:27 +0000

Object Bucket

An easy and fun way to store python objects.

Star it on GitHub

Description

Object Bucket is a python package that allows you to store python objects permanently in a more user friendly way.

Installation

The object-bucket package can be installed by using pip.

pip install object-bucket

Usage

Creating new bucket.

  from object_bucket import Bucket

  test_bucket = Bucket("name-of-the-bucket")

Adding droplets to the bucket, droplets are considered as objects that you want to save permanently.

  test_obj = [1, 2, 3, 4]
  test_bucket.add_droplet("droplet-name", test_obj)

Trying to add a droplet with the same name will cause an error.

Adding multiple droplets. To add multiple droplets you have to have a dictionary that contains all the names and objects of the droplet. To add the dictionary you can use the add_droplets method.

  droplets = {
        "one": 1,
        "two": 2,
        "three": [2, 3, 4]
    }

    test_bucket.add_droplets(droplets)

Modifying a droplet

  new_obj = {1: "a"}
  test_bucket.modify_droplet("droplet-name", new_obj)

Trying to modify a droplet that does not exists will cause an error.

Saving a bucket

All the things mentioned above will not be added or saved permanently, to do so it is necessary to save the bucket.

  test_bucket.save_bucket()

Retrieving values from a bucket.

 from object_bucket import Bucket
 test_bucket = Bucket("name-of-the-bucket")
 a = test.bucker.get_droplet("droplet-name")
 print(a)  # {1: "a"}

Trying to get a droplet that does not exists will cause an error.

Get all the runtime droplets

 drop1 = [1, 2, 3, 4]
 drop2 = "Hello"
 drop3 = {1: "a", 2: "b"}
 test_bucket.add_droplet("drop1", drop1)
 test_bucket.add_droplet("drop2", drop2)
 test_bucket.add_droplet("drop3", drop3)

 # to get all the droplets
 a = test_bucket.get_all_droplets()
 print(a)

 # output
 {"drop1": [1, 2, 3, 4], "drop2": "Hello", "drop3": {1: "a", 2: "b"}}

Deleting a bucket To delete the bucket and to clear the runtime storage of all the droplets.

 test_bucket.delete_bucket()

You can also delete a bucket using remove_bucket function

  from object_bucket import remove_bucket
  remove_bucket("name-of_bucket_to_be_removed", bucket_file_path="file-path-of-the-bucket")

Using the context manager.

It might be a hastle to remember to save to bucket, so you can use the context manager to avoid using the ""save_bucket"" method.

Note Using ""Bucket().delete_bucket"" inside the context
manager is useless as at the end the file will be saved automatically.

from object_bucket import Bucket

with Bucket("name-of-the-bucket") as b:
  # code to execute
  b.add_droplet("name", 1)
  # ...etc
  b.delete_bucket()  # wont work as the file will be again saved,
  # but the runtime contents will be cleared

Some more stuff

You can use the if statement to check whether a bucket is empty or not

  from object_bucket import Bucket
  t = Bucket("name")
  if t:
    print("Hello")  # -> does not print anything as bucket is empty

  t.add_droplet("demo", 1)
  if t:
    print("Hello 2")  # -> prints "hello 2" as the bucket has at least one droplet

To get the number of droplets in a bucket you can use the len method

  from object_bucket import Bucket
  t = Bucket("name")
  print(len(t)) # -> 0
  t.add_droplet("demo", [1, 2, 3])
  print(len(t)) # -> 1

This will work with any python object, including functions, classes, lambda functions ..etc

Storing python objects inside PysonDB

Adwaith Rajesh — Sun, 13 Jun 2021 02:14:27 +0000

Let's see how we can store python objects, this include strings, list, function, etc.., inside a DB like pysonDB

Consider I've function foo which takes an argument x and return x * (x + x). Which I want to store inside a DB.

def foo(x: float) -> float:
    return x * (x + x)

First things first we nee to convert this object to byte string. To do that we can use the pickle. module.

import pickle
byte_string = pickle.dumps(foo)

This byte string cannot be directly added to a PysonDB database, as byte strings are not JSON serializable. So a simple workaround will be to add quotes around the byte string which can be easily done like this.

obj_string = f"{byte_string}"

This string can be now added to the DB

from pysondb import db

a = db.getDb("test.json")
a.add({
    "name": "foo_function",
    "obj": obj_string
})

To get the object back we can do the following steps.

data = a.getBy({"name": "foo_function"})

Now data will look something like this.

[{'name': 'foo_function', 'obj': "b'\\x80\\x04\\x95\\x14\\x00\\x00\\x00\\x00\\x00\\x00\\x00\\x8c\\x08__main__\\x94\\x8c\\x03foo\\x94\\x93\\x94.'", 'id': 316182400052721056}]

To convert the obj string back to a callable function. We can do the following.

str_obj = data[0]["obj"]

Remember that str_obj is still a string and not a byte string, which is what we need, since the required byte string is inside this string, we can simply evaluate the string.

import ast
byte_obj = ast.literal_eval(str_obj)

To call the function we can do this.

call_obj = pickle.loads(byte_obj)
print(call_obj(3))

# output
18

Entire Code

import ast
import pickle

from pysondb import db

def foo(x: float) -> float:
    return x * (x + x)

byte_string = pickle.dumps(foo)
obj_string = f"{byte_string}"

a = db.getDb("test2.json")
a.add({
    "name": "foo_function",
    "obj": obj_string
})

data = a.getBy({"name": "foo_function"})

str_obj = data[0]["obj"]
byte_obj = ast.literal_eval(str_obj)

call_obj = pickle.loads(byte_obj)
print(call_obj(3))

So we have successfully stored a python object inside a DB. The steps are the same for all the objects like list or dict.

Activating a virtualenv from Python

Adwaith Rajesh — Fri, 11 Jun 2021 01:34:10 +0000

Spoiler alert, you cannot.

Through out this post we will discuss on how venv works, why you cannot activate a venv from python.

How does a venv work or gets activated.

Let's first go through a code snippet here.

@echo off

set "VIRTUAL_ENV=C:\Users\Adwaith\env"

if defined _OLD_VIRTUAL_PROMPT (
    set "PROMPT=%_OLD_VIRTUAL_PROMPT%"
) else (
    if not defined PROMPT (
        set "PROMPT=$P$G"
    )
    if not defined VIRTUAL_ENV_DISABLE_PROMPT (
        set "_OLD_VIRTUAL_PROMPT=%PROMPT%"
    )
)
if not defined VIRTUAL_ENV_DISABLE_PROMPT (
    set "ENV_PROMPT="
    if NOT DEFINED ENV_PROMPT (
        for %%d in ("%VIRTUAL_ENV%") do set "ENV_PROMPT=(%%~nxd) "
    )
    )
    set "PROMPT=%ENV_PROMPT%%PROMPT%"
)

REM Don't use () to avoid problems with them in %PATH%
if defined _OLD_VIRTUAL_PYTHONHOME goto ENDIFVHOME
    set "_OLD_VIRTUAL_PYTHONHOME=%PYTHONHOME%"
:ENDIFVHOME

set PYTHONHOME=

REM if defined _OLD_VIRTUAL_PATH (
if not defined _OLD_VIRTUAL_PATH goto ENDIFVPATH1
    set "PATH=%_OLD_VIRTUAL_PATH%"
:ENDIFVPATH1
REM ) else (
if defined _OLD_VIRTUAL_PATH goto ENDIFVPATH2
    set "_OLD_VIRTUAL_PATH=%PATH%"
:ENDIFVPATH2

set "PATH=%VIRTUAL_ENV%\Scripts;%PATH%"

This is the same code that gets executed when you run.

env\Scripts\activate

Let's break down the code a little bit and see whats happening.

In the first line it sets the value of VIRTUAL_ENV to the absolute path to the venv.
In the first and second if block it checks whether a PROMPT exists or not. The prompt is the name of the venv in (brackets), that appear when you activate a venv. If it already exists it changes its value to the current venv name, else it will define a new PROMPT.
In the third if block it checks whether a PYTHON_HOME is set, or
_OLD_VIRTUAL_PYTHONHOME is set. Or in simple terms, is their any other venv activated in the current session of the terminal. If yes, it changes it to the path of the current venv or sets news values to it.
Once all the variables are set, the venv is considered activated.

Why you cannot activate a venv from a python scripts.

From the above explanation It's clear that activating a venv is just a matter of changing the values of a bunch of environment variables.

When python code is executed a new child process starts from the current session of the terminal, this child process has a copy of all the environment variables that the terminal had when the process was created. So when a python scripts tries to activates a venv, It will set all the variable but in the child process and not in the parent process. So when the code execution is done the child process is killed and the venv is not activated in the main process or in the terminal.

This is the same result if you try to execute a bat script that can activate the venv from a python script. The bat script will get executed in the child process and not in the parent so it's changes will not be reflected in the parent process.

Some simple workarounds.

Now if you want to activate a venv from python script It's not possible. Let's say you are building a simple venv manager package in python there is a way to activate a venv from a package.

Add the script that can activate the venv in a folder.
In the setupy.py file add the following code.

    from setuptools import setup
    setup(
        ...
        scripts=["<folder-name>/test.bat"]
        ...
    )

When the package is installed the script can be called from the terminal.

  test  # this will execute the test script

Here the bat script is getting executed directly in parent process and not by a child process started by python. So the effects of running the bat script will persist even after the command execution is done.

Description

All the facts and code discussed here are things that I've found during my coding research and learning, somethings that I point out or explained about might be wrong.

correct me in the comment section if I'm wrong

Get meta info about python modules

Adwaith Rajesh — Thu, 10 Jun 2021 03:39:01 +0000

Get extra info about a python module, like imports, functions called inside other funcs etc.

Installation

pip install py-module-info

Why use this?

Well Idk, the inspect module in python requires you to import the module.
This package does not require you to import, but rather just looks at the python file like a normal text file and parses it. The problem with importing is that it may execute code that can harm your PC. So maybe ?

What does it do?

Get info about the imports used

# test.py

import json
import pprint as p
from typing import List as l
from json import load, dump as d
from py_module_info.main import ModuleInfo

To get the imported names use

from py_module_info import ModuleInfo
m = ModuleInfo("test.py")
imports = m.get_imports()
print(imports.get_imported_names())

# output
['json', 'pprint', 'List', 'load', 'dump', 'ModuleInfo']

# in order to get the alias names used
print(imports.get_imported_names(use_alias=True))
# output
['json', 'p', 'l', 'load', 'd', 'ModuleInfo']

To get the literal import string in the module use

from py_module_info import ModuleInfo
m = ModuleInfo("test.py")
imports = m.get_imports()
print(imports.get_import_strings())

# output
['import json', 'import pprint', 'from typing import List',
'from json import load', 'from json import dump', 'from py_module_info.main import ModuleInfo']

# use alias names insted
print(imports.get_import_strings(use_alias=True))

# output
['import json', 'import pprint as p', 'from typing import List as l',
'from json import load', 'from json import dump as d', 'from py_module_info.main import ModuleInfo']

Note

The imports will be split into individual imports:

 import test, json

will become

import test
import json

Get info about the function in the module.

# test.py

import json
from pprint import pprint

def foo():
    pprint({"Hello": "World"})


    with open("test.json") as f:
        print(json.load(f))

To get info about the functions use

from py_module_info import ModuleInfo

m = ModuleInfo("test.py")
print(m.get_funcs_info())

# output
{'foo': {'args': [], 'defaults': [], 'arg_count': 0, 'calls': ['json.load(f)', 'open("test.json")', 'pprint({"Hello": "World"})', 'print(json.load(f))']}}

The output includes the args passed in function, the defaults value for the the arguments, the argument count, and all the function calls that happened inside the function.

The function calls are all expanded and includes the entire call string.

In order to get just the function calls use

from py_module_info import ModuleInfo

m = ModuleInfo("test.py")
print(m.get_funcs_info(only_func_name=True))

# output
{'foo': {'args': [], 'defaults': [], 'arg_count': 0, 'calls': ['load', 'open', 'pprint', 'print']}}

Get info about the classes in a module

# test.py
class Foo(A.Test):

    def __init__(self, name):
        self.name = name

    def n():
        pass


class Bar():

    def a():
        print('Hello')

To get meta info about the classes in the module use

from py_module_info import ModuleInfo

m = ModuleInfo("test.py")
print(m.get_classes_info())

# output
{'Foo': {'bases': ['A.Test'], 'methods': {'__init__': {'args': ['self', 'name'], 'defaults': [], 'arg_count': 2, 'calls': []}, 'n': {'args': [], 'defaults': [], 'arg_count': 0, 
'calls': []}}}, 'Bar': {'bases': [], 'methods': {'a': {'args': [], 'defaults': [], 'arg_count': 0, 'calls': ["print('Hello')"]}}}}

Currently the output includes the information about the base classes and the info about different methods in the class.