DEV Community: Madhav Jha

Deep Learning notes

Madhav Jha — Tue, 09 Apr 2024 02:22:51 +0000

An unusual intro to deep learning.

The only primitive operation that one needs to understand to get started on in Deep Learning is dot product: vectors r and x of same dimension d can be dotted to produce a scalar $⟨r,x⟩=∑i∈[d]rixi\langle r,x \rangle = \sum_{i\in [d]}{r_i x_i}$ .

To do this operation in parallel across multiple vectors $r$ , one can write it as a matrix vector product. One way to think about these dot product is that each vector r is a measurement of (an unknown) signal x. For example if vector r is [1, 0, 0, 0], then it is measuring the first coordinate of x. Likewise vector r [0.25, 0.25, 0.25, 0.25] is measuring the average value of coordinates of x. Notice we can make many measurements of x by doing this matrix vector multiplication where each measurement is independent (and possibly redundant) and done in parallel.

As an aside, are you wondering what exactly is a dot product geometrically? It is simply a (scaled) projection of the high dimensional signal along a "line". The line could be x-axis (e.g. when r i s [1, 0, 0, 0]) or some 45-degree line passing through origin.

So given an "unknown" x, one is interested in asking the question, hey which direction should I measure x in to get maximum bang for buck. More on this later.

Suppose we have an image represented as a vector x. To make things concrete, imagine it is a 10x10 image with each value in [0, 1] representing the grayscale measure normalized to lie in [0, 1]. This image can be represented as a 100-dim column vector. We want to deduce if it is an image of a dog or a cat.

Mini World for LLMs

Madhav Jha — Tue, 09 Apr 2024 02:09:02 +0000

What would a mini world for LLM look like from training perspective?

Digital Ocean Getting Started

Madhav Jha — Wed, 20 Mar 2024 00:15:21 +0000

The first thing to do is generate an SSH key as follows.

ssh-keygen -t ed25519

NextJS self hosting

Madhav Jha — Mon, 18 Mar 2024 17:06:02 +0000

Getting started.

npx create-next-app@latest
cd my-app
npm run dev

Startup Idea: Chat with your database

Madhav Jha — Wed, 21 Feb 2024 01:15:25 +0000

Here's a simple idea. Allow users to chat with their database in natural language.

Deep Learning Books

Madhav Jha — Fri, 16 Feb 2024 20:04:26 +0000

Understanding Deep Learning
by Simon J.D. Prince

Chainlit Quick Refresher

Madhav Jha — Tue, 05 Dec 2023 21:06:12 +0000

One of the best places to prototype an OpenAI's ChatGPT API based app is chainlit.

Let us make sure we have a python (> 3.8 ) environment and latest dependencies installed. I recommend using conda.

conda create -n conda-py-3.11-env2 python=3.11

pip install chainlit openai

Then here's a quick way to test OpenAI. First make sure to create a .env file with the following content: OPENAI_API_KEY=sk-XXX.

from openai import OpenAI
from dotenv import load_dotenv

load_dotenv()

client = OpenAI()

completion = client.chat.completions.create(
  model="gpt-3.5-turbo",
  messages=[
    {"role": "system", "content": "You are a helpful assistant"},
    {"role": "user", "content": "What is the capital of France?"},
  ]
)
print(completion.choices[0].message)

Here's another quick snippet to test OpenAI Visual model.

from openai import OpenAI
from dotenv import load_dotenv

load_dotenv()

client = OpenAI()

response = client.chat.completions.create(
  model="gpt-4-vision-preview",
  messages=[
    {
      "role": "user",
      "content": [
        {"type": "text", "text": "What’s in this image?"},
        {
          "type": "image_url",
          "image_url": {
            "url": "https://upload.wikimedia.org/wikipedia/commons/thumb/d/dd/Gfp-wisconsin-madison-the-nature-boardwalk.jpg/2560px-Gfp-wisconsin-madison-the-nature-boardwalk.jpg",
          },
        },
      ],
    }
  ],
  max_tokens=300,
)

print(response.choices[0])

Finally here's a minimal code to get started on chainlit which can be run as follows: chainlit run app.py -w.

import chainlit as cl
from openai import AsyncOpenAI
import os

import pdb

client = AsyncOpenAI(api_key=os.environ["OPENAI_API_KEY"])

settings = {
    "model": "gpt-3.5-turbo",
    "temperature": 0.0,
}


@cl.on_chat_start
def start_chat():
    cl.user_session.set(
        "message_history",
        [{"role": "system", "content": "You are a helpful assistant"}],
    )


@cl.on_message
async def main(message: cl.Message):
    print(message)
    pdb.set_trace()
    message_history = cl.user_session.get("message_history")
    message_history.append({"role": "user", "content": message.content})

    msg = cl.Message(content="")
    await msg.send()

    stream = await client.chat.completions.create(
        messages=message_history, stream=True, **settings
    )

    async for part in stream:
        if token := part.choices[0].delta.content or "":
            await msg.stream_token(token)
    message_history.append({"role": "assistant", "content": msg.content})
    await msg.update()

Here is how to display an image back.

import chainlit as cl

@cl.on_message
async def on_message(msg: cl.Message):
    if not msg.elements:
        await cl.Message(content="No file attached").send()
        return

    # Processing images exclusively
    images = [file for file in msg.elements if "image" in file.mime]

    if images:
        for image in images:
            await cl.Message(content=f"Received image", elements=[image]).send()
    else:
        await cl.Message(content="No image found in the message").send()

Pytest Quick Refresher

Madhav Jha — Sun, 23 Jul 2023 23:45:39 +0000

Welcome to the PyTest refresher guide. This guide will provide an overview of the pytest library, focusing mainly on its usage for writing and managing tests, especially in relation to pytest fixtures.

Overview of PyTest

PyTest is a full-featured testing tool for Python that supports simple unit tests to complex functional testing. It allows for more readable and easier-to-write tests by using Python's assert keyword directly, without any wrappers around it. Its auto-discovery feature makes it easier to set up and run tests, and its detailed output for failed tests makes debugging a smoother process.

PyTest Fixtures

One of the most powerful features of pytest is its fixture system. A fixture is a function that is run before (and sometimes after) the tests. It is used to set up some state or data that will be used by the tests. Fixtures are modular and can be created to serve specific tests or reused across multiple tests, modules, or even the entire project.

Here's a simple example of a fixture:

import pytest

@pytest.fixture
def example_data():
    return "some data"

In this example, example_data is a fixture that simply returns a string. This fixture can be used in any test that requires this string.

Using Fixtures in Tests

To use a fixture in a test, you simply need to include an argument in your test function with the same name as the fixture:

def test_example(example_data):
    assert example_data == "some data"

In the above test, pytest sees that example_data is a fixture and therefore calls the example_data fixture function and passes the returned value into the test function.

This system allows for powerful modular testing setups and can be used to represent things like database connections, data, mock objects, and state setup/teardown.

Parameterizing Fixtures

Another powerful feature of pytest is the ability to parameterize fixtures. This allows you to run a test multiple times with different inputs from the fixture. Here's an example:

import pytest

@pytest.fixture(params=[0, 1, 2])
def example_data(request):
    return request.param

def test_example(example_data):
    assert 0 <= example_data <= 2

In this example, the test test_example will be run three times, once for each value in params. The current value is accessed via the request.param attribute.

Fixture Scope

By default, pytest will call a fixture function for every test that uses it, creating new fixture data for each test. However, pytest allows you to control the scope of a fixture using the @pytest.fixture(scope='...') decorator. The available scopes are:

function (default): Run the fixture function before each test that uses it.
class: Run the fixture function once per test class, regardless of how many test methods use it.
module: Run the fixture function once per module, regardless of how many tests or test classes use it.
package: Run the fixture function once per package.
session: Run the fixture function only once per session (i.e., per test run).

For example, here's how to create a fixture with module scope:

import pytest

@pytest.fixture(scope="module")
def example_data():
    return "some data"

In this case, example_data will be called only once per module, and the same data will be used for every test in the module that uses the fixture.

Managing Multiple PyTest Fixtures

When dealing with multiple fixtures in PyTest, the order in which they're declared in your test function does not dictate the order of their execution. PyTest manages each fixture independently, setting up and tearing down each in the appropriate order.

import pytest

@pytest.fixture
def fixture_a():
    print("Setting up fixture_a")
    yield
    print("Tearing down fixture_a")

@pytest.fixture
def fixture_b():
    print("Setting up fixture_b")
    yield
    print("Tearing down fixture_b")

def test_example(fixture_b, fixture_a):
    print("Running test_example")

In this scenario, the print statements will execute in the following sequence:

Setting up fixture_a
Setting up fixture_b
Running test_example
Tearing down fixture_b
Tearing down fixture_a

Despite fixture_b being listed first in the test_example function arguments, pytest orchestrates the setup of all fixtures before executing the test. The teardown of fixtures occurs in the reverse order of their setup, ensuring a smooth handling of dependencies between fixtures. This remains true even when the fixtures share the same scope. If the fixtures have differing scopes (e.g., session vs function), the fixtures with broader scopes are set up prior to those with narrower scopes.

Mixing Fixture and Non-Fixture Arguments

A test function can indeed have a mix of fixture and non-fixture arguments. When doing so, the non-fixture arguments must be passed via pytest's mark.parametrize decorator to feed in the values. Here's an example:

import pytest

@pytest.fixture
def fixture_a():
    return "fixture_a value"

@pytest.mark.parametrize("param", ["value1", "value2"])
def test_example(fixture_a, param):
    print("Running test_example with", fixture_a, "and", param)

In this example, the test function test_example uses a fixture fixture_a and a parameterized non-fixture argument param. PyTest will run test_example twice, once with param as "value1" and once with "value2", setting up and tearing down fixture_a for each run.

This allows for very flexible and powerful test setups, combining the benefits of both fixtures and parameterized inputs.

Using Fixture Instances Across Multiple Fixtures and Tests

When working with PyTest fixtures, you may often find yourself in situations where you need to share an instance of an object between multiple fixtures or tests. PyTest makes this easy and ensures that the same instance is used across your tests, promoting consistency and reducing redundancy.

Consider an example where we have a Database class and a Store class. The Store class uses an instance of the Database class.

class Database:
    def __init__(self):
        self.data = "database data"

class Store:
    def __init__(self, database):
        self.database = database
        self.data = "store data"

We can create PyTest fixtures for these classes and use the instance of Database in both the Store fixture and the test function.

import pytest

@pytest.fixture
def context():
    print("Setting up context")
    return Database()

@pytest.fixture
def store(context):
    print("Setting up store with context:", context.data)
    return Store(context)

def test_example(store, context):
    print("Running test_example")
    assert store.database is context

In the above code, the context fixture sets up and returns an instance of the Database class. This instance is then used in the store fixture to set up a Store instance. In the test_example function, we assert that the Database instance within the Store instance (store.database) is the same as the context instance. This assertion will pass because PyTest ensures that the store fixture and the test_example function receive the same instance of the context fixture.

This example illustrates how PyTest manages instances and dependencies, allowing us to create sophisticated and consistent test scenarios. The ability to share and manage instances across fixtures and tests is a powerful feature that makes PyTest an excellent choice for testing in Python.

Common Patterns in PyTest

Parameterized testing

PyTest allows you to parameterize tests so that you can run the same test multiple times with different arguments.

import pytest

@pytest.mark.parametrize("test_input,expected", [("input1", "output1"), ("input2", "output2")])
def test_eval(test_input, expected):
    assert eval(test_input) == expected

In the example above, test_eval will run twice, once with test_input="input1" and expected="output1", and once with test_input="input2" and expected="output2".

Exception Testing

Testing that certain code raises an exception can be done using pytest.raises as a context manager:

import pytest

def test_zero_division():
    with pytest.raises(ZeroDivisionError):
        1 / 0

In the above example, the test will pass if 1 / 0 raises a ZeroDivisionError, and fail otherwise.

Skipping or Expecting Test Failures

You can skip tests or expect them to fail under certain conditions:

import pytest
import sys

@pytest.mark.skip(reason="no way of currently testing this")
def test_the_unknown():
    ...

@pytest.mark.skipif(sys.version_info < (3,7), reason="requires python3.7 or higher")
def test_function():
    ...

@pytest.mark.xfail
def test_function():
    ...

In the examples above, test_the_unknown is unconditionally skipped, test_function is skipped if the Python version is lower than 3.7, and another test_function is expected to fail.

Protobuf Quick Refresher

Madhav Jha — Sun, 23 Jul 2023 22:07:11 +0000

Protocol Buffers, for those needing a quick refresher, is a language-agnostic binary format developed by Google. The format is leveraged to structure and store data, and is quite popular due to its compact size and fast serialization/deserialization properties.

In our day-to-day work, we use protobufs to define message structures and services, and the protobuf compiler protoc generates data access classes for us in our language of choice, which in our case is Python.

Protobuf Schema Definition (.proto files)

Let's start by writing a .proto file, which will be the blueprint for our data.

syntax = "proto3";

package tutorial;

message Person {
  string name = 1;
  int32 id = 2;
  string email = 3;

  enum PhoneType {
    MOBILE = 0;
    HOME = 1;
    WORK = 2;
  }

  message PhoneNumber {
    string number = 1;
    PhoneType type = 2;
  }

  repeated PhoneNumber phones = 4;
}

In the example above, Person is a message type, with various fields like name, id, and email. Notice the number assignment to the fields, these numbers are used to identify your fields in the message binary format, they won't be reassigned.

We've also defined an enum PhoneType and a nested message PhoneNumber within Person, which is then used in the repeated field phones (equivalent to a list in Python).

Generating Python Code

Once we have our .proto file, we can generate Python code using the protoc compiler.

protoc --python_out=. ./person.proto

This generates a person_pb2.py file which includes Python classes for our defined messages, enums, and services.

Serializing and Deserializing Data

Next, let's create, serialize, and then deserialize a Person message in Python.

import person_pb2

# Create a new Person
person = person_pb2.Person()
person.name = "John Doe"
person.id = 1234
person.email = "jdoe@example.com"

phone = person.phones.add()
phone.number = "555-4321"
phone.type = person_pb2.Person.HOME

# Serialize to a byte string
person_bytes = person.SerializeToString()

# Deserialize to a new person
new_person = person_pb2.Person()
new_person.ParseFromString(person_bytes)

Here, we've created an instance of Person, filled in its fields, then serialized it to a byte string using SerializeToString(). Then, we've taken that byte string and deserialized it back into a Person object using ParseFromString().

Reading and Writing to Files

You can also write these byte strings directly to files or read them from files:

# Write to a file
with open("person.bin", "wb") as fd:
    fd.write(person_bytes)

# Read from a file
with open("person.bin", "rb") as fd:
    new_person = person_pb2.Person()
    new_person.ParseFromString(fd.read())

Optional Fields and Default Values

In protobuf 3, all fields are optional, and fields not set take their default values. For scalar types (like int32, bool, string), default values are the "zero value" for the type - 0 for integers, false for bools, and empty string for strings. Enum fields have a default value of the first value listed in the enum's type definition.

Here's an example:

syntax = "proto3";

message Weather {
  string location = 1;
  int32 temperature = 2;
  bool is_raining = 3;
}

In Python:

import weather_pb2

weather = weather_pb2.Weather()
weather.location = "San Francisco"

print(weather.temperature)  # prints: 0
print(weather.is_raining)  # prints: False

In the example, we didn't set temperature or is_raining, but they still have default values.

Upgradeability

Protobuf shines when it comes to upgradeability, i.e., the ability to evolve your data structures over time without breaking existing systems. Essentially, you can add new fields to your messages or stop using old ones, and your protobuf data will remain backwards and forwards compatible.

Here's a simple example of a Person message:

Version 1

Let's say we start with this message definition:

syntax = "proto3";

message Person {
  string name = 1;
}

And in our Python code, we create a Person and serialize it:

import person_pb2

person = person_pb2.Person()
person.name = "John Doe"
person_bytes = person.SerializeToString()

Version 2

Then, at a later time, we update our message to include a new field age:

syntax = "proto3";

message Person {
  string name = 1;
  int32 age = 2;  // New field
}

Even though our Python code doesn't know about the age field yet, it can still deserialize the Person bytes:

old_person = person_pb2.Person()
old_person.ParseFromString(person_bytes)
print(old_person.name)  # prints: John Doe

This example illustrates backward compatibility, where newer message versions can be read by older software. The older version simply ignores fields it doesn't recognize.

Version 3

Let's say we then decide to remove the age field:

syntax = "proto3";

message Person {
  string name = 1;  // 'age' field removed
}

And we serialize a new Person object:

new_person = person_pb2.Person()
new_person.name = "Jane Doe"
new_person_bytes = new_person.SerializeToString()

The Python code from version 1 (which expects the name field and doesn't know about age) can still read this:

v1_person = person_pb2.Person()
v1_person.ParseFromString(new_person_bytes)
print(v1_person.name)  # prints: Jane Doe

This illustrates forward compatibility, where older message versions can be read by newer software.

Key takeaway: Whenever you change a message type by adding or removing fields, be mindful not to reuse the old field numbers. This will ensure that your updated code maintains compatibility with older versions.

Remote Procedure Calls (RPCs) and Services in Protobuf

In addition to defining message types, Protobuf also allows us to define services, which specify a set of RPCs (Remote Procedure Calls) that can be implemented on a server and called on a client. This makes Protobuf a great choice for designing the interface of microservices.

Here's how we could define a simple service:

syntax = "proto3";

message HelloRequest {
  string greeting = 1;
}

message HelloResponse {
  string reply = 1;
}

service Greeter {
  rpc SayHello (HelloRequest) returns (HelloResponse);
}

In this .proto file, we define a Greeter service with a single RPC SayHello. This RPC takes a HelloRequest message and returns a HelloResponse message.

To generate service interfaces and bindings for Python, we need to use the grpc_tools package, which is an extension of protoc with GRPC-specific features.

python -m grpc_tools.protoc -I. --python_out=. --grpc_python_out=. ./greeter.proto

This will generate a greeter_pb2.py file (as before), but also a greeter_pb2_grpc.py file, which includes GRPC-specific code.

Now we can implement this service on the server side:

import grpc
from concurrent import futures
import greeter_pb2
import greeter_pb2_grpc

class Greeter(greeter_pb2_grpc.GreeterServicer):
    def SayHello(self, request, context):
        return greeter_pb2.HelloResponse(reply='Hello, ' + request.greeting)

server = grpc.server(futures.ThreadPoolExecutor(max_workers=10))
greeter_pb2_grpc.add_GreeterServicer_to_server(Greeter(), server)
server.add_insecure_port('[::]:50051')
server.start()
# keep running...

And call it from the client side:

import grpc
import greeter_pb2
import greeter_pb2_grpc

channel = grpc.insecure_channel('localhost:50051')
stub = greeter_pb2_grpc.GreeterStub(channel)
response = stub.SayHello(greeter_pb2.HelloRequest(greeting='world'))
print(response.reply)  # prints: Hello, world

This way, we can define both the structure of our data and the interface of our services using Protobuf, keeping our microservices clean, consistent, and easy to work with.

Conclusion

You can refer to the Protocol Buffers Language Guide for more detailed information.

Dockerized postgres for local development

Madhav Jha — Sun, 23 Jul 2023 20:10:08 +0000

This post describes a quick setup which allows one to develop a postgres sql backed application locally. (This is taken from this eggheadio video.)

Create a docker-compose.yml file as follows.

version: "3.8"
services:
  db:
    image: "postgres:12"
    ports:
      - "54320:5432"
    volumes:
      - ./pgdata:/var/lib/postgresql/data
    environment:
      - POSTGRES_USER=alice
      - POSTGRES_PASSWORD=wonderland
      - POSTGRES_DB=myawesomedb

Create a local folder which will persist data and start the service.

mkdir pgdata
docker-compose up -d

Once it is up and running, verify you can log into postgres shell.

docker-compose run db bash

Inside the docker container run the following and enter password wonderland defined above.

psql --host=db --username=alice --dbname=myawesomedb

You should see psql shell. You can exit by pressing Ctrl+D twice.

At this point most likely you already have prisma setup locally by some variation of the following commands.

npx prisma init
npx prisma generate

Now you can run the following command to create and apply migrations.

npx prisma migrate dev --name init

Bleeding edge NextJS and Supabase stack

Madhav Jha — Sun, 16 Apr 2023 00:23:55 +0000

This post extends the original NextJS Blog post and adds Supabase.

Task runner for NextJS

Madhav Jha — Sat, 15 Apr 2023 17:36:39 +0000

Vercel is the most popular platform to spin up an app these days. Even though it has Cron like features, one missing piece is background task runner.

In looking for solutions, I found two shiny looking options.

Ultimately I decided to go with https://www.inngest.com/ because of it's open source ethos and the ability to run it locally.

https://github.com/maddyonline/coffee-app

There are some nuances about running it in dev mode vs staging mode vs prod mode. But overall pretty doable.