Learnings from GenAI on AWS at Deloitte workshop

Noting in 4K — Tue, 04 Jun 2024 03:18:45 +0000

I attended an in-personal workshop provided by Deloitte and AWS for NZ TechWeek24 on 22nd of May, noted down some key points that I probably can learn further with hands-on projects later.

Key concepts:

LLMOps
Considerations for shortlisting LLMs
Hallucination & Retrieval-Augmented Generation (RAG) pattern
Embeddings
Conversational Buffer Memory
Prompt Engineering Techniques
Fine Tuning (just lightly touched)

Some use cases in Deloitte we went through:

Customer support GenAI POC - understand customer query, extract relevant parts, draft email/slack responses (100% consistency of response msgs), and then provides links to knowledge base - 25% decreased request handling time
Knowledge Base Summarisation for Chorus - more to read
Query Structured Data from internal supported vector data store - using the same stack/tools we used in labs below

The stack and tools we used in the labs:

Python boto3
Amazon Bedrock - fully managed service for using foundation models from Amazon and third parties
LangChain - Python and JS libraries, provides convenient functions for interacting with Amazon Bedrock’s models and related services like vector databases
Streamlit - quickly creates web UI from Python without much frontend skills, great for POCs (Streamlit API Reference)
Amazon Titan Embeddings - converts natural language text into numerical representations for later use cases such as searching or comparing semantic similarity

If you are interested in GenAI on AWS, there are a few skill builder free labs for AI Readiness to explore around.

Troubleshoot dependencies in Pipfile/pipenv

Noting in 4K — Thu, 09 May 2024 03:05:00 +0000

The Docker build failed in one of my projects on building pip wheel. As a Python newbie, I was not sure what it refers to, all I got is a dependency build stack trace that ends with AttributeError: cython_sources. I assume this is one of the packages hidden in the pip dependency tree. Eventually I managed to locate it but it took me for a while, thus I think it might be helpful to note down the troubleshooting process that I went through - at least for future me who has gold fish memory.

First I scratched the pip surface:

pip list to print the installed dependencies, got a looong list but nothing exactly matched the errored source. (Also tried pip freeze but didn't get any luck.)
searched in Pipfile.lock file for cython but nothing popped up.

Then I went to Uncle Google for cython and got this issue from last year:
https://github.com/yaml/pyyaml/issues/724
Good news - it's closed!

From this issue, I learnt that cython is a dependency for pyyaml. I probably can look for this thing instead.

I figured that we can install and print out the pip dependency tree from the pipfile project, some Q&As on Stack Overflow, and the helpful manual of pipenv cli.

pipenv shell to spawn a shell within the virtualenv;
pipenv verify to ensure the Pipfile.lock is up-to-date;
pipenv sync -d to install all packages and dev packages specified in the Pipfile.lock;
pipenv graph to display currently installed dependency tree.

MY Pipfile:


[[source]]
url = "https://pypi.python.org/simple"
verify_ssl = true
name = "pypi"

[packages]
flask = "*"
requests = "*"

[requires]
python_version = "3.10"

[dev-packages]
pre-commit = "*"
black = "*"

The dependency tree I got:


$ pipenv graph
black==22.10.0
├── click [required: >=8.0.0, installed: 8.1.3]
├── mypy-extensions [required: >=0.4.3, installed: 0.4.3]
├── pathspec [required: >=0.9.0, installed: 0.10.1]
├── platformdirs [required: >=2, installed: 2.5.2]
└── tomli [required: >=1.1.0, installed: 2.0.1]
Flask==2.2.2
├── click [required: >=8.0, installed: 8.1.3]
├── itsdangerous [required: >=2.0, installed: 2.1.2]
├── Jinja2 [required: >=3.0, installed: 3.1.2]
│   └── MarkupSafe [required: >=2.0, installed: 2.1.1]
└── Werkzeug [required: >=2.2.2, installed: 2.2.2]
    └── MarkupSafe [required: >=2.1.1, installed: 2.1.1]
pre-commit==2.20.0
├── cfgv [required: >=2.0.0, installed: 3.3.1]
├── identify [required: >=1.0.0, installed: 2.5.7]
├── nodeenv [required: >=0.11.1, installed: 1.7.0]
│   └── setuptools [required: Any, installed: 65.5.0]
├── PyYAML [required: >=5.1, installed: 6.0]
├── toml [required: Any, installed: 0.10.2]
└── virtualenv [required: >=20.0.8, installed: 20.16.5]
    ├── distlib [required: >=0.3.5,<1, installed: 0.3.6]
    ├── filelock [required: >=3.4.1,<4, installed: 3.8.0]
    └── platformdirs [required: >=2.4,<3, installed: 2.5.2]
requests==2.28.1
├── certifi [required: >=2017.4.17, installed: 2022.9.24]
├── charset-normalizer [required: >=2,<3, installed: 2.1.1]
├── idna [required: >=2.5,<4, installed: 3.4]
└── urllib3 [required: >=1.21.1,<1.27, installed: 1.26.12]

Finally I found PyYAML is one of the dependencies of the pre-commit dev packages.

What solved my issue is to re-sync Pipfile.lock file from Pipfile, even tho I didn't update anything in it. Because I use wildcard * to pull the latest version for dev packages, when I do pipenv install, there are many dependency changes in the lock file. PyYAML has bumped from 6.0 to 6.0.1 - and this fixed the docker build issue. Yay!

DEV Community: Noting in 4K

Learnings from GenAI on AWS at Deloitte workshop

Key concepts:

Some use cases in Deloitte we went through:

The stack and tools we used in the labs:

Troubleshoot dependencies in Pipfile/pipenv