DEV Community: Davide Fornelli

Running LLMs on Android: A Step-by-Step Guide

Davide Fornelli — Fri, 10 May 2024 20:41:31 +0000

In this blog post, we'll explore how to install and run the Ollama language model on an Android device using Termux, a powerful terminal emulator. This tutorial is designed for users who wish to leverage the capabilities of large language models directly on their mobile devices without the need for a desktop environment.

Step 1: Install F-Droid

F-Droid is an installable catalogue of FOSS (Free and Open Source Software) applications for the Android platform. Download and install the F-Droid app from https://f-droid.org/.

Step 2: Install Termux

Once F-Droid is installed, open it and search for Termux. Install Termux, which will serve as your Linux environment on Android.

Step 3: Update and Upgrade Packages

Open Termux and update its package repository to ensure you have access to the latest software versions:

pkg update
pkg upgrade

Step 4: Install Proot-Distro and Debian

Proot-Distro allows you to run different Linux distributions within Termux. Install it using:

pkg install proot-distro

pd install debian

For more details, visit the Proot-Distro GitHub page.

Step 5: Login to a Linux Distribution

For this tutorial, we will use Debian. Log in to your Debian environment with:

pd login debian

Step 6: Install Tmux

Tmux is a terminal multiplexer that allows you to run multiple terminal sessions simultaneously:

apt update
apt upgrade
apt install tmux

Step 7: Install Ollama

Within the Debian environment, download and install Ollama:

curl -fsSL https://ollama.com/install.sh | sh

This script installs Ollama and its dependencies.

Step 8: Create a New Tmux Session

Start a new tmux session dedicated to running Ollama:

tmux new -s llm

Step 9: Run Ollama Server

In the tmux session, start the Ollama server:

ollama serve

Step 10: Create a New Pane in Tmux

Split the current window into two panes by pressing Ctrl+b ".

Step 11: Run an Ollama Model

In the new pane, run a specific Ollama model. For example, to run the Gemma 2B model:

ollama run gemma:2b

Depending on your device, you can run also run phi3:

ollama run phi3

Step 12: Test the Model

Once the model is running, you can interact with it directly from the terminal. Enter prompts and observe the model's responses, effectively using your Android device as a powerful AI tool.

Conclusion

This tutorial provides a foundation for exploring further applications of LLMs on mobile devices, opening up possibilities for mobile-based AI development and experimentation.

Summarize a text with Azure Text Analytics in Python

Davide Fornelli — Thu, 11 Nov 2021 06:00:00 +0000

Source code

List of used gists.

Chef

Ingredients

Example

The following example is based on specific commits:

Chef
- commit: ac2a18c678a457b59535272a8d95cdc0c58ea02c
Ingredients
- commit: b01c0ab18d3d50d7e2836b71e22749e805fadac6

Notebook

%pip install httpimport


# - httpimport

# Description:
# Summarize a list of texts with Azure Text Analytics

import httpimport

# GitHub username
gist_user = 'Comm3'


gist_chef_id = '1bc116f05d09e598a1a2dcfbb0e2fc22'


gist_url_chef = f"https://gist.githubusercontent.com/{gist_user}/{gist_chef_id}/raw"
with httpimport.remote_repo(['chef'], gist_url_chef):
    import chef


gist_ingredients_id = '5c75b7cdea330d15dcd93adbb08648c3'
gist_ingredients_url = f"https://gist.githubusercontent.com/{gist_user}/{gist_ingredients_id}/raw"

chef.process_ingredients(
    gist_ids=[(gist_ingredients_id, ['az_cs_summarization.py'])]
)
with httpimport.remote_repo(['az_cs_summarization'], gist_ingredients_url):
    import az_cs_summarization


Requirement already satisfied: azure-ai-textanalytics==5.2.0b2 in /home/daforne/.pyenv/versions/3.7.9/envs/global379/lib/python3.7/site-packages (5.2.0b2)
Requirement already satisfied: azure-common~=1.1 in /home/daforne/.pyenv/versions/3.7.9/envs/global379/lib/python3.7/site-packages (from azure-ai-textanalytics==5.2.0b2) (1.1.27)
Requirement already satisfied: six>=1.11.0 in /home/daforne/.pyenv/versions/3.7.9/envs/global379/lib/python3.7/site-packages (from azure-ai-textanalytics==5.2.0b2) (1.16.0)
Requirement already satisfied: azure-core<2.0.0,>=1.19.1 in /home/daforne/.pyenv/versions/3.7.9/envs/global379/lib/python3.7/site-packages (from azure-ai-textanalytics==5.2.0b2) (1.20.1)
Requirement already satisfied: msrest>=0.6.21 in /home/daforne/.pyenv/versions/3.7.9/envs/global379/lib/python3.7/site-packages (from azure-ai-textanalytics==5.2.0b2) (0.6.21)
Requirement already satisfied: requests>=2.18.4 in /home/daforne/.pyenv/versions/3.7.9/envs/global379/lib/python3.7/site-packages (from azure-core<2.0.0,>=1.19.1->azure-ai-textanalytics==5.2.0b2) (2.26.0)
Requirement already satisfied: requests-oauthlib>=0.5.0 in /home/daforne/.pyenv/versions/3.7.9/envs/global379/lib/python3.7/site-packages (from msrest>=0.6.21->azure-ai-textanalytics==5.2.0b2) (1.3.0)
Requirement already satisfied: certifi>=2017.4.17 in /home/daforne/.pyenv/versions/3.7.9/envs/global379/lib/python3.7/site-packages (from msrest>=0.6.21->azure-ai-textanalytics==5.2.0b2) (2021.10.8)
Requirement already satisfied: isodate>=0.6.0 in /home/daforne/.pyenv/versions/3.7.9/envs/global379/lib/python3.7/site-packages (from msrest>=0.6.21->azure-ai-textanalytics==5.2.0b2) (0.6.0)
Requirement already satisfied: idna<4,>=2.5 in /home/daforne/.pyenv/versions/3.7.9/envs/global379/lib/python3.7/site-packages (from requests>=2.18.4->azure-core<2.0.0,>=1.19.1->azure-ai-textanalytics==5.2.0b2) (3.3)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /home/daforne/.pyenv/versions/3.7.9/envs/global379/lib/python3.7/site-packages (from requests>=2.18.4->azure-core<2.0.0,>=1.19.1->azure-ai-textanalytics==5.2.0b2) (1.26.7)
Requirement already satisfied: charset-normalizer~=2.0.0 in /home/daforne/.pyenv/versions/3.7.9/envs/global379/lib/python3.7/site-packages (from requests>=2.18.4->azure-core<2.0.0,>=1.19.1->azure-ai-textanalytics==5.2.0b2) (2.0.7)
Requirement already satisfied: oauthlib>=3.0.0 in /home/daforne/.pyenv/versions/3.7.9/envs/global379/lib/python3.7/site-packages (from requests-oauthlib>=0.5.0->msrest>=0.6.21->azure-ai-textanalytics==5.2.0b2) (3.1.1)


texts = [
    '''
 A computer is a machine that can be programmed to carry out sequences of arithmetic or logical operations automatically. Modern computers can perform generic sets of operations known as programs. These programs enable computers to perform a wide range of tasks. A computer system is a "complete" computer that includes the hardware, operating system (main software), and peripheral equipment needed and used for "full" operation. This term may also refer to a group of computers that are linked and function together, such as a computer network or computer cluster.
 A broad range of industrial and consumer products use computers as control systems. Simple special-purpose devices like microwave ovens and remote controls are included, as are factory devices like industrial robots and computer-aided design, as well as general-purpose devices like personal computers and mobile devices like smartphones. Computers power the Internet, which links hundreds of millions of other computers and users.
 Early computers were meant to be used only for calculations. Simple manual instruments like the abacus have aided people in doing calculations since ancient times. Early in the Industrial Revolution, some mechanical devices were built to automate long tedious tasks, such as guiding patterns for looms. More sophisticated electrical machines did specialized analog calculations in the early 20th century. The first digital electronic calculating machines were developed during World War II. The first semiconductor transistors in the late 1940s were followed by the silicon-based MOSFET (MOS transistor) and monolithic integrated circuit (IC) chip technologies in the late 1950s, leading to the microprocessor and the microcomputer revolution in the 1970s. The speed, power and versatility of computers have been increasing dramatically ever since then, with transistor counts increasing at a rapid pace (as predicted by Moore's law), leading to the Digital Revolution during the late 20th to early 21st centuries.
 Conventionally, a modern computer consists of at least one processing element, typically a central processing unit (CPU) in the form of a microprocessor, along with some type of computer memory, typically semiconductor memory chips. The processing element carries out arithmetic and logical operations, and a sequencing and control unit can change the order of operations in response to stored information. Peripheral devices include input devices (keyboards, mice, joystick, etc.), output devices (monitor screens, printers, etc.), and input/output devices that perform both functions (e.g., the 2000s-era touchscreen). Peripheral devices allow information to be retrieved from an external source and they enable the result of operations to be saved and retrieved.
 '''
]

summary_text = az_cs_summarization.summarize(
    texts=texts,
    cs_endpoint='https://westus2.api.cognitive.microsoft.com/',
    cs_key='use_your_own',
    language='en'
)

print(summary_text)


[ExtractSummaryResult(id=0, sentences=[SummarySentence(text=A computer is a machine that can be programmed to carry out sequences of arithmetic or logical operations automatically., rank_score=1.0, offset=9, length=120), SummarySentence(text=These programs enable computers to perform a wide range of tasks., rank_score=0.95, offset=205, length=65), SummarySentence(text=A broad range of industrial and consumer products use computers as control systems., rank_score=0.79, offset=583, length=83)], warnings=[], statistics=None, is_error=False)]