DEV Community: vianel

I said I didn't want my agents on my phone. I changed my mind.

vianel — Tue, 10 Nov 2026 18:00:25 +0000

In my last post about sticking with herdr over Orca, I wrote this about phone notifications:

When I'm away from my desk, I want to be away and disconnect so checking on agents from my phone isn't a feature I'm missing.

Well. herdr has a phone app now (https://herdr.dev/mobile/) , I installed it, and now my agents are on my phone. So let me eat my words properly.

I still don't want my phone buzzing every time an agent finishes. That part hasn't changed. What changed is that I kept walking away from my desk in the middle of a session, and the agent would sit there blocked on a question until I came back. I don't want notifications. I want to continue the same session from wherever I am.

Getting there took me about an hour and four different errors, so here's the whole thing in order. Hopefully it takes you ten minutes.

The app asks for an address, a user and a port

That's it. No account, no pairing code. If those three fields look familiar it's because it's just SSH. The app connects to your machine the same way herdr --remote does.

Which means your Mac has to accept SSH connections, and mine didn't.

Problem 1: Remote Login was off

The obvious fix is one command:

sudo systemsetup -setremotelogin on

And the obvious fix didn't work:

setremotelogin: Turning Remote Login on or off requires Full Disk Access privileges.

Instead of giving my terminal Full Disk Access just for this, I used the toggle: System Settings → General → Sharing → Remote Login. Done.

Then the three fields are:

Address: your Mac's local IP (ipconfig getifaddr en0)
User: your macOS username (whoami)
Port: 22

Problem 2: the phone needs to be allowed in

The app generates its own SSH key and shows you the public half. That goes into authorized_keys on the Mac:

echo 'ssh-ed25519 AAAA... herdr-mobile' >> ~/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys

Small tip from someone who got it wrong three times in a row: paste that as one line. My terminal wrapped it, the chmod ran without a file name, and I spent a few minutes staring at a permission denied that had nothing to do with the actual problem.

The app also shows you a fingerprint for the machine it's talking to. You can check it's really your Mac with:

ssh-keygen -lf /etc/ssh/ssh_host_ed25519_key.pub

Problem 3: "Herdr isn't installed yet"

This one confused me the most, because herdr was very much installed. I was literally using it.

Two things were going on.

My PATH. herdr lives in ~/.local/bin, and I add that folder to my PATH in ~/.zshrc. But an SSH session that runs a command doesn't load .zshrc, only .zshenv. So for the phone, that folder didn't exist:

echo 'export PATH="$HOME/.local/bin:$PATH"' >> ~/.zshenv

It wasn't asking for herdr. It was asking for Herdr Core, a separate small binary called hc. The app tells you how to get it:

curl -fsSL https://releases.herdr.dev/core/install.sh | sh

(I read the script before running it, like a responsible adult. It checks the checksum and the code signature and drops the binary in ~/.local/bin.)

After that the app connected. Finally.

Problem 4: "No agents yet"

Connected, and empty. All my panes were running on the Mac and the phone showed nothing.

This is the part worth understanding: the phone shows terminals that run inside Herdr Core, not the panes in your herdr session. An agent you started the normal way isn't visible. You have to start it through hc:

hc run --bg --name stripe claude
hc attach stripe

The first line starts Claude Code in a terminal that Core keeps alive. The second puts your current terminal on it. You work as usual, ctrl+\ detaches, and the agent keeps running. Open the phone and the same session is there, same conversation, same everything.

That's exactly what I wanted. But I wasn't going to type two commands every time.

Making it invisible

My flow is: open a pane, type claude. I wanted to keep that. So claude is now a small function in my .zshrc:

claude() {
  if [[ -n $HC_TERMINAL_ID || ! -t 0 || ! -t 1 || ( $# -gt 0 && $1 != -* ) \
        || -n ${@[(r)(-p|--print|-v|--version|-h|--help)]} ]]; then
    command claude "$@"; return
  fi
  local name=${${${PWD:t:l}//[^a-z0-9_-]/-}[1,32]} out
  out=$(hc run --bg --rm --json --name "$name" -- claude "$@" 2>/dev/null) \
    || out=$(hc run --bg --rm --json -- claude "$@") || { print -r -- "$out"; return 1; }
  hc attach "${${out#*\"address\":\"}%%\"*}"
}

It looks worse than it is:

Interactive sessions go through Core. Plain claude, claude --continue, claude --resume.
Everything else is left alone. claude -p, claude --version, claude mcp ..., anything piped. Scripts and other agents calling Claude don't notice a thing.
The terminal is named after the folder, so on the phone I see the directory name and not t7. If the name is taken it just falls back to an unnamed one.
No nesting. Inside a Core terminal, HC_TERMINAL_ID is set, so it runs the real binary.
It cleans up. --rm removes the terminal when Claude exits.

And if I ever want the old behaviour, command claude skips the whole thing.

What I'd tell you before you try it

It's local network only, out of the box. The address is your Mac's local IP, so the phone has to be on the same Wi-Fi. For real "away from home" access you'd put something like Tailscale on both devices and use that address. I haven't set that up yet.

Your local IP can change. If the app stops connecting one day, check ipconfig getifaddr en0 again before you debug anything else.

You are opening SSH on your machine. Key-only access from one device on my own network is a trade I'm fine with, but it's a decision, not a default.

So, was I wrong?

Half wrong. I still don't want to be pinged by my agents. When I'm away, I'm away.

But there's a difference between an agent interrupting me and me choosing to pick up where I left off. The first one is a notification. The second one is just my terminal, in my pocket. Turns out I did want that.

Are you running your agents from your phone? Or is that where you draw the line?

My LLM Generates Text 🚀 Now Let’s Make It Faster 🚀

vianel — Thu, 05 Nov 2026 15:00:34 +0000

In my last post, we saw how an LLM generates text one token at a time:

Read the prompt
Score every possible next token
Pick one token
Add it to the prompt
Repeat

That is cool, but there is a small problem: it can be slow.

If you did not read the first post, start here: How an LLM Generates Text, One Token at a Time.

Today we are going to make the same small Qwen model faster in three steps:

Let PyTorch use the best device available
Try torch.compile() and understand the warm-up run
Use a KV cache so the model does not process the old prompt again and again

No training. Same model. Same greedy token selection. We are only changing how efficiently it runs.

Pre-requisites

This uses the same setup as the first article:

pip install -U torch transformers

The code downloads the open Qwen/Qwen3-0.6B-Base model the first time you run it. CPU works fine for learning, but a GPU makes the difference easier to see.

Step 1: let the code pick the device

Before loading the model, we ask PyTorch what hardware we can use.

def get_device():
    if torch.cuda.is_available():
        print("Using NVIDIA CUDA GPU")
        return torch.device("cuda")

    if torch.backends.mps.is_available():
        print("Using Apple Silicon GPU (MPS)")
        return torch.device("mps")

    print("Using CPU")
    return torch.device("cpu")

Easy peazy:

NVIDIA GPU? Use cuda.
Mac with Apple Silicon? Use mps.
Nothing else? CPU still works.

Then we move both the model and input tokens to that device:

device = get_device()
model.to(device)

input_token_ids = torch.tensor(
    tokenizer.encode(prompt), device=device
).unsqueeze(0)

The model and its tensors need to live in the same place. A model on the GPU cannot process tokens that are still sitting on the CPU.

Step 2: first, our original generation loop

This is the same simple loop from the last article. It sends every token written so far back through the model on every iteration.

@torch.inference_mode()
def generate_text_basic(model, token_ids, max_new_tokens, eos_token_id):
    input_length = token_ids.shape[1]
    model.eval()

    for _ in range(max_new_tokens):
        logits = model(input_ids=token_ids).logits[:, -1]
        next_token = torch.argmax(logits, dim=-1, keepdim=True)

        if torch.all(next_token == eos_token_id):
            break

        token_ids = torch.cat([token_ids, next_token], dim=1)

    return token_ids[:, input_length:]

Imagine the model has generated three tokens already:

Prompt + token 1 + token 2 + token 3

To generate token 4, this basic loop reads all of that again. It works, but it repeats work.

Step 3: what happens when we use `torch.compile()`?

torch.compile() asks PyTorch to look at the model operations and try to run them more efficiently. It does not train the model and it does not change its answers.

model_compiled = torch.compile(model, dynamic=True)

I am using dynamic=True because our input becomes longer as new tokens are added. Without it, the compiler may try to recompile when it sees a new sequence length.

Now let’s run the same prompt three times:

for run in range(3):
    start_time = time.time()

    output_token_ids = generate_text_basic(
        model_compiled,
        input_token_ids,
        max_new_tokens=100,
        eos_token_id=tokenizer.eos_token_id,
    )

    total_time = time.time() - start_time

    if run == 0:
        print("Warm-up run finished")
    else:
        tokens_per_second = output_token_ids.numel() / total_time
        print(f"Run {run}: {tokens_per_second:.1f} tokens/second")

Why ignore the first result? The first run has extra work: PyTorch is preparing the compiled version of the model. The second and third runs are the ones worth comparing.

This does not remove the repeated prompt processing. It only tries to make those calculations faster.

Step 4: now remove compile and use KV cache

For this next test, use the original model, not model_compiled:

# This is the normal, non-compiled model.
model.eval()

Then use Hugging Face generation with caching turned on:

@torch.inference_mode()
def generate_text_with_cache(model, token_ids, max_new_tokens, eos_token_id):
    all_token_ids = model.generate(
        input_ids=token_ids,
        max_new_tokens=max_new_tokens,
        do_sample=False,
        use_cache=True,
        eos_token_id=eos_token_id,
    )

    # `generate` returns the prompt plus new tokens.
    return all_token_ids[:, token_ids.shape[1]:]

The result is still greedy decoding:

pick the highest-scoring token → append it → repeat

But use_cache=True tells the model to save its previous attention work.

Without cache:

Prompt → predict token 1
Prompt + token 1 → predict token 2
Prompt + token 1 + token 2 → predict token 3

With cache:

Prompt → process once and save attention work
token 1 → use saved work
token 2 → use saved work
token 3 → use saved work

The model still has access to the full prompt. The cache stores the keys and values it needs from the old tokens, so it does not need to calculate them again.

Run it like this:

start_time = time.time()

output_token_ids = generate_text_with_cache(
    model,
    input_token_ids,
    max_new_tokens=100,
    eos_token_id=tokenizer.eos_token_id,
)

total_time = time.time() - start_time
tokens_per_second = output_token_ids.numel() / total_time

print(f"KV cache: {tokens_per_second:.1f} tokens/second")
print(tokenizer.decode(output_token_ids[0], skip_special_tokens=True))

So which one is better?

They improve different things:

Method	What it changes
GPU / MPS	Runs the same calculations on faster hardware
`torch.compile()`	Tries to execute the same model operations more efficiently
KV cache	Stops recalculating attention data for old tokens

KV cache is usually the bigger win for token-by-token generation, because it removes repeated work instead of only making that repeated work faster.

Your numbers will depend on your model, prompt length, PyTorch version, and hardware. Benchmark your own machine instead of trusting my numbers.

The complete example

import time

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer


MODEL_ID = "Qwen/Qwen3-0.6B-Base"


def get_device():
    if torch.cuda.is_available():
        print("Using NVIDIA CUDA GPU")
        return torch.device("cuda")
    if torch.backends.mps.is_available():
        print("Using Apple Silicon GPU (MPS)")
        return torch.device("mps")

    print("Using CPU")
    return torch.device("cpu")


@torch.inference_mode()
def generate_text_basic(model, token_ids, max_new_tokens, eos_token_id):
    input_length = token_ids.shape[1]
    model.eval()

    for _ in range(max_new_tokens):
        logits = model(input_ids=token_ids).logits[:, -1]
        next_token = torch.argmax(logits, dim=-1, keepdim=True)

        if torch.all(next_token == eos_token_id):
            break

        token_ids = torch.cat([token_ids, next_token], dim=1)

    return token_ids[:, input_length:]


@torch.inference_mode()
def generate_text_with_cache(model, token_ids, max_new_tokens, eos_token_id):
    all_token_ids = model.generate(
        input_ids=token_ids,
        max_new_tokens=max_new_tokens,
        do_sample=False,
        use_cache=True,
        eos_token_id=eos_token_id,
    )
    return all_token_ids[:, token_ids.shape[1]:]


def print_stats(name, output_token_ids, start_time, tokenizer):
    total_time = time.time() - start_time
    tokens_per_second = output_token_ids.numel() / total_time
    text = tokenizer.decode(output_token_ids[0], skip_special_tokens=True)

    print(f"{name}: {tokens_per_second:.1f} tokens/second")
    print(text)


device = get_device()
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForCausalLM.from_pretrained(MODEL_ID).to(device)
model.eval()

prompt = "Explain large language models in a single sentence."
input_token_ids = torch.tensor(
    tokenizer.encode(prompt), device=device
).unsqueeze(0)

max_new_tokens = 100

# Test 1: compile the original loop three times.
model_compiled = torch.compile(model, dynamic=True)
for run in range(3):
    start_time = time.time()
    output_token_ids = generate_text_basic(
        model_compiled,
        input_token_ids,
        max_new_tokens,
        tokenizer.eos_token_id,
    )

    if run == 0:
        print("Compiled warm-up run finished")
    else:
        print_stats(f"Compiled run {run}", output_token_ids, start_time, tokenizer)

# Test 2: remove compile and let `generate` use its KV cache.
start_time = time.time()
output_token_ids = generate_text_with_cache(
    model,
    input_token_ids,
    max_new_tokens,
    tokenizer.eos_token_id,
)
print_stats("KV cache", output_token_ids, start_time, tokenizer)

One last thing

This article uses a normal growing KV cache because it is the easiest version to understand. A growing cache is great for generation, but it can make torch.compile() harder because the cache shape changes every token.

That is not a problem we need to solve today.

Today, the important part is this:

GPU makes calculations faster.
Compile tries to make operations faster.
KV cache avoids doing old calculations again.

And that is how we start making an LLM faster.

For more detail about cache strategies and their memory/speed trade-offs, see the official Hugging Face KV-cache documentation.

How an LLM Generates Text, One Token at a Time

vianel — Wed, 04 Nov 2026 17:41:04 +0000

Ever wonder how an AI “thinks”? Let me break down how it generates text

First, a small correction: an LLM is not literally thinking like a person. At its core, it is very good at predicting the next token from the tokens that came before it. Do that again and again, and suddenly it looks like it is writing.

Let’s make that happen with a small pre-trained Qwen model.

What we are building

We give the model this prompt:

prompt = "Explain large language models in a single sentence."

Then we let it generate one token at a time, up to 100 times. This is the most simple version: each time, we pick the highest-scoring next token.

Pre-requisites

Python 3.10 or newer
PyTorch and Hugging Face transformers
Around 1.5 GB free for the Qwen 0.6B model weights (the script downloads them the first time)

Install the two libraries:

pip install -U torch transformers

Save the next code block as tiny_generate.py, then run:

python tiny_generate.py

CPU is completely fine for learning. It is just slower than a GPU.

Step 1: turn the prompt into token IDs

The model does not receive a Python string. The tokenizer breaks the text into pieces called tokens and represents them as integer IDs. A token is not always a whole word: it can be a word, part of a word, punctuation, or a space.

input_ids = tokenizer.encode(prompt)
input_tensor = torch.tensor(input_ids, device=device)
input_token_ids = input_tensor.unsqueeze(0)

That unsqueeze(0) adds the batch dimension. The shape changes from:

[number_of_tokens]

to:

[1, number_of_tokens]

The 1 means: “here is one prompt in this batch.” No magic, just the shape the model expects.

Step 2: load the model

The model is a big set of learned numbers. AutoModelForCausalLM loads the right Qwen architecture directly from Hugging Face. Calling eval() tells PyTorch we are using it for inference, not training.

model = AutoModelForCausalLM.from_pretrained("Qwen/Qwen3-0.6B-Base")
model.to(device)
model.eval()

Step 3: ask for the next-token scores

When we pass the full prompt into the model, it returns a score for every vocabulary token at every input position. Those raw scores are called logits. They become probabilities only after applying softmax.

For generation, we care about the scores at the final input position. That final row answers: “given everything in the prompt so far, what token should come next?”

logits = model(input_ids=input_token_ids).logits[:, -1]

The : means “keep every item in the batch,” and -1 means “take the final token position.” It does not throw away the earlier prompt; the model has already used the whole prompt to produce that final row.

Here is a tiny fake 2D view of the idea. A real model does this for every token in its vocabulary, not just five words on a graph.

In this example, models has the largest score, so greedy decoding chooses it.

Step 4: pick a token and add it back to the prompt

argmax gives us the ID with the highest score. Then torch.cat appends that new ID to the existing sequence. On the next loop, the model sees the original prompt plus the token it just wrote.

next_token = torch.argmax(logits, dim=-1, keepdim=True)
token_ids = torch.cat([token_ids, next_token], dim=1)

That is the whole trick: predict, append, repeat.

The complete runnable example

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer

MODEL_ID = "Qwen/Qwen3-0.6B-Base"

@torch.inference_mode()
def generate_text_basic(model, token_ids, max_new_tokens, eos_token_id):
    input_length = token_ids.shape[1]
    model.eval()

    for _ in range(max_new_tokens):
        logits = model(input_ids=token_ids).logits[:, -1]
        next_token = torch.argmax(logits, dim=-1, keepdim=True)

        if torch.all(next_token == eos_token_id):
            break

        token_ids = torch.cat([token_ids, next_token], dim=1)

    return token_ids[:, input_length:]


device = torch.device("cpu")
tokenizer = AutoTokenizer.from_pretrained(MODEL_ID)
model = AutoModelForCausalLM.from_pretrained(MODEL_ID)
model.to(device)
model.eval()

prompt = "Explain large language models in a single sentence."
input_token_ids = torch.tensor(
    tokenizer.encode(prompt), device=device
).unsqueeze(0)

output_token_ids = generate_text_basic(
    model=model,
    token_ids=input_token_ids,
    max_new_tokens=100,
    eos_token_id=tokenizer.eos_token_id,
)
print(tokenizer.decode(output_token_ids.squeeze(0).tolist()))

The line below is the important part of the loop:

logits = model(input_ids=token_ids).logits[:, -1]

The model reads all tokens so far, produces a huge list of possible next-token scores, and we select one. Then the selected token becomes part of the input for the next round. After enough rounds, token IDs become text again:

text = tokenizer.decode(output_token_ids.squeeze(0).tolist())

One important thing: this demo is deliberately boring

argmax always picks the most likely token. That makes the result deterministic and easy to understand, but it can also make the model repetitive. Chat apps often sample from the distribution instead: they use softmax, temperature, and sometimes top-p sampling to allow several plausible next tokens.

But before we add those knobs, it is worth seeing the simple loop. Generate a score vector. Pick a token. Append it. Repeat.

That is how the model starts writing.

Why I'm sticking with herdr over Orca as my ADE

vianel — Tue, 13 Oct 2026 15:06:20 +0000

We're all in the same boat right now: trying to figure out the best development setup. We've always done this in the past with VS Code extensions, LSPs, dotfiles but the question has changed. Now it's how do I connect, optimize and orchestrate a bunch of coding agents on my machine?

As a Vim/Neovim user, my first pick for an ADE (agentic development environment) was herdr. I've been using it for a few weeks and it's been working great. Then a new competitor showed up, Orca, so I gave it a fair shot: I read the docs, installed it, and ran my usual work through it.

I'm still sticking with herdr. Here's why.

Orca is genuinely good

Let me be fair first. Orca's GUI is polished and friendly. Seeing Claude and Codex usage and rate-limit resets at a glance is great, worktrees are front and centre, and features like fanning one prompt out to several agents and annotating AI diffs line by line are excellent.

But I live in the terminal. Neovim, tmux, CLIs is just home for me. Even though Orca's GUI does a good job of feeling like a terminal, it isn't my terminal, and switching to a separate app always felt slightly off for me.

What herdr gives me that fits my workflow

It runs inside the terminal I already use. herdr is a single Rust binary, no Electron. It runs in my terminal next to Neovim, with my fonts, my theme, my keybindings.

Coming from tmux, there was nothing to learn. Prefix keys, splits, ctrl+b q to detach. My muscle memory just worked on day one. (And the mouse works too when I feel like clicking.)

Agents can drive it, not just me. This is the big one. herdr exposes a CLI that agents can use to spawn panes, prompt each other, and wait until another agent finishes or gets blocked. My main Claude Code session hands mechanical work to Codex on its own, based on routing rules in my CLAUDE.md:

| long-running or needs supervision | gpt-5.5 | herdr pane — see "herdr delegation" |

### herdr delegation
- Prompt with `herdr agent prompt <name> "…" --wait`. Never sleep/poll.
- `blocked` = approval/question UI → show me what it's asking; don't approve it yourself.

Claude opens a Codex pane, waits on its state (no polling, no wasted tokens), and stops to ask me whenever Codex hits an approval prompt.

Here in this small screenshot the codex panes were open by claude

I always know which agent is stuck. Every pane is marked working, blocked or idle. When something needs me, herdr tells me (I even gave Codex its own sound).

Closing the window doesn't kill the work. Sessions live on a background server, so I can detach, close my laptop or drop my SSH connection, and everything keeps running. herdr reattaches.

My whole setup is a TOML file. It lives in my dotfiles, so moving from my Mac to Linux, or onto a remote box over SSH, is completely seamless. Same keys, same popups, same everything.

It doesn't care which agent I use. Claude Code, Codex, Gemini, Cursor, OpenCode and a long list more. herdr doesn't wrap them; it just owns their terminals.

Stealing Orca's best ideas

Trying Orca wasn't wasted time. I took its best ideas and rebuilt them on top of herdr.

Fan-out. One prompt, several agents, each in its own git worktree, then compare and merge the winner. In herdr that's a Claude Code skill: it creates the worktrees, starts the agents with herdr agent start, prompts them in parallel, runs the tests in each, and gives me a comparison table. Nothing gets merged until I approve.

/fanout “add retry with backoff to the S3 uploader” --agents codex,codex,opus

Diff review. Orca's annotate the diff feature is chef's kiss. My version is a small script bound to a herdr popup. It opens the diff in Neovim, I add #! comments under the lines I care about, and on save they're sent to whichever agent I pick, with file and line references.

[[keys.command]]
key = "prefix+alt+r"
type = "popup"
command = "herdr-review"
width = "90%"
height = "90%"

I use the same popup trick for gh dash and lazygit, so PRs and diffs are one keystroke away without leaving the terminal.

Phone notifications. Doable with a hook and a push service, but honestly I don't want it. When I'm away from my desk, I want to be away and disconnect so checking on agents from my phone isn't a feature I'm missing.

Design Mode. Clicking a UI element to send its HTML and CSS to an agent is clever. I'm not doing frontend work lately i feel like i could use it in the future for sure and a browser MCP gets me close enough when I need it.

So which one should you use?

If you like a GUI, want everything built in, and prefer to steer agents by hand, Orca is a great choice.

If you live in the terminal, want agents that can orchestrate other agents, and like owning your setup as config files, herdr is hard to beat. The features it doesn't ship with, you can build in an afternoon.

For me, the deciding question was simple: do I want a new app, or a better terminal? I wanted a better terminal.

What's your ADE setup? I'd love to hear what's working for you.

We scanned 69 random Shopify stores for accessibility. 97% had detectable failures.

vianel — Wed, 30 Sep 2026 16:18:42 +0000

Most accessibility studies look at the biggest sites. We wanted ordinary Shopify stores, and not just home pages.

Method: we shuffled the .com and .us domains in the Tranco top-1M list (V349N), found 130 live Shopify stores selling at least one product, and scanned four pages of each (home, a collection, a product, the cart) with axe-core 4.13 and the WCAG 2.0/2.1/2.2 A and AA rules. 69 stores loaded at least 3 of 4 pages; most of the rest were cut off by Shopify's per-visitor rate limit.

What we found:

Problem	Stores
Low color contrast	88%
Links with no accessible name	61%
Invalid or misused ARIA	51%
Tap targets too small (WCAG 2.2)	51%
Images with no text alternative	46%
Keyboard problems	35%
Buttons with no accessible name	25%
Form fields with no label	22%
Pinch-to-zoom turned off	20%

97% had at least one detectable failure (WebAIM Million: 95.9% of home pages).
Product pages (91%) and cart pages (90%) were no cleaner than the home page.
All 11 stores running an accessibility overlay still had failures. Small sample, but an overlay runs on top of the page and leaves the markup as it was.

Limits: automated checks find only part of the problem, and a failure isn't a legal finding. We don't name any store.

Full write-up, including where each problem is usually fixed in Shopify: https://adatriage.com/shopify-accessibility-study-2026

Flow to Source implementation with Akka streams

vianel — Mon, 03 Aug 2020 23:58:30 +0000

NOTE: This post requires previous knowledge of scala using the Akka framework specifically Akka streams.

Akka streams structure

The Akka streams module of Akka provides a beautiful DSL that will help you to move data in an asynchronous way. The base of an Akka stream is composed of 3 main components:

Source ~> Flow ~> Sink

Source: Which is in charge of emitting the data through the stream, only have outputs ports
Flow: receive the data, transform it and send it to the next component
Sink: The end of the stream it will be the last step of any stream, only have input ports.

The problem

So far so good but what happens when we need to get the data from a source and this receives the key on the fly? i.e we need to download a file but we don't know the name of the file which arrives frow a previous step and we are using a library to download the file which is a source.

FlatMapConcat FTW

The FlatMapConcat flow allows us to transform any input element into a source this means we can connect a flow with a source, something like this:

Source ~> Flow ~> Source ~> Sink

A simple sample would look like this

1    //Simple Source
2    val src = Source(List(1, 2, 3))
3
4    val fromSrc = b.add(Flow[Int].log("from-Src"))
5
6    //Emit new list per element received
7    val flow = b.add(Flow[Int].flatMapConcat{ element =>
8      Source(List(4, 5, 6))
9    })
10
11    //¯\_(ツ)_/¯ nothing to do, just for sample purposes
12    val map = b.add(Flow[Int].map{ element =>
13      element
14    })
15
16    src ~> fromSrc ~>flow ~> map ~> sink

At line 16 in the main flow, we can see the 3rd step of our process is a regular flow but digging into it at line 8 we notice this flow is actually a Source that receives data from the flow FromSrc and emits new data to the map flow. 🙌🏻

That's it we just connected a source in the middle of a stream that emits new data based on the received data.

...
In case you want to run the sample check out my gist here

How to build a Web Scraper using golang with colly

vianel — Sat, 18 Jul 2020 02:49:57 +0000

I have seen a lot of examples of how to build a web scraper in lots of programming languages mostly in python specifically using scrapy tool but only a few in golang.

As many golangs fans know, golang has tons of benefits when we talk about concurrency and parallelism, all of these features combined with a modern framework allow us to scratch the web in an easy and fastest way, but first of all, let's start with what a web scraper does?

Explaining Web scraper like I'm five

Quoting Wikipedia definition:

Web scraping, web harvesting, or web data extraction is data scraping used for extracting data from websites. The web scraping software may access the World Wide Web directly using the Hypertext Transfer Protocol or through a web browser

So a web scraping is a technique used to extract data from websites using HTTP, think of this a web scraper is basically a robot that can read the data from a website like the human brain can read this post, a web scraper can get the text from this post, extract the data from the HTML and it can use them for many purposes.

Web scraper vs Web crawler

In order to keep this short, a web crawler is a bot that can browse the web so a search engine like google can index new websites and a web scraper is responsible of extract the data from that website.

Back to business

Now we know what we are building let's start to get our hands dirty, our first step will be to create a simple server in golang with a ping endpoint, using the standard lib it will look like this

package main

import (
    "log"
    "net/http"
)

func ping(w http.ResponseWriter, r *http.Request) {
    log.Println("Ping")
    w.Write([]byte("ping"))
}

func main() {
    addr := ":7171"

    http.HandleFunc("/ping", ping)

    log.Println("listening on", addr)
    log.Fatal(http.ListenAndServe(addr, nil))
}

That's it we just created a simple server with golang 👏🏻 to test it just run it like any other golang program using go run main.go you will see a log that our server is listening to the port 7171. To test it with curl just run curl -s 'http://127.0.0.1:7171/ping'

Thanks to the http lib we can easily create a server just calling the ListenAndServe function specifying a port and using the HandleFunc we can manage different endpoints for our API.

Now we have a server up and running. Let's create another endpoint, this one will extract some data from the colly website.

First of all, we need to install the colly dependency to do this I highly recommend to use go module just run go mod init <project-name> this will generate the go.mod file where all dependencies used in the project will be. Open the go.mod and add the colly dependency in the require section

require (
github.com/gocolly/colly v1.2.0
)

and that's it go module will take care of download the dependency to your local machine.

We are all set to extract all the data from the websites so let's create a function to get all links from any website

func getData(w http.ResponseWriter, r *http.Request) {
//Verify the param "URL" exists
    URL := r.URL.Query().Get("url")
    if URL == "" {
        log.Println("missing URL argument")
        return
    }
    log.Println("visiting", URL)

//Create a new collector which will be in charge of collect the data from HTML
    c := colly.NewCollector()

//Slices to store the data
    var response []string

//onHTML function allows the collector to use a callback function when the specific HTML tag is reached 
//in this case whenever our collector finds an
//anchor tag with href it will call the anonymous function
// specified below which will get the info from the href and append it to our slice
    c.OnHTML("a[href]", func(e *colly.HTMLElement) {
        link := e.Request.AbsoluteURL(e.Attr("href"))
        if link != "" {
            response = append(response, link)
        }
    })

//Command to visit the website
    c.Visit(URL)

// parse our response slice into JSON format
    b, err := json.Marshal(response)
    if err != nil {
        log.Println("failed to serialize response:", err)
        return
    }
// Add some header and write the body for our endpoint
    w.Header().Add("Content-Type", "application/json")
    w.Write(b)
}

This function will extract all links from any website specified in the url param.

And finally, let's add our new endpoint to our server

func main() {
    addr := ":7171"

    http.HandleFunc("/search", getData)
    http.HandleFunc("/ping", ping)

    log.Println("listening on", addr)
    log.Fatal(http.ListenAndServe(addr, nil))
}

Done we have our new search GET endpoint which receives the url param and it will extract all links from the website specified. For test it using curl you can use curl -s 'http://127.0.0.1:7171/search?url=http://go-colly.org/' it will response something like

["http://go-colly.org/","http://go-colly.org/docs/","http://go-colly.org/articles/","http://go-colly.org/services/","http://go-colly.org/datasets/","https://godoc.org/github.com/gocolly/colly","https://github.com/gocolly/colly","http://go-colly.org/","http://go-colly.org/","http://go-colly.org/docs/","http://go-colly.org/articles/","http://go-colly.org/services/","http://go-colly.org/datasets/","https://godoc.org/github.com/gocolly/colly","https://github.com/gocolly/colly","https://github.com/gocolly/colly","http://go-colly.org/docs/","https://github.com/gocolly/colly/blob/master/LICENSE.txt","https://github.com/gocolly/colly","http://go-colly.org/contact/","http://go-colly.org/docs/","http://go-colly.org/services/","https://github.com/gocolly/colly","https://github.com/gocolly/site/","http://go-colly.org/sitemap.xml"]

Conquer all the web

Congrats! you just created a web scraper API in golang 🙌🏻.
before conquering all the web remember to check the robots.txt file of the website to ensure you can extract that data. For those who are not aware of what robots.txt is.
Is a file on all the websites where it specifies the instructions to the robots like web crawler or web scraper how to handle their data or what endpoint the robots can use. For example, in google will be https://www.google.com/robots.txt

Next step

Keep going further with colly documentation, it has a lot of examples of web scraper and web crawlers.

DEV Community: vianel

I said I didn't want my agents on my phone. I changed my mind.

The app asks for an address, a user and a port

Problem 1: Remote Login was off

Problem 2: the phone needs to be allowed in

Problem 3: "Herdr isn't installed yet"

Problem 4: "No agents yet"

Making it invisible

What I'd tell you before you try it

So, was I wrong?

My LLM Generates Text 🚀 Now Let’s Make It Faster 🚀

Pre-requisites

Step 1: let the code pick the device

Step 2: first, our original generation loop

Step 3: what happens when we use torch.compile()?

Step 4: now remove compile and use KV cache

So which one is better?

The complete example

One last thing

How an LLM Generates Text, One Token at a Time

Ever wonder how an AI “thinks”? Let me break down how it generates text

What we are building

Pre-requisites

Step 1: turn the prompt into token IDs

Step 2: load the model

Step 3: ask for the next-token scores

Step 4: pick a token and add it back to the prompt

The complete runnable example

One important thing: this demo is deliberately boring

Why I'm sticking with herdr over Orca as my ADE

Orca is genuinely good

What herdr gives me that fits my workflow

Stealing Orca's best ideas

So which one should you use?

We scanned 69 random Shopify stores for accessibility. 97% had detectable failures.

Flow to Source implementation with Akka streams

Akka streams structure

The problem

FlatMapConcat FTW

How to build a Web Scraper using golang with colly

Explaining Web scraper like I'm five

Web scraper vs Web crawler

Back to business

Conquer all the web

Next step

Step 3: what happens when we use `torch.compile()`?