DEV Community: Ilja (ilya) Nevolin

Unlocking Swarm Intelligence with Big Agentic Brain and Claude Code

Ilja (ilya) Nevolin — Wed, 21 Oct 2026 22:12:44 +0000

Originally published at nevolin.be. Disclosure: I'm involved in building Big Agentic Brain, the open-source system tested here.

AI coding agents start every session from zero. When the fix is in the code, that's fine. When it depends on something a teammate's agent already learned, like a vendor quirk, a runbook convention or a spec that lives in another repo, the agent can read every file and still never find it.

Big Agentic Brain (BAB) is a distributed, self-evolving memory for a whole organization's agents. It is self-hosted, runs as one container on one SQLite file and speaks MCP. Agents search it first, record what they learned at the end, and react to each other's entries so the useful ones rise.

We measured what that changes on bugs that need team knowledge. Sonnet went from 12% to 79% with BAB, and to 100% (24/24) with a one-line search-first nudge. Opus went from 12% to 88% (92% with the nudge), and Haiku from 0% to 75%. With each agent in its own isolated sandbox, a fix one agent found reached a teammate on another machine in 96 to 100% of cases, against 13 to 67% with local notes only. And for Haiku, the team's cost per verified fix roughly halved, from $0.00672 to $0.00333.

That's from 1,104 graded runs for about $29, plus 378 distributed-team sessions. Below is the short version of the study, including where it falls short. The full 21-page report is here, and the code is on GitHub.

What BAB is

BAB is an open-source, self-hosted memory for AI agents. It runs as one container on one SQLite file and speaks MCP, so Claude Code and any other MCP-capable agent can connect to it. Each agent follows a simple loop:

Search before starting, to find what other agents already learned.
Learn when finished, recording the fix, the vendor quirk or the team convention as a structured entry.
React to other agents' entries. Votes and reactions change what ranks first, so the memory gets better with use without anyone curating it by hand.

Because it is one server, it isn't tied to one laptop. Everyone in an organization who uses Claude Code, or any mix of agents, can point at the same BAB and join one swarm. Every fix one agent makes becomes something every other agent can find.

How it fits together:

Agent A, Agent B, Agent C: can run on different machines.
Each one connects over MCP to one shared memory (BAB).
Through that memory they search first, learn at the end, and react to each other's entries, which changes what ranks first over time.

Why it matters for teams

Here's one of the bugs from our benchmark. A service sends order webhooks to Shipfast, a carrier aggregator. Since this month, every request comes back 401 "signature verification failed". The unit tests pass. The code looks reasonable:

body = json.dumps(event).encode("utf-8")
signature = hmac.new(secret.encode("utf-8"), body, hashlib.sha256).hexdigest()
headers = {
  "Content-Type": "application/json",
  "X-Shipfast-Signature": f"sha256={signature}",
}

The root cause isn't in this repository. Shipfast retired that header on 1 September and now expects Shipfast-Signature: t=<unix seconds>,v1=<hmac>, signed over "<timestamp>." + body. The only copy of that spec lives in a different repo, where a teammate's agent already hit the same 401 and fixed it.

An agent working on the webhook repo can read every file and rerun the tests all day and still never see the new format. That's team knowledge: vendor quirks, runbook conventions, a partner's ID check digit. It lives in someone's head or someone else's repo, and AI coding agents start every session from zero. BAB is how one agent's lesson reaches the next.

From 12% to 100% on bugs that need team knowledge

We built 8 bug families like Shipfast, where the missing fact exists only in what a colleague's agent had learned. Without shared memory, the agents were stuck: Sonnet fixed 12%, Haiku 0% and Opus 12%. With BAB they fixed 79%, 75% and 88%, and a one-line nudge to search first took Sonnet to 100%. On the Shipfast family itself, Sonnet went from 0 of 3 to 3 of 3.

Team-knowledge bugs fixed

Model	No shared memory	With BAB	BAB + search-first nudge
Sonnet 5.5	12%	79%	100%
Haiku 5.5	0%	75%	88%
Opus 5.5	12%	88%	92%

Share of team-knowledge bugs fixed, 24 runs per cell (8 bug families, 3 trials each), graded by hidden tests.

A bigger model is no substitute: Opus, the largest of the three, fixed the same 12% as Sonnet without memory. For Sonnet, the gain from default BAB was +67 points (95% CI +33 to +92).

A fix on one machine reaches the whole team

Real teams don't share one laptop. So in a third round we gave each agent its own isolated sandbox and workspace, standing in for its own machine, with no shared files. Each agent first fixed one bug where the answer was documented. Then memory was frozen, and a related bug in another repo went to a teammate. We compared no memory, each agent's own local notes, and local notes plus a shared BAB for the team.

Teammate fix rate on a related bug

Model	Teams	No memory	Local notes only	Local + shared BAB
Sonnet 5.5	4	67%	67%	100%
Haiku 5.5	8	8%	13%	96%
Opus 5.5	2 (exploratory)	33%	67%	100%

How often a teammate fixed a related bug in another repo, after a different agent had solved the first one on its own machine.

Local notes are great for the agent that wrote them: on a repeat of its own problem, Sonnet went from 58% to 100%. But they stay on one machine. With shared BAB, teammates fixed the bug 96 to 100% of the time, against 13 to 67% with local notes only. Haiku, the smallest model, fixed its teammate's bug 96% of the time with BAB, more reliably than Sonnet with no memory at all (67%).

Cheaper per delivered fix

Reading and writing memory costs tokens, so a single session with BAB is not cheaper. What a team pays for is fixes that land. Counting every session, including the first solvers and the cost of writing what they learned into memory, shared BAB gave the lowest cost per verified fix for Sonnet and Haiku. For Haiku it was about half the no-memory cost.

Team cost per delivered fix (USD)

Model	No memory	Local notes only	Local + shared BAB
Sonnet 5.5	$0.0427	$0.0408	$0.0363
Haiku 5.5	$0.00672	$0.00414	$0.00333
Opus 5.5	$0.1285	$0.0908	$0.0950

API cost per verified fix, counting every session a team ran: the first solvers, the cost of writing what they learned into memory, and the teammates. Lowest per model in bold.

Lean by design

Tool definitions are sent with every request, used or not. BAB's full toolset added 8,598 tokens to each one. A lean setup with only bab_search, bab_learn and bab_react adds 648, and on team-knowledge tasks it kept most of the benefit (75% for Sonnet) at 1.18× the no-memory cost per run, against 1.51× for the full toolset.

Toolset	Tokens added per request	Cost per run vs no memory (Sonnet, team-knowledge tasks)
Full tool surface	8,598	1.51×
Lean (3 tools)	648	1.18×

The study turned straight into changes for BAB:

a lean default toolset;
auto-retrieval of the top memories before the first turn, which also reached 100% for Sonnet, at 1.34× baseline cost;
a search-first hint;
corrected search instructions, after we found that BAB's own advice to filter by kind returned nothing.

How we tested

Agents fixed seeded bugs in small benchmark repos and were graded by hidden tests. We ran Claude Sonnet 5.5, Haiku 5.5 and Opus 5.5: 1,104 graded runs on 19 October 2026 for about $29 in API spend, plus 378 graded sessions in the distributed-team round on 21 October for $17.41. Memories were recorded by agents that had solved the source tasks, not written by hand, and the tasks and hypotheses for the later rounds were frozen before any paid run. Agents ran in a minimal MCP agent loop rather than Claude Code itself. An audit of the distributed-team round's transcripts found no access to hidden tests and no way around the memory freeze.

Caveats

Built for the case BAB targets. The team-knowledge tasks were designed so the missing fact exists in a colleague's memory. They show the memory channel works when the knowledge is there, not how often real work looks like this. Samples are small, and the tasks were written by an AI agent before any runs.
Stale memory hurts. With only outdated entries, success fell back to the no-memory baseline, and agents flagged the stale entry in only 4% of runs. Visible age, verify-before-acting prompts and down-weighting old entries matter as much as ranking.
Retrieval at scale. Recall@5 was 0.93 with 20 distractor memories and 0.81 with 1,000. On a harder test with look-alike distractors, the right memory was ranked first 56% of the time (13% for blind-written queries) and made the top five 85% of the time.
Per session, memory costs more. The saving is per delivered fix. For Opus, with two teams, local notes alone edged out shared BAB at team level ($0.0908 vs $0.0950 per fix).
Simulated machines. In the distributed-team round, the separate machines were isolated sandboxes on one host. A real org-wide deployment is the next thing to measure.

Footnote: on 12 easy bugs that the code already explains, every setup fixed every task (108/108 Sonnet runs, 180/180 Haiku runs), so memory had nothing to add there and cost 1.50× per run on Sonnet.

📄 The full report (PDF, 21 pages)
Methodology, every condition and model, the statistics, retrieval tests and deviations.

Try it

BAB is open source and runs as one container on one SQLite file, so a single server can serve every agent in your organization: Big Agentic Brain on GitHub. If it's useful to you, a star helps others find it.
The eval suite is being contributed to the repo (pull request #26), so you can rerun every condition or point it at your own tasks and agents.

If your results look different from ours, I'd like to hear about it.

Kubernetes Resource Management Best Practices

Ilja (ilya) Nevolin — Wed, 19 Aug 2026 21:24:48 +0000

TL;DR:
set requests.cpu, requests.memory, and limits.memory, but remove limits.cpu.

"Always set memory limit == request. Never set CPU limit"
~ Tim Hockin, Kubernetes co-founder (source)

Google's GKE docs say the same thing: for the CPU request, "specify the minimum CPU needed... according to your own SLOs," then "set an unbounded CPU limit" (source).

Disclaimer: this post is a short version of the full analysis I posted on GitHub here. Some of the images below should be animated (but SVGs are blocked on DEV.to), so I highly recommend reading it there as well.

Most teams still set a CPU limit anyway, then spend their time fighting the throttling that follows. This analysis explains why the advice is right, and why so many people still get it wrong. Most charts set two CPU numbers, a request and a limit, and people treat them like the same setting with a bit of headroom.

The request is how much CPU is reserved for your pod if it needs it.
The limit is a hard cap: hit it and the kernel throttles the pod, even when the node still has spare CPU.

There are exceptional use cases for using CPU limits (they exist for a reason), but your situation is unlikely one of them, meaning you probably should NOT set CPU limits (see conclusion).

Why CPU requests are enough

Linux already shares a node's CPU fairly among pods. That sharing is called CFS (Completely Fair Scheduler). A request is a weight in the cgroup, not a pinned core. If the node is busy, CFS splits time in proportion to those weights: a 16 CPU request gets about sixteen times the CPU of a 1 CPU request. If the other pods are idle, a pod can use the leftover; when they need CPU again, those cores go back. None of this requires a CPU limit.

A limit is a separate cap, also enforced by CFS. Every 100 ms the kernel gives your pod a budget of CPU time, and every thread in the pod shares that budget. When it is gone, the pod is throttled until the next 100 ms. The node can be idle and the pod still waits. A 1 CPU limit on a 32-core node can still run on all 32 cores for a few milliseconds, then sit out the rest of the window. Four threads working at once burn a 500m budget in about 12 milliseconds. It is an average, not a reserved core.

Grafana shows the CPU is fine! No, your metrics are lying.

The CPU graph on your Kubernetes dashboard usually averages a minute. CPU throttling lasts a tenth of a second. So the graph can look fine while the app is being throttled all the time. Watch container_cpu_cfs_throttled_periods_total, not average CPU.

Two views of the same 30 seconds: an average CPU graph that looks flat and fine, next to a CPU throttling graph on the same window showing the pod repeatedly hitting its limit

People put a limit on because they are afraid some other app will starve their pod. That is what the request is for. The app you care about is protected by its own request. A runaway next door can use leftover CPU, but it cannot take the share you reserved. If leftover on the node is huge, the requests are too small. If teams can deploy with no request at all, give them a default request instead of putting a CPU limit on whoever looks greedy.

The same blindness poisons right-sizing. Usage recorded under a CPU limit can never go above the limit: the cap clips every burst, so the history shows what the kernel allowed, not what the app wanted. Size a request from that history and you copy the cap's distortion into the request. Drop the limit first, let the app run for a while, then measure and set requests from numbers that were free to move.

Proof: the same app with and without a CPU limit

In a controlled burst test, adding a CPU limit took typical latency from 23 ms to roughly 87 ms (about 4x slower), with the limited pod throttled in half of all CFS windows, while the average CPU graph looked fine the whole time.

How CPU limits cause memory issues and OOMKills

A service with any kind of in-memory queue: a consumer buffering messages from Kafka or RabbitMQ faster than it can process them, an unbounded channel between two components, a server holding request payloads while handlers run behind.

Any kind of application that has a garbage collector (GC) is a potential victim. In my experiment: every request builds a reference-dense graph of 10,000 small objects (~1.5 MiB) and holds it only while doing real work; once the request finishes, the graph is garbage. Nothing is retained anywhere, on purpose. What runs short here is the garbage collector's own CPU budget: on the capped pod (limits.cpu: 100m, the same value as the production incident this test is based on), in-flight requests pile up, the live object count grows with them, each GC cycle gets more expensive, and the collector fights the workload for the same shrinking quota.

Even a simple HTTP API server is not spared. As http requests come in, and each one is holding its memory. Nothing stops the pile-up: web frameworks (ASP.NET Core, Node, Go, and friends) do not cap how many requests may run at once, and the await hands the thread back but keeps the buffer, so the thread pool does not cap it either.

All three shapes end the same way on a graph: memory climbing into the limit. It looks like a leak, the usual fix is a bigger memory limit, and the actual cause is the CPU limit. Check container_cpu_cfs_throttled_periods_total before you blindly raise limits.memory.

Conclusion and FAQs

Dropping limits.cpu moves a pod from Guaranteed to Burstable QoS. In practice this rarely matters: the kubelet evicts for memory pressure, not CPU, and it ranks pods by how far usage exceeds the request, not by QoS class alone. Keep requests.memory equal to limits.memory and eviction exposure barely moves. Full answer, with the eviction docs link, in the FAQ.

FAQ

Should I set CPU limits in Kubernetes? For most services, no. Keep requests.cpu so CFS reserves your share, keep limits.memory, and drop limits.cpu. The exceptions (benchmarks, pinned cores, per-customer billing) are listed in the conclusion (read more).

Doesn't a limit stop a bad pod from eating the node? Monopolizing a node is a myth. Spare CPU is borrowed, not taken: the moment another pod wants CPU, CFS pulls those cores back within milliseconds and splits time by request weights again. A limit on the busy pod only stops it using CPU that would otherwise sit idle. It does not give CPU to anyone else: the neighbor is protected by its request. If several pods burst at once, the leftover is not first come first served, CFS divides it in proportion to their requests. If leftover on the node is huge, the requests on that node are too small.

After I drop limits, can a burster hurt the node itself? Not the other pods, but kubelet, containerd, the CNI, and log shippers often run with tiny or no CPU reservation. A pod bursting into all spare CPU can delay exec probes and flap readiness. The fix is system-reserved and kube-reserved in the kubelet config, which carves out CPU for the node's own daemons. A per-pod CPU limit is the wrong tool for this.

A cli tool to be more organized and productive in 2026

Ilja (ilya) Nevolin — Sun, 01 Feb 2026 19:12:04 +0000

We all "love" microservices, allegedly. But you know what isn't fun? Realizing you need to bump a dependency, run a build, git pull and create PRs across 15 different repositories at the same time.

You usually have two options (if you don't want to use AI agents):

The "Tab Hoarder" Strategy Manually open 15 terminal tabs, cd into each one, run the command, and pray you didn't miss one.
The "Unix Wizard" Strategy You try to construct a one-liner that iterates over directories. You start typing:

   find . -maxdepth 1 -type d -exec sh -c 'cd "{}" && git pull' \;

It works, but it's painfully slow because it runs serially (one at a time). So you switch to xargs to run it in parallel:

ls -d */ | xargs -P 4 -I {} bash -c "cd {} && pnpm update"

Now you have a problem. xargs (by default) breaks on directory names with spaces. To fix it, you need to add -print0 to find and -0 to xargs, and suddenly your "quick one-liner" is 80 characters long and unreadable.

Enter in-cli 🚀 https://github.com/inevolin/in-cli

Usage:

in [OPTIONS] [DIRECTORIES...] [--] COMMAND...

I got tired of typing those boilerplate loops. I wanted a tool that does one thing well: runs a command in every directory, fast. So I built in-cli. It’s a zero-dependency CLI (pure bash) that acts as a force multiplier for your shell.

How it actually works

Instead of wrestling with pipes and flags, you just tell it what to do.

Scenario 1: The Morning Routine You get to work and need to update all 20 repos in your ~/work folder.

# Updates every repo in ~/work/ directory
in ~/work/* git pull

Scenario 2: The "Critical Fix" Deployment You need to trigger a build script across all your services right now.

# Run 'make build' in parallel across all subdirectories
in -P 8 ~/work/* make build

Scenario 3: Dependency Hell Need to update lodash in every service because of a CVE?

in ~/work/* 'pnpm update lodash && git commit -am "updating lodash" && git push'

You'll find +50 other real-world scenarios in the EXAMPLES.md file.

Why use this?

Stop wasting time on tooling boilerplate. If you're managing a monorepo or a folder full of microservices, stop writing throwaway scripts to manage them. Check it out on GitHub, and let me know if it saves you a few keystrokes (and your sanity).

Enjoy!

Speech-to-Text Discord bot written in Go

Ilja (ilya) Nevolin — Sat, 27 Jan 2024 22:42:16 +0000

As part of my personal journey to learn Go, I've decided to rewrite one of my open source projects in GoLang.

The project is a standalone (offline) speech-to-text bot for Discord. Basically it transcribes everything you say in a voice channel. This is useful if you want to have custom voice commands (eg. while gaming), or enhance the communication experience for hearing impaired/deaf people.

Repository: https://github.com/inevolin/DiscordEarsGo

The project makes use of the Vosk library, which does not work well on Mac OS (M1), so by default it is designed to only work on Linux x86 systems (since you would likely be hosting it on Linux). But the great thing is that Vosk works offline, open source and comes with a ton of models and languages (english, german, french, chinese, ...) for download https://alphacephei.com/vosk/models

One of the annoying things was also to decode Opus packets to PCM, it requires opus libraries to be installed. The same is true for the NodeJS version (which now requires ffmpeg to be installed). It would be nice if there was a tiny library/snippet that does this (not external library).

Enjoy!

Work on Open Source projects to stand out in 2024

Ilja (ilya) Nevolin — Tue, 02 Jan 2024 14:49:51 +0000

Happy new year everyone!

I thought of sharing something cool and fun with my fellow coders. A few years ago I created a speech-to-text bot for Discord,, initially designed to control my own music bot (at the time), but then people started using it to communicate better with their deaf gaming friends in real-time.

This is the project: https://github.com/inevolin/DiscordEarsBot (the music bot is in another repo which needs a big update). The STT methods include offline and online APIs, so you get to choose the quality/performance you want. But with the rise in AI and ML jobs, this can serve as a great framework to build projects to stand out in your interviews:

Use this technology in another tool (Slack, Zoom, phone, browser, ...)
Use this technology to build voice-based games (maybe for handicapped kids)
Improve the STT results by adding another ML model to correct the text
Create a fun bot that "talks back" or tells jokes based on the conversation

If you want to collaborate on a project, post in the comments below and match up with other people!

The dark truth behind Cryptocurrencies

Ilja (ilya) Nevolin — Sun, 20 Nov 2022 18:05:52 +0000

The environmental impact of blockchains and cryptocurrencies cannot be overlooked. In particular Proof-of-Work systems such as Bitcoin have a huge (understated) energy footprint. In conventional cryptocurrency systems, tokens are awarded to “miners” for successfully mining blocks on the blockchain. The core problem is that mining consumes incomprehensible amounts of electrical energy.

How bad is it?
According to several studies (link), a single Bitcoin transaction requires 2,180kWh, with this amount of energy you can execute over 1.4 million Visa transactions, or fully charge 29 Tesla cars (Model 3, 75kWh). Another research reveals Bitcoin system alone consumes half of the energy of the banking industry (source). Pretty bad is an understatement...

If you ask any expert, Bitcoins are a dramatic waste of natural resources. Even if Bitcoin is fully run on renewable energy, its benefits will never outweigh the energy consumption (mind the natural resources, maintenance & production, material waste from solar panels, wind turbines, etc). You may also have seen articles trying to debunk Bitcoin fuds, but if you conduct some research, many of those authors are merely crypto influencers, acting as Hollywood scientists (examples).

How can we solve this?

I’m building the Carbon3 cryptocurrency that does not use a blockchain (requires no mining). It has better security and privacy wallet aspects than existing cryptocurrencies, stimulates good capitalistic behavior and enforces eco-friendly consumerism.

About me
I work in the web3.0 industry with a particular stake in the decentralized-identity space. Whether we like it or not, blockchain technologies (in particular PoW ~ Bitcoin) are wasting precious energy resources from our planet 🌎, but fail to universally solve the centralized monetary issues 💰. The big institutions still dominate the financial sector, and are dominating the crypto industry as well. And with enforced Travel Rule worldwide, crypto loses its face as an anonymous system.

Get in touch to learn more https://www.linkedin.com/in/iljanevolin/

My First Month of Running an API-first Startup

Ilja (ilya) Nevolin — Tue, 06 Apr 2021 21:51:17 +0000

Early March I was hired to run and grow Spurwing. I am in charge of the business development, sales and marketing aspects, and co-responsible for several engineering projects. In this post I'll share my personal insights, tips and tricks from my first month at an early-stage Startup company.

The company

Our core product is the Appointment Scheduling API. Our primary clients are engineering teams and managers. The API allows developers to innovate quicker, build cheaper and scale easier any kind of time sensitive process.

The most common use cases we focus on are Time Management Solutions. The most simple use cases are custom booking and scheduling widgets. The more complex solutions entail enterprise calendar management, remote team collaboration tools and supply chain management. Our products fit in any industry and can be used by most roles.

We are not directly competing with products like Calendly. Our mission is providing engineering resources and tools to development teams to solve problems quicker, easier and cheaper.

Spurwing is actually a spin-off product developed for Healthie Inc. where its technology is used to schedule and manage millions of annual bookings and events.

First week

I have been working in software engineering, product and business development for over 10 years. So it was a fairly easy and efficient on-boarding process for me. Some of you may relate, but most of the things we do at Spurwing, and the challenges we face, I already faced before. I have a pretty solid idea of how to tackle most challenges and what it takes to execute.

This being said, the first week I spent a lot of time communicating with the directors on our vision and strategy for Spurwing. The goal we set is building a large, free and open-source Marketplace. This marketplace will contain a ton of resources and tools for engineering teams and developers. These include API libraries, widgets, chat bots, management dashboards and many integrations with 3rd party providers.

Giving away all these resources for free provides us with great marketing opportunities and a big advantage over the competition as well.

Once we had our goal clearly defined, it was time to execute. I started building the libraries for the API in several programming languages: Python, NodeJS, JavaScript/Ajax and we are working on Java, C++ and C# as well. Thanks to these libraries we can now easily and quickly build new tools and platforms.

Second week

Research is an important activity I do on a daily basis. Specifically I research and note down all types of projects and solutions for us to work on.

I started building the most crucial ones first: 1:1 appointment scheduling widgets for websites, calendar links and the like.

Third week

I spent that week mostly on research, prospecting and talking to potential clients. Most communication happens on LinkedIn or email. LinkedIn is great, but it's not easy to get people to talk to you. Most senior engineers and managers have no time to chat with randoms. It's all about finding the ones that are active and engage with them.

A quite popular solution we built that week was a Facebook Messenger Chat Bot with booking / scheduling capabilities:

We are still in the process of improving this chat bot, enhancing the user experience and experimenting with new features. Aside from that we have several more chat bot integrations in the pipeline: Discord, Slack, WhatsApp, Skype, Intercom, Amazon Lex, Google Chat and more.

We started building a CI/CD pipeline for all our libraries, widgets and tools on the marketplace. Over the years I became very proactive of Test Driven Development and the value that CI/CD pipelines provide to engineering & QA teams. These days I no longer build products without having some decent automated testing (and deployment) included.

Fourth week

Mostly prospecting, building marketing content and researching product ideas. In addition to that we built a simple scheduling too for international teams across different time zones:

Tools like these allow us to save precious time and be more productive internally. Most of our employees and partners are located at different time zones, thus setting up meetings can sometimes be a hassle. This tool allows all participants to submit their availability for a certain date range. The organizer of the event can then view a strict overlap of all the submissions and choose a time slot that works for all. It's a quite simple and basic implementation but provides a huge value.

Conclusion

You may wonder how the heck we get so much done in just under a month. It's all about great management mixed with experience & expertise.

We use Slack / Discord to easily and quickly communicate with all our business and engineering teams. We spend very little time on email or phone communication. We only have one dedicated video chat call per week to catch up. As you can see, we try to minimize and reduce any unnecessary bottlenecks. This allows us to be extremely productive.

If you have any specific questions, post them down below!

Useful links

Spurwing blog on DEV
Open-Source Marketplace
Brochure

About me

I'm a modest Software engineer with +10yrs of experience in building software for web, desktop and some Android stuff. Multi-Startup founder and product developer. Feel free to connect: https://www.linkedin.com/in/iljanevolin/

Ciao!

Do you even Refactor? 003

Ilja (ilya) Nevolin — Sun, 14 Feb 2021 22:34:19 +0000

Code refactoring is crucial but often overlooked. It can improve the design and performance of existing code.

The Python code below takes about 14 seconds to complete. Refactor the getData function to make it run in less than 10 seconds. Post your answer in the comments.

import time

def getData():
  arr = []
  for i in range(1000*1000*50):
    arr.append(i)
  lo, hi = 0, 0
  for x in arr:
    if x < lo:
      lo = x
    if x > hi:
      hi = x
  print(lo, hi)
  return arr

def timed(func):
  def run():
    Tstart = time.time()
    func()  
    Tend = time.time()
    Tdt = round(Tend - Tstart, 2)
    print(Tdt, 'seconds')
  return run

@timed
def main():
  data = getData()
  print('len:', len(data), 'sum:', sum(data))

main()

Do you even Refactor? 002

Ilja (ilya) Nevolin — Sat, 13 Feb 2021 16:29:43 +0000

Code refactoring is crucial but often overlooked. It can improve the design and performance of existing code.

The Python code below takes about 17 seconds to complete. Refactor the getData function to make it run in less than 5 seconds. Post your answer in the comments.

import time

def getData():
  arr = [5] * 1000 * 1000 * 100
  for i in range(1, len(arr)):
    if i % 5 == 0:
      arr[i] = 0
  return arr

def timed(func):
  def run():
    Tstart = time.time()
    func()  
    Tend = time.time()
    Tdt = round(Tend - Tstart, 2)
    print(Tdt, 'seconds')
  return run

@timed
def main():
  data = getData()
  print('len:', len(data), 'sum:', sum(data))

main()

Do you even Refactor? 001

Ilja (ilya) Nevolin — Sat, 13 Feb 2021 12:16:51 +0000

Code refactoring is crucial but often overlooked. It can improve the design and performance of existing code.

The Python code below takes about 12 seconds to execute. Refactor the getData function to make it run in less than 1 second. Post your answer in the comments.

import time

def getData():
  arr = []
  for i in range(1000*1000*100):
    arr.append(5)
  return arr


def timed(func):
  def run():
    Tstart = time.time()
    func()  
    Tend = time.time()
    Tdt = round(Tend - Tstart, 2)
    print(Tdt, 'seconds')
  return run

@timed
def main():
  print(len(getData()))

main()

MultiVariate Anomaly Detection

Ilja (ilya) Nevolin — Sun, 07 Feb 2021 16:21:37 +0000

Discovering anomalies in complex multivariate and multidimensional data can be quite a challenge. Visualizing these anomalies can be even trickier, especially if you want to keep it simple without having to go over thousands of charts to filter out issues from false positives and noise. Using statistical methods we can aggregate complex data to be displayed on a single heatmap. By hovering over specific cells, we can quickly display the individual data on charts.

Heatmap: https://healzer.github.io/Industrial-Data-Analysis/hmap1.html?testdays=3&testlike=1

This system was first implemented for a high-tech enterprise's CI/CD pipeline. It is being used by R&D, Q&A and management to keep track of all processes and variables throughout the development lifecycle. Any anomalies can easily be identified and pinpointed as soon as they appear on the heatmap. Anomalous decreases (green) indicate performance improvements (time, memory and parameter reductions), while degradations (red) imply performance issues.

Very subtle improvements or degradations are tricky to identify and detect, but overall the system has a very high accuracy. The best use of this system is for data that should remain static over time, it may not work very well if you have alternating/seasonal data. Separate charts can be built for mapping GitHub commits to each individual data point, allowing the team to instantly pinpoint which code change caused which performance change.

The demo URL can contains three parameters:

testdays: how many days from the data should be used as test data (versus baseline data)
testlike: filter data whose test value should contain a certain string (these are the Y-axis values)
annotate: (0 or 1) primarily for debugging reasons, indicating whether to show Z-values on each cell (may slow down your browser!)

This code may not be directly usable for your purposes, but the general idea of using statistical functions like MADe and Z-values to detect anomalies in datasets may be very useful in CI/CD pipelines but also many industrial processes.

GitHub: https://github.com/healzer/Industrial-Data-Analysis

Multipurpose tables as CRM and to-do lists

Ilja (ilya) Nevolin — Sat, 06 Feb 2021 12:32:50 +0000

Most of you know CRM by the many advanced tools out there such as Salesforce, Capterra and Hubspot. But millions of small companies still use Excel/Word/Files/Directories to manage their accounts, invoices and projects. I have met very few individual contractors who use productivity tools/apps, even simple things like notepad apps to record customer requests and issues. Even fewer apply basic CRM practices to stay in touch with their customers, such as keeping their contact details and/or sending out promotional emails.

I understand that for many businesses it may seem like a lot of overhead with little or no immediate value. But some of these do wish to grow and expand their operations, thus seek low-cost solutions.

Since I started the PyCRM project I have been thinking of very simple yet effective ways of doing CRM. Since every business is different, everyone wants a slightly different interface, customization is key.

I remembered that there are many Javascript libraries that provide customizable tables/forms. In addition many of these use very simple data structures which can easily be serialized as JSON and stored as a file without using a complex (No)SQL database.

In the latest addition to our PyCRM project, we can use the JS Tabulator library which provides this functionality. You can easily customize the columns, fields and hundreds of other features.

Since it's a SaaS page, it works pretty well on mobile devices. You can use it anywhere as long as you have an internet connection. The save button saves any changes on the server.

This is a very basic implementation, so be very careful when it is used by multiple people at once. When multiple people save then only ones' version will be stored, the other changes will be overridden and lost.

An improved implementation would be to commit individual fields instead of the entire dataset. But also keep any changes automatically synced over all active users. Once again this isn't difficult to implement but depends on whether you need it or not.

Apart from using this as a CRM, you can turn this into a simple project management tool, or a to-do list or a sales tracking tool, etc. Enjoy!