DEV Community: Nicholas

I think it’s time for Aster’s come back. Or just what to do when you finished your project.

Nicholas — Thu, 17 Sep 2026 15:44:26 +0000

Hey guys ;)

So as the title said I’ve finished my Music Classification project. You could check it here Music Classification Project.
Sorry that I didn’t deploy this app. I hope if someone will be interested in it, then this person will just copy my repo.

What can I tell you?

Actually I don’t know, this project was pretty easy, because only librosa was new for me. All the other libraries I’ve used before. Maybe I just should say the main points of how to transform data into models. I guess that will be good.

First step

Maybe it will be unexpected, but you should choose a topic in which you’re interested. I think it’s important for every project you’re working on. If you don’t have fun with it, then why are you doing it?
For example all my projects built on my interests and hobbies.

Second step

Now, you need data. It’s good if you will find an amazing dataset for your topic, or you just could scrape all data manually. The more proper data you have, the more accurate your model will be. I mean if we only have 10 students for example and half of them are good students and the other half are bad, then how to classify an average student?

Third step

Here you need to work with data I guess. Clean it, fill None values, normalize data and split data into training and test datasets. That’s easy, maybe normalization is a bit difficult, because you should choose for example between StandardScaler and MinMaxScaler. Also for some models you need to encode target.

Final step

Choose the right model and make it accurate. Yep there are Regression models and Classification models, and there are Linear models, ensemble models and so on. How to choose right? You need to read a bit of theory.

What’s next?

Again the title said everything already. I’m going to come back to my Aster - personal assistant project. Actually I had 3 options - Chess Project, Aster and brand new AI for Cuphead.

But I think I need to work with Aster, at least upgrade it, and it would be great to make Aster “alive”. So, for the next few weeks I’ll be writing about Aster. That’s all for today, see you guys later 😘

Which classification have I used? As I know it should be KNN, but stuff wasn’t so easy.

Nicholas — Thu, 10 Sep 2026 17:01:15 +0000

Hey guys ;)

This time I also don’t have much to say. But let’s try to explore some interesting things that happened this week.

Back to Uni

So, the most important reason why I have less time to work is University. But I must confess that I heard something that gave me more power to continue my journey. So, this week I worked on Music Classification only yesterday. On Monday and Tuesday I was just bored or something like that and didn't write any code. Then one lecture in uni gave me motivation and understanding that there are still so many things in the world that I don’t know yet. Maybe it’s the reason why I’m moving forward.

Music Classifier

Now I could say that the project is almost done. Yep it’s kinda strange, but learning models is the easiest part. So first I trained the KNN model, when I got 68% test accuracy I decided to change model. The first option was Random Forest, it gave me 78% accuracy. +10%, pretty impressive, but I still wanted more. So then I chose the XGBoost model. And finally I got 85% accuracy. Yep maybe it’s not perfect, maybe it would be great to gain 90% accuracy, but for now I think the model is pretty accurate. Here I’ll add some images (Confusion matrix and Top-15 important features).

So what do I need to do? Today I wanna finish the main part of the project. Write predict.py which will load trained models, extract all features, normalize features, and predict genres. And next week I need to write README and add some conclusions into notebook. Maybe compare KNN, Random Forest and XGBoost. I don't know yet what I’ll write there.

May I know your favorite songs?

I just wanna test my classifier later, so it’d be great to check it on your favorite songs. Yep, there will be one problem, my classifier only knows 10 genres, all of them you could see on the Confusion Matrix plot.

So it’s all for today, see you later guys 😘

Summer is over. But I continue my funny journey

Nicholas — Thu, 03 Sep 2026 17:02:00 +0000

Hey guys ;)
Summer ended, but September's days are still so hot. Hope you’re doing well.

So I actually don’t remember if I told you about my new project in my last post. Maybe I only said that it will be a KNN based project. It was hard to choose an interesting project to build. I mean I don’t wanna do something big, because I still should finish the Chess Project. So after a few hours of struggling and conversations with ChatGPT I decided that I should try to build Music Genre Classification and for interface use Streamlit.

So actually I don’t know what to say, because this project is much easier than the previous. I’ll use GTZAN Dataset for training KNN, yesterday I built a feature extractor from an audio file. Today I added the Streamlit interface. Next week I’ll train KNN, save it using Joblib and then launch a full pipeline. So as I said, the project is pretty easy and I don’t have to say much about it.

But I’d like to hear your favorite song and when the app is done, I’ll check its genre.

So see you next time, hope with more interesting info 😘

Unexpected, but Secret Santa is cooked. Let's explore bit more about role of Django here.

Nicholas — Thu, 27 Aug 2026 10:42:15 +0000

Hey guys ;)
Hope you’re doing well, summer is almost over and some of us will go back to University (for example me) or maybe even some of us will even go to school. My summer was pretty good, actually I’ve finished 2 projects and 1 is in the final stage (yep I still remember about the Chess Project 😂). Okay, let’s move to the main topic of this post.

Secret Santa done.

Actually almost done, I mean all logic and design were cooked, there is still one task - README.md. I’m going to do it after finishing this post. Secret Santa now available for everyone, you could access it via this link https://secret-santa-baf4.onrender.com/
But you need to wait about 50 secs because I deployed it with free Render trial (at least deployed, not localhost XD )

What can my Secret Santa do ?

Okay, Santa working with a simple data pipeline, first you enter all required data and then just send an email. Now more detailed.

Participants input with 3 modes:
- manual input mode;
- file input mode;
- Faker script generator mode;
Party general info (name, time, date, location, host email)
Results and sending email

What more interesting All this you could do for free, even sending the emails.

More technical info about Django

I couldn’t say that after this little project I became a super Django user, I’m a beginner (I even didn’t finish my tutorial). But I’ll tell you some important stuff that I’ve learned.
In a previous post I described basics like views and urls in general. Today I wanna talk about how Django sees data.

So basically when we talk about big websites it has a database. Django allows work with different databases and changes nothing in logic. So for working with data usually you need 2 things - model and form. I think all we know is what it is. The model describes something and how it should look, form could be built on this model and then you just create form inside of view and transfer it to the frontend.

But what to do, if I don’t need to save my data to the database. We use for it sessions(cookies). My pipeline is simple, I collect data from participants form, save it to request.sessions["participants"], do the same for party info and in result page use pop method, which clear all sessions data after using it. So what I’ve done in Secret Santa.

Created forms with specific fields (like CharField, IntegerField, FileField and EmailField). All fields are available in Django.Forms.
Import created forms in views.py. We need to create 2 forms. One for the POST method and the other just an empty form. And obviously we need to add our form into the context dictionary.
And finally we could access forms in HTML.

You could create Forms based on Models (for this in Form Class after all fields add Meta Class) or just ordinary Form. The difference in Model form will be very strict, if you change Form and forgot to change Model then you’ll get some troubles.

How to send an email ?

Actually I thought that it would be the hardest part of the project. But I was wrong. First of all the situation is similar to databases. You write logic once, then just change settings in settings.py. So what do we need to send an email?

We could use many different APIs, like Resend or Brevo. But the problem is that you need an email domain for this. I don’t have one. But you could still do something.
And it’s Google. All you need is to add 2 step verification for your account and create an App and get a password for it. Save your email (account where you created the App) and password (usually 16 symbols) to the .env file, and that’s all. Now you could use @gmail.com domain for sending email for everyone you want. So it’s looks like that

It was really easier than I thought. I was planning to finish Secret Santa and some KNN projects by the summer end. But the KNN project I’ll start next week, I hope it will be interesting, and the more important it will be enough for getting my first job, yep I wanna try my luck on freelance.

So, see ya in September, hope you have a nice summer.
Oh yeah, I’ll add a link to my X post with 3 videos, where I tried to show Secret Santa, but there were 2 fails 😂.
// Detect dark theme var iframe = document.getElementById('tweet-2092911178183766208-374'); if (document.body.className.includes('dark-theme')) { iframe.src = "https://platform.twitter.com/embed/Tweet.html?id=2092911178183766208&theme=dark" }

Is it possible to code while learning something new? Of course, it’s more fun.

Nicholas — Wed, 19 Aug 2026 16:08:37 +0000

Hey guys;)
Long time no see.

General

Actually I wanna change my blog format. Now I’m planning to do a few posts on X, and then 1-2 times per week do a big post here. And I wanna try not only to show what I’ve done, but also how to do it. Some kind of teaching I don’t know yet.

New project. What is it about?

Okay, so since my last post I’ve finished Yakaboo Sales Analysis. You could check it here - Yakaboo Project.
The new project is basic - it’s Secret Santa. But if I wrote just an algorithm it wouldn't be so interesting. So I decided to add Django here and make some websites. But here was a problem, I’ve never worked with Django before. But thanks to Corey Schafer I learnt some basics and even built something. I’ll show the video in the end.

A bit about Django

So it’s a great framework which allows us to build websites using Python. In the beginning I thought that I’ll explain almost everything that I’ve learnt, but I don’t think it’s necessary. So in short, to begin writing you should do

Create a project directory and venv (of course you could do it without venv).
Install Django.
Create the project using django-admin startproject name_of_project.
Create an app using django-admin startapp name_of_app

After that we could do what we wanted.

Another amazing library

One of my favorite libraries is Faker, which allows generating data. I used it for creating participants (their full name, email and wishlist) for a party. That was the easiest part, after that I started exploring Django. So, I’ll skip all the theories and say only the most important.

To write html and css inside the app directory you need to create templates (for html) and static (for css and js, about js I’m not sure), and inside these directories create a folder with the same name as app name.
Inside the views.py file you create a function which returns html response and gives arguments which we could use inside html.
After creating views functions you could use it in the url of the app, add a pattern to the project and don’t forget to add the app into project settings.

Okay now it’s time to watch some results for now.
Link to video
See you in the next post. I hope it was interesting to read.

A long break from post, but I’ve cocked some stuff like Walter White did.

Nicholas — Wed, 12 Aug 2026 19:11:00 +0000

Hey guys ;)
It’s been a while since my last post. Actually I was a bit busy and forgot to write about progress. And this time I really have what to say. So what happened in short?

Scraper ended data extending.
Pageviews script added pageview info for all 6k books.
My first analysis notebook is done.

So, here is a fragment of my extended data before pageviews.

And here is a fragment of dataset after pageviews.

Is pageview extending false?

As I said in a previous post, sometimes Wikipedia pageviews give some nonsense data, and it’s true. Barely it gives an exact match, more often it gives the author page, and most of the time it gives 0 or some nonsense.
When I’ll start my second analysis notebook, I’ll do something about that, maybe erase some data, for example if the saved Wiki title is in author or in title, I don’t know yet.
Here is my pageview script fragment.

First conclusions

When I began this project I asked a question, do people read less nowadays? Actually I don’t know. According to my data I couldn’t say that people read less or people read more nowadays. All I could say it’s that people still read some books indeed.
Again all ,my conclusion here will be true according to my scrapped data, custom standard of books popularity and validity for Ukraine reading culture.
So, people prefer physical books more, the most popular author is Hiro Mashima, because of the number of his manga volumes. The most popular publishers are those who produce manga and comics, again because of the number of volumes and parts. The amount of books released in the last 10 years is bigger than the amount of books released 10-20+ years ago. And the most popular categories are manga, comics, autobiography and memoirs, self-improvement and children literature. Charts from notebook I’ll add too.

What do I mean by using the words popular or famous? So in my dataset there are 6k books. And for each analysis I’ve used count. So the most popular author is who has the most amount of books. And I should remind you, that it isn’t some 6k books, it's the best books according to my custom standard.

Next step

I think the next step is going to be the last. I don’t know, maybe I’ll do all the code tomorrow, or next week. So what do I need to do?

Analyse books, their popularity according to standard and to pageviews.
Write something between code.
Write final conclusion and README.md (with some screenshots to show what this project is about very quickly, without opening notebooks).
Push and commit everything.

What do marketing tags, broken author arrays, and an unexpected email from Derek Sivers have in common?

Nicholas — Sat, 01 Aug 2026 13:38:41 +0000

Hey guys 👋

I don't have a massive technical feature to showcase today, but I do have a solid batch of data pipeline updates, a bug fix story, and a pretty inspiring personal highlight.

While refining the secondary parser for the Ukrainian book market, I ran into classic data pollution issues, stripped out marketing noise, and got a reminder of why building side projects should be fun.

Here is the weekly progress report!

Fixing the "Comma-in-Category" Bug

After rewriting the extended scraper, I noticed a subtle data formatting bug during early inspection.

Certain books on retail platforms list multiple authors. My initial DOM selection logic grabbed only the primary author string. However, because of how the HTML elements were structured, the trailing author separators (commas and secondary tags) bled directly into the Category column!

Corrupted: Category = ", Fiction, Bestsellers"
Cleaned:   Category = "Fiction"

To fix this:

Updated the scraper logic to properly parse multi-author arrays.
Manually sanitized and patched 20 affected JSON objects in the clean dataset to keep data integrity intact.

Stripping "Garbage" Categories & Preprocessing

I’m currently finishing up the final data preparation script, specifically handling empty values (NaNs) and filtering non-standard categories.

Originally, I expected platform categories to function purely as literary sub-genres (e.g., Sci-Fi, Historical Fiction, Biography). However, e-commerce platforms heavily mix genre metadata with promotional marketing tags:

"Reader's Choice"
"Top Bestsellers 2023"
"Must-Read Recommendations"

These tags provide zero value when trying to analyze actual genre trends over time. My preprocessing script is currently dropping these noise categories along with empty rows to leave a pure, genre-focused analytical dataset.

An Unexpected Reply from Derek Sivers

On a non-technical note, something really cool happened this week.

I recently finished reading Anything You Want by Derek Sivers-a short, brilliant book about entrepreneurship, independence, and doing things your own way. At the end of the book, Derek invites readers to send him an email, introduce themselves, and share what they are working on.

I decided to write to him, not really expecting a reply. To my surprise, he actually responded! He shared some thoughtful feedback and even recommended a few more great books to check out.

It was a great reminder that building side projects-whether it's a Python chess engine or a market scraper-should come from a place of curiosity and freedom.

What's Next?

Finishing the final NaN drop and category mapping pass in Pandas.
Running exploratory data analysis (EDA) to generate our very first trend charts.
Starting one of Derek's recommended books during my 3-day rest period!

Have you ever reached out directly to an author or creator who inspired you? How do you handle promotional marketing tags when cleaning e-commerce data? Let's chat in the comments below! 👇

No APIs?! Analysis is almost here, but I’ve mistaken again.

Nicholas — Wed, 29 Jul 2026 14:56:14 +0000

"Why hit API request limits for 111,000 books when a single log10 formula and direct web scraping can solve the problem?"

Hey guys ;)

Looks like my focus is completely locked on data analysis right now. I can’t help it-this project is just too fun.

Over the past two or three posts, I kept promising that I would enrich my book dataset using the Google Books API and OpenLibrary. Well... forget about the APIs.

After wrestling with rate limits, poor metadata quality, and HTML structures all week, I dropped the APIs entirely and built my own data enrichment pipeline. Here is how it happened, why I deleted 1,451 scraped books today, and what’s coming next.

The Problem with Books APIs

When I started planning the data enrichment phase for my 111,000 unique scraped books, I ran into two massive brick walls:

Google Books API Limits: The free tier caps you at 1,000 requests per day. At that rate, enriching 111k books would take over three months (I know about payment account and increasing of limits).
OpenLibrary Limitations: It handles Ukrainian literature poorly without exact ISBN matches. Title and Author search combinations gave way too many false positives.

I quickly realized I didn't actually need all 111,000 books. Thousands of them had zero reviews, no ratings, and no active search interest. Processing them was just adding noise.

Filtering the Catalog and Popularity Formula

To isolate the most relevant titles, I designed a custom metric to score and rank the entire catalog:

popularity_rating = log10(amount_of_reviews + 1) * rating

Using the logarithmic scale prevents titles with hundreds of reviews from completely breaking the ranking curve while still giving heavy weight to actual reader engagement.

This formula immediately isolated ~3,000 highly active titles. I expanded the threshold to capture the top 6,000 books for the final analytical sample.

Parsing Metadata Directly

To analyze publishing trends over time, I needed the exact publication year for a specific edition, not just the general first release year of a work.

To get that accuracy from APIs, I would need to parse the book's individual URL anyway to extract its ISBN. That's when the realization hit me: If I'm already scraping the specific book pages for ISBNs, why not just scrape the missing metadata directly from Yakaboo?

So from Monday until yesterday, I wrote a dedicated secondary parser to process my top 6,000 URL list, extract extended properties, and map them into structured category buckets.

Here is my best books selection code

Here is scraper fragment

And how I deal with categories for final dataset preprocessing

The HTML Trap and deleting 1,451 Books

It wouldn't be a real project without a good mistake.

While running the extended parser, I noticed that Yakaboo's HTML structure reuses generic CSS classes across different metadata blocks while hiding or omitting unique element IDs. Because of a misplaced selector logic, my parser was extracting Author names and saving them into the Publisher field. 😅

After 1,451 books were parsed, I caught the bug, wiped the corrupted JSON file, and fixed the DOM selector.

Today, I’m re-running the script on the top 6,000 clean URLs. The most important lesson in Data Engineering: never give up, even when you have to wipe your output and start over!

Summary & What's Next

GitHub: All changes are finally committed and pushed! I'm much more cautious here than in my Chess project, pushing only when the pipeline logic is 100% verified.
Current Task: Re-running the extended parser for the top 6k books without the Author/Publisher swap bug.
Next Goal: Data cleaning, category normalization, and generating our first analytical charts to answer if people are reading fewer books over time.

Have you ever built a custom metric to downsample a massive dataset? What’s your worst HTML scraping horror story? Let’s chat in the comments! 👇

#python #datascience #webscraping #pandas #showdev

Weekends with translator for book analysis

Nicholas — Sat, 25 Jul 2026 11:29:37 +0000

Hey guys ;)

Today I’m going to tell you about some progress in books analysis and my plans for the weekend. Let’s start.

So from my previous post, I’ve been working with the author translation problem. What is it? Okay, so for extending books data I should use some API, and for that I need an ISBN that will give a certain book with 99% accuracy, or use a Title + Author combination. When a book is from a Ukrainian publisher, it will find the book with 80-90% accuracy. But what happens if the book is from a foreign publisher? Well, in fact, the accuracy will definitely be less.

Another problem was the amount of unique books that I parsed from 16 categories. That amount surprised even me: 117,000 unique books and 63,000 unique foreign authors.

So before I found out the exact amount of unique books, I wrote a script based on the deep-translator library (Google Translator). My first few requests were done without a rate limit, so my IP got blocked. After that, I added time.sleep(2). Even after adding a dictionary for saving unique authors and reducing the number of requests, the execution time was too long - like 8 hours or more.

After that, I found an interesting library called argostranslate. The main difference is that it’s an offline library, and it works entirely on laptop resources. The process of translating took only 1h 30 min to 2 hours.

(Though someone forgot about saving! For Parquet files, Python needs the pyarrow library. So I installed pyarrow and will start the translation process on Monday.)

Okay, now about some plans. Of course, for book analysis, the next step after translation is working with an API. Now I’m thinking about a few variants:

Google Books API: With an ordinary free account, I have only 1,000 requests per day. If I connect a payment account, I’ll get 50,000 requests.
Open Library API: Free, but this API doesn’t have much info about Ukrainian publishers.

So the best option could be Google Books API. But that’s for next week!

Now it's the weekend, so I can rest, play games, and watch something interesting. Speaking of watching, first I’m going to watch some Django tutorials (because in the future I’ll definitely need it) and maybe something else.

Have a great weekend!
#python, #dataanalysis, #learning, #buildinpublic

Building a Chess AI & Book Scraper - Part 5: Memory-Efficient JSON Loading & Switching to a 4/3 Work Balance

Nicholas — Thu, 23 Jul 2026 09:40:32 +0000

"Even when your scripts are doing the heavy lifting automatically, watching them run while trying to maintain a 6/1 schedule will burn you out."

Hey guys ;)

I’m posting a day later than usual today (Thursday instead of Wednesday), and honestly, that’s part of the update. Over the past few weeks, I’ve been living in a 6 days on, 1 day off routine. When that single day off finally arrived, I felt an overwhelming pressure to cram in every missed habit, chore, and task. Instead of resting, I ended up feeling completely drained.

So, I’m changing my framework: I’m moving to a 4/3 schedule (4 days of intense focus, 3 days of genuine rest and recovery).

Here is what’s happening with my hardware, my data pipelines, and my mental bandwidth!

Memory-Efficient Data Ingestion in Pandas

On Tuesday, I officially wrapped up the main scraping phase for the Ukrainian book market. I finally ran git push on the repository and started building the processing pipeline in Pandas.

Instead of calling pd.read_json() directly, I wrote a custom loader function using with open() to stream the JSON dictionary before passing it to pd.DataFrame().

Why not just use `pd.read_json()`? 🤔

Standard read_json() can be a memory hog because Pandas attempts to infer structures and allocate buffer space for the entire file at once. While my raw file is currently around 50MB, this approach guarantees that as the dataset grows (after API enrichment), my machine won't choke on RAM limits.

The Case of the Missing 400 Pages

While testing the loader function and inspecting the dataset using .head(), I noticed a weird anomaly: the top entries were all children's literature (which is Category #2 in my setup).

Somewhere during the long automated execution, the first 400+ pages of the Fiction category evaporated! 😅

So here is my immediate technical checklist for today:

[ ] Re-scrape the missing Fiction category pages.
[ ] Run a full deduplication pass across all 16 categories.
[ ] Integrate the Google Books API to pull missing metadata (publication years, page counts, sub-genres).

♟️ Chess RL Engine: Extending the Self-Play Deadline

Because I’m adjusting my schedule to a 4/3 split, I am officially extending my self-play generation deadline for the chess project.

People often think: "Why are you tired? You're just sitting there watching a script scrape pages or a chess engine play itself."

It’s true that the CPU does the calculation, but managing routines, tracking habits and debugging background crashes. Extending the deadline gives me the breathing room to build a high-quality dataset without turning my personal projects into a stressful second job.

Here is my screenshots of daily habits.

💬 Let's Chat!

How do you manage your schedule when building side projects? Have you ever had to restructure your work/rest ratio to prevent burnout? Let’s chat in the comments! 👇

#python #pandas #mentalhealth #datascience #showdev

Building a Chess Engine & Book Scraper - Part 4: Capping the Catalog & 4,500+ Games Cooked

Nicholas — Sat, 18 Jul 2026 15:45:38 +0000

"The data pipeline is perfectly stable, the automation is running smoothly, but I'm not pushing these changes to GitHub yet." 😅

Hey guys ;)

I’m a bit tired after this week's data marathon, so I'll let the numbers and the architecture do the talking today.

Over the last seven days, my workstation has been split into two universes: generating high-quality training datasets for my chess AI on a strict daily quota, and building a scalable market analysis tool for the book industry.

Here is the weekly recap, some crucial optimizations, and the roadmap ahead.

♟️ Chess Database: 4,500+ Games Cooked

The automated parallel self-play setup I described in my previous post is performing beautifully. My hardware survived the daily quota of 716 games per day.

The Weekly Total: 6 * 716 games + 260 baseline test games = 4,556 games successfully generated and saved into the local database.

The dataset is expanding exactly as planned, laying down a bulletproof foundation before I open the Jira backlog for the Reinforcement Learning model training phase.

📖 The Book Scraper: Smart Capping & Strategy

While the chess engine runs its daily simulations, I’ve been heavily focusing on my new target platform: Yakaboo (the largest book platform in Ukraine). The platform contains 16 distinct main categories.

Initially, I went deep and scraped over 410 pages for the Fiction category alone. But looking at the data, I realized I needed to optimize my traversal mechanics. I decided to introduce a strict limit: a maximum of 200 pages per category.

Why cap the dataset at 200 pages? 🤔

Online retail storefronts naturally sort their catalog by popularity, search volume, and sales velocity on the initial pages. The deeper you go into page 300+, the higher the density of out-of-print titles, placeholders, or niche products. Scraping thousands of deep pages won't help us identify active consumer trends-it just adds unnecessary noise and processing bloat to the database.

Here is a sneak peek of how the raw item structure looks right now:

    {
        "title": "Generations in the World of Work",
        "author": "Корі Сіміллер",
        "price": "2947 грн",
        "rating": "0",
        "reviews": "0",
        "status": "Фізична книга",
        "href": "https://www.yakaboo.ua/ua/generations-in-the-world-of-work.html"
    },
    {
        "title": "Paralegal Career For Dummies",
        "author": "Ліза Зіммер Хатч",
        "price": "1832 грн",
        "rating": "0",
        "reviews": "0",
        "status": "Фізична книга",
        "href": "https://www.yakaboo.ua/ua/paralegal-career-for-dummies-3295617.html"
    },
    {
        "title": "Performance Appraisals & Phrases For Dummies",
        "author": "Кен Ллойд",
        "price": "1513 грн",
        "rating": "0",
        "reviews": "0",
        "status": "Фізична книга",
        "href": "https://www.yakaboo.ua/ua/performance-appraisals-phrases-for-dummies.html"
    },
    {
        "title": "A Pluralistic Approach to Leadership: Interdisciplinary Perspectives",
        "author": "Натан В. Хартер",
        "price": "3982 грн",
        "rating": "0",
        "reviews": "0",
        "status": "Фізична книга",
        "href": "https://www.yakaboo.ua/ua/a-pluralistic-approach-to-leadership-interdisciplinary-perspectives.html"
    },

And of course some screenshots

It's a chess database fragment. Yep some games is short, and some is more then 100 moves.

I parsed Yakaboo for 3 days and already parsed almost 23MB of data.

My laptop straggling to parse data while I'm writing posts.

And what I see as a progress. Of course I could just open Chrome and watch how pages scrolling.

Building a Chess Engine in Python - Part 3: Letting the DB Cook & Launching a New Book Market Scraper

Nicholas — Thu, 16 Jul 2026 11:29:29 +0000

"What do you do when your laptop is too weak to train a neural network and run game simulations at the same time? I'm letting the database cook, and starting a new small project."

Hey guys ;)

Sorry for being a bit late with this update, but I promise I have some very exciting news.

Right now, my main Chess Project is entering a transition phase. To train a proper Reinforcement Learning model, I need a massive, high-quality dataset of self-play games. But running heavy simulations and coding at the same time is a heavy burden for my hardware.

So, I made a strategic decision: take a 2-week development break to generate the database, while kicking off a brand new Data Science project on the side.

Here is what's happening on both fronts!

♟️ Chess Project: The Daily Generation Quota

To keep my laptop healthy while still building a solid dataset, I’ve set up a strict automated daily schedule. From now until July 27th, I am running my parallelized self-play script with a specific daily quota:

716 games per day in total (distributed as 358 games per active CPU core).
The depth distribution: 300 games at depth 2, 55 games at depth 3, and 3 games at depth 4.

Even this "limited" run takes about 2 hours and 30 minutes every single day. Once the database reaches its target size on July 27th, I will officially close the generation phase and jump straight into the model training backlog, which is already fully planned and ready in my Jira!

📖 New Project: Analyzing the Book Industry

While the chess script is running in the background, I decided to tackle a question that has been on my mind for a while: Are people actually reading fewer books nowadays?

To answer this with cold, hard data, I decided to build a custom web scraper to gather, clean, and analyze retail book market trends.

The Pivot: From Amazon to Yakaboo

Initially, my first target was Amazon. However, I quickly ran into two major roadblocks:

Aggressive Anti-Bot Protection: Amazon does not like scrapers, making automated extraction highly inefficient without expensive proxy setups. Yep I parsed some book data from Amazon via undetected-chromedriver, but there is second problem.
Data Limitations: The accessible, free-tier product details were too sparse for the deep analysis I wanted to conduct. To compare on Yakaboo there is a books cards with title, author, rating and count of reviews, of course price. So for my analysis I only need to get amount of pages(for example via GoogleBooks API or just right from Yakaboo details page) and interest in certain book(I'm planning to get it via Google Trends)

So, I pivoted! My new target is Yakaboo, the largest online book platform in Ukraine. It has incredibly rich metadata regarding which is exactly what I need. And of course their web source code is pretty easier to read.

I’m currently writing the automation scripts combining Selenium and BeautifulSoup to handle paginated lists and extract the raw dataset.

What's Next?

For the next two weeks:

My CPU will be sweating to generate those 716 chess games daily.
I will be finalizing the Yakaboo parser, cleaning the scraped book data, and preparing my first analytical dashboard.

I’m super excited to share actual data insights about what people are reading (and buying) in one of the next post!

Have you ever had to pause a heavy computational task to prevent your machine from melting? What’s your go-to tool for bypassing anti-bot systems? Let’s share some stories in the comments!

#python #datascience #webscraping #chess #showdev

DEV Community: Nicholas

I think it’s time for Aster’s come back. Or just what to do when you finished your project.

What can I tell you?

First step

Second step

Third step

Final step

What’s next?

Which classification have I used? As I know it should be KNN, but stuff wasn’t so easy.

Back to Uni

Music Classifier

May I know your favorite songs?

Summer is over. But I continue my funny journey

Unexpected, but Secret Santa is cooked. Let's explore bit more about role of Django here.

Secret Santa done.

What can my Secret Santa do ?

More technical info about Django

How to send an email ?

Is it possible to code while learning something new? Of course, it’s more fun.

General

New project. What is it about?

A bit about Django

Another amazing library

A long break from post, but I’ve cocked some stuff like Walter White did.

Is pageview extending false?

First conclusions

Next step

What do marketing tags, broken author arrays, and an unexpected email from Derek Sivers have in common?

Fixing the "Comma-in-Category" Bug

Stripping "Garbage" Categories & Preprocessing

An Unexpected Reply from Derek Sivers

What's Next?

No APIs?! Analysis is almost here, but I’ve mistaken again.

Weekends with translator for book analysis

Building a Chess AI & Book Scraper - Part 5: Memory-Efficient JSON Loading & Switching to a 4/3 Work Balance

Memory-Efficient Data Ingestion in Pandas

Why not just use pd.read_json()? 🤔

The Case of the Missing 400 Pages

♟️ Chess RL Engine: Extending the Self-Play Deadline

💬 Let's Chat!

Building a Chess Engine & Book Scraper - Part 4: Capping the Catalog & 4,500+ Games Cooked

"The data pipeline is perfectly stable, the automation is running smoothly, but I'm not pushing these changes to GitHub yet." 😅

♟️ Chess Database: 4,500+ Games Cooked

📖 The Book Scraper: Smart Capping & Strategy

Why cap the dataset at 200 pages? 🤔

Building a Chess Engine in Python - Part 3: Letting the DB Cook & Launching a New Book Market Scraper

"What do you do when your laptop is too weak to train a neural network and run game simulations at the same time? I'm letting the database cook, and starting a new small project."

♟️ Chess Project: The Daily Generation Quota

📖 New Project: Analyzing the Book Industry

The Pivot: From Amazon to Yakaboo

What's Next?

Why not just use `pd.read_json()`? 🤔