DEV Community: Mikkel

AI on Mobile Devices

Mikkel — Tue, 03 Dec 2024 03:04:52 +0000

Introduction

We’ve all heard about the major advancements in AI, including large language models, image recognition, and video generation. These are relatively demanding models, requiring significant computational power and hardware capabilities. But what about smaller devices, like smartphones, smartwatches, or Internet of Things (IoT) devices? Is it possible to take a picture on your phone and, within seconds, determine if your plant is dying due to neglect or a rare disease—letting you off the hook for the blame? In this post, we’ll take a closer look at that possibility.

Computing Power on Mobile Devices

When it comes to AI on mobile devices, why can’t we just download ChatGPT and use it as we please?

The primary reason is the sheer number of mathematical computations involved in running an AI model. Mobile devices have significant limitations in terms of how many calculations they can perform per second, how much memory they can store at once, and how much power they can consume for a task.

Let’s use OpenAI’s GPT-3 as an example to understand these challenges. GPT-3 has 175 billion parameters—the numerical values used to generate responses to your questions. These parameters require about 700 GB of memory to run. Even the most expensive smartphones, typically equipped with 16–24 GB of RAM, fall far short of what’s needed to handle such a model.

Now, let’s compare this to a specialized mobile model like MobileNet, which is optimized for on-device performance. MobileNet has only 4 million parameters and requires approximately 16 MB of RAM—a massive reduction in computational and memory demands.

While this comparison is somewhat exaggerated (as ChatGPT runs on specialized servers, meaning your phone or computer doesn’t actually handle the heavy computations), it effectively illustrates the constraints of mobile devices. If you want an AI model to run directly on your phone without relying on an internet connection, it’s crucial to design and optimize the model specifically for mobile use.

Mobile Sensors

With nearly everyone owning a smartphone these days, it’s only natural to make use of the many sensors they contain. Most people are familiar with the microphone, camera, and GPS, but smartphones also come equipped with accelerometers and gyroscopes.

Audio, image, and video are relatively intuitive applications. These can be used to identify anomalies or classify events. For example, audio can be used for diagnostics in the healthcare sector or to detect animal sounds in environmental monitoring.

Where accelerometers and gyroscopes have proven particularly useful is in activity tracking. Common applications include step counters or running apps. Beyond these relatively simple uses, smartphones have the potential to play a much larger role in public health.

Gait analysis has been shown to be a valuable tool for diagnosing several cognitive disorders, such as dementia and Alzheimer’s, where the brain undergoes degeneration. Even in the early stages, deviations in a person’s walking pattern can be detected, allowing for early intervention. With specialized AI models, we could capture these changes without the need for expensive and bulky gait analysis equipment.

A simpler but still impactful application related to cognitive disorders is monitoring how often a patient is active. By tracking when the phone, and consequently the patient, is in motion, we can estimate daily activity levels. This is particularly valuable for individuals who may struggle to remember their day-to-day activities. Since physical activity is a crucial factor in slowing the progression of cognitive disorders, this type of monitoring can provide meaningful insights and support for both patients and healthcare providers.

Existing Mobile Models

There are already several AI models optimized for mobile devices and smaller hardware. A few examples include Gemini, Yamnet, and MobileNet.

Gemini is Google’s flagship AI product. It’s a versatile model with specialized versions. While it can be used similarly to ChatGPT, it also has smaller, optimized versions that make it practical for mobile use as an enhanced Google Assistant. By 2024, the mobile version is expected to function as an AI assistant capable of actively planning tasks for you, using inputs such as emails and calendar data.

Yamnet is an audio classification model built on a large YouTube dataset. It can classify sounds into 512 different categories. Available on mobile, it can be used for classification tasks and also for converting audio into spectrograms or embeddings, which can be utilized by other models.

MobileNet is an image classification model optimized for mobile devices, as its name suggests. The model is versatile and can be applied to tasks like object detection, facial recognition, and geolocation.

Use Cases

At Convai, we have worked on several mobile apps that either integrate AI or are in the process of doing so.

As mentioned in the post Sound Event Detection, we collaborated with Sonohaler, a company using a mobile app with AI to classify audio events and measure their quality. In some of our experiments, we utilized Yamnet as it offers a straightforward method for extracting spectrograms from audio data.

Another project we’re working on is Taptics, an app designed to help football clubs train and test their players’ tactical understanding. One of the simpler ways we use AI here is as a quiz assistant. The assistant can generate new questions or answer options by analyzing existing quizzes. Additionally, it can evaluate players' free-text answers by comparing them to the coach's correct responses, saving the coach from having to manually review every answer. This allows for more complex answer structures compared to multiple-choice questions, without adding an extra feedback burden on the coach.

Sound Event Detection

Mikkel — Tue, 03 Dec 2024 03:00:04 +0000

TL;DR

Sound is vibrations in the air that are converted into electrical signals—either through the ear to the brain or via a microphone to a computer. Sound has several characteristics, such as frequency and amplitude, which can be used to identify patterns. Spectrograms are a way to visualize which frequencies are present and when during a sound. They can help determine when an event starts and ends, analyze what happens during the event, or assess the quality of the event. Use cases include healthcare, where sound can evaluate the quality of a patient’s exercise, or environmental monitoring, where sound can identify which birds are present in an area.

Introduction

If a tree falls in the forest and no one is around to hear it, does it make a sound? A paradox rendered irrelevant by AI and sound event detection (SED). AI and advanced audio analysis can detect and classify sounds in various environments, even when no humans are present.

We can uncover patterns in sound that are imperceptible to the human ear. These patterns can be used to identify machine malfunctions, monitor animals in large ecosystems, or assess the quality of a patient’s condition.

Some of the tools used today include spectrograms, while the future may bring innovations like transformers and transfer learning.

Sound and Its Digital Representation

Sound consists of vibrations in the air known as sound waves. These waves are captured by the ear as they impact the membrane in the ear, commonly known as the eardrum. From there, the sound is amplified by small bones and passed into a fluid-filled structure called the cochlea. Inside the cochlea, tiny hair cells convert these vibrations into electrical signals, which are sent to the brain via the auditory nerve.

When we work with sound digitally, it must first be converted into a digital format. This is usually done through a microphone, which transforms sound waves into electrical signals. Much like the human ear, a microphone uses a membrane that reacts to sound. The membrane’s movement pushes a coil back and forth near a magnet, generating electrical signals that a computer can process.

If you’ve taken high school physics, you might remember the study of waves—and perhaps found it less than thrilling. But if you listened, you may recall that waves have various characteristics, such as frequency, intensity, and wavelength. These characteristics form the foundation for how AI processes sound.

One example of these characteristics is frequency—the number of times a wave moves up and down per second. Frequency determines whether a sound is perceived as a deep bass or a high-pitched tone. A high frequency corresponds to a higher-pitched tone, while a low frequency results in a deeper sound.

Sound in AI

When processing sound with AI, we aim to identify patterns in raw audio or its characteristics. Two of the most popular tools, spectrograms and MFCCs, focus on analyzing frequencies.

To translate sound into frequencies, a mathematical tool called Fourier Transformations (FT) is used. These transformations identify which frequencies are present in an audio clip and the magnitude of those frequencies. The result can be visualized as a graph with frequencies on the x-axis and magnitude on the y-axis, which can then be used for both filtering and pattern recognition.

An example could be a recording made in a room with wall outlets and wiring in the ceiling. In such a scenario, a significant amount of sound is often detected at 50Hz, caused by the electrical current from the power grid oscillating at that frequency. If the sound of interest occurs above 50Hz, a filter can be applied to remove lower frequencies.

Fourier Transformations are also used to create spectrograms, which allow us to analyze sound in both the frequency and time domains. To create a spectrogram, the audio is divided into smaller segments, and FT is applied to each segment. Filters are then applied to further distinguish between frequencies by dividing them into bands. Finally, these segments are combined into a plot where time is represented along the x-axis, frequency bands along the y-axis, and the magnitude of each frequency is shown through color. Without detailed analysis, one can already identify four distinct events in the clip.

Use case - Sonohaler

At Convai, we have collaborated with the company Sonohaler, which uses sound in the healthcare sector. Their mission is to make quality tools accessible to everyone by utilizing mobile devices as a replacement for traditional electronic measuring equipment. Their app incorporates AI that analyzes sound and tracks the user's progress. Convai’s role has been to work on event detection and quality measurement of the events.

For event detection, our task was to identify the start and end times of an event. We approached this as a classification problem, analyzing small segments of sound to determine whether they contained the event.

Our process began with collecting audio data containing the events of interest and annotating the start and end times. The data was then divided into small segments, each labeled as either "event" or "non-event." For each of these segments, spectrograms were generated and used in the model for training, testing, and implementation in the app. This process resulted in a model capable of analyzing an audio file and detecting where it believes an event occurs. Below is the result of the spectrogram shown earlier, with the green bars indicating where the model predicts an event.

For quality assurance, our task was to identify the intensity of an event. We approached this as a regression problem. Once again, we utilized spectrograms, as there is often a correlation between the intensity of an event and the frequency content in the sound. Similar to the classification approach, the data was broken into smaller segments and transformed into spectrograms.

The key difference here is that instead of assigning each segment a label (e.g., "event" or "non-event"), the correct answer is a continuous value representing the intensity. This intensity can range from 0 to 100, and it is up to the model to predict the appropriate intensity value.

The result is that the sound is converted into intensity values, which can later be used for analysis within the app or by a professional using the app.

The Potential of Sound

With Sonohaler, we have explored just a small portion of what sound can do for us. We utilized spectrograms for both classification and regression, but technologies like transformers, which are used in large language models, can also be applied to sound—and they are continually improving. Transformers have the potential to train on much larger datasets and to work directly with raw audio instead of relying on spectrograms.

One of the challenges of working with sound is that analysis often needs to be performed on small devices, such as phones or sensors connected to microprocessors. This makes the size and efficiency of AI models and feature extraction techniques crucial. It requires creative thinking to distill the process down to its fundamental components.

Beyond the healthcare sector, the environmental sector is another area where AI has been widely applied to sound. A classic example is identifying bird species in natural habitats. Birds can be difficult to spot, making image recognition impractical. However, each bird has a unique sound, both as a species and as an individual. By recording audio in an area and allowing AI to analyze the sound, it is possible to identify which birds are present in the audio clip.

In the industrial sector, sound is also used for anomaly detection, which involves identifying unusual sounds that are not normally present. This capability allows for the detection of machinery with reduced or faulty performance, helping to identify and fix issues before they become critical.

Some of the large-scale sound models freely available online are highly comprehensive classification models trained on thousands of YouTube videos. These models can distinguish between more than 200 different events, including people talking, cars driving by, gunshots, and other everyday occurrences. These models also provide opportunities for further training on data tailored to your specific problem.

Conclusion

Sound contains a wealth of information that can be used for far more than just playing music or facilitating conversations. By digitizing and analyzing sound, we can gain deep insights in fields such as healthcare and environmental monitoring.

Tools like spectrograms have been indispensable in recent years, but technologies like transformers may pave the way for even greater advancements. They have the potential to enable new models that perform better while remaining accessible on devices like smartphones.

We are only at the beginning of exploring what sound can offer us. With the rapid advancements in AI and machine learning, the future of sound analysis is both exciting and full of possibilities.

Retrieval-Augmented Generation

Mikkel — Tue, 03 Dec 2024 00:57:59 +0000

TL;DR

Retrieval-Augmented Generation (RAG) builds upon large language models (LLMs) like ChatGPT and Gemini by enabling them to access relevant internal and external documents. These documents can serve as references to answer questions, assist in writing reports, or support decision-making. By leveraging references, RAG increases trust in the generated responses, enables exploration of large knowledge bases, and prevents the model from "hallucinating" answers. Hosting on platforms like Azure ensures the setup remains private and GDPR-compliant.

Introduction

For over 15 years, Google has been synonymous with information retrieval. The internet became the world's library, with Google's advanced algorithm serving as our librarian. However, this paradigm has recently shifted to an all-knowing sage called ChatGPT. This sage has too much time on their hands and has read everything in the library for you. They can recall and explain it to you in seconds—even simplifying complex topics as though you were five years old.

Now imagine this sage takes a crash course in your company. Perhaps you’ve amassed hundreds of documents, including internal protocols, annual reports, and methodologies. Within hours, this sage could become an expert in your organization. This crash course is called Retrieval-Augmented Generation (RAG). With RAG, you can have an AI assistant that answers questions, generates reports, and supports decision-making based on your company’s specific knowledge base.

In this blog post, we’ll briefly explore what RAG is, the technologies behind it, and—most importantly—how it can benefit you. To understand RAG and specialized chatbots, it’s helpful to know a bit about NLP, embeddings, and LLMs.

Building Blocks

Natural Language Processing (NLP)

Natural Language Processing (NLP) is the branch of AI that deals with understanding language. It includes applications like text-to-speech, image generation from text, and speech-to-image. Some of the biggest AI breakthroughs in recent years have come from this field. OpenAI has been a key player, introducing models like ChatGPT for language and DALL-E for image generation.

Embeddings

To process human language, computers need to convert words and sentences into numbers—a process known as embedding. Text is broken into smaller pieces called tokens, and each token is assigned a vector (a series of numbers) in a lookup table. For example:

King: [0.8, 0.6, 0.1]
Man: [0.5, 0.3, 0.1]
Woman: [0.6, 0.4, 0.1]
Queen: [0.9, 0.7, 0.1]

This numerical representation enables fascinating operations. By converting words into numbers, we can perform mathematical calculations on their meanings. For example:

King - Man + Woman = Queen

King - Man = [0.8, 0.6, 0.1] - [0.5, 0.3, 0.1] = [0.3, 0.3, 0.0]
Result + Woman = [0.3, 0.3, 0.0] + [0.6, 0.4, 0.1] = [0.9, 0.7, 0.1]

Thus, the result is Queen, represented by the vector [0.9, 0.7, 0.1]. This is a simplification, but the principle is the same.

ChatGPT embeddings operate in 1,536 dimensions, enabling it to understand the semantic relationships between words and sentences far beyond this simple example.

By calculating the “cosine similarity” between two vectors, we can measure how similar the meanings of two words are. For example, if we compare the words "cat" and "dog," which have closely related vectors, we will obtain a high similarity score, indicating their semantic closeness. This enables AI models to understand the context and relationships between words in a highly nuanced way, forming the foundation for advanced NLP applications such as machine translation, sentence meaning comparison, and much more.

LLMs

The next major component of RAG is large language models, or LLMs. Popular examples include OpenAI's ChatGPT, Google’s Gemini, and Meta’s Llama. LLMs are built on transformer architectures—a revolutionary design that allows models to recognize patterns and derive meaning from vast amounts of data. For instance, ChatGPT-3 was trained on 570GB of text data, equivalent to reading a 316-million-page PDF. And newer models have only grown larger.

Transformers were introduced in Google’s groundbreaking paper, “Attention is All You Need.” Before this, language models primarily relied on recurrent or convolutional neural networks, which process data sequentially, step by step. These architectures were time-consuming and lacked the ability to run computations in parallel. Transformers addressed this by introducing attention mechanisms—a way to assign meaning to words based on their surrounding context.

This attention mechanism subtly modifies the vectors representing words. For example, in the earlier example, the word king was represented by the vector [0.8, 0.6, 0.1]. In the sentence “The king loves his queen,” the vector for king might transform into [0.4, 0.5, 0.6]. These transformations occur through a series of calculations that evaluate the relationships between word vectors.

When you ask ChatGPT a question, it responds based on all the knowledge it has accumulated during training. This foundational design enables the model to generalize and perform well across a wide range of topics. It can quickly answer questions about science, history, technology, and more, understanding the context of your inquiries to provide relevant and accurate responses. However, challenges arise when dealing with specialized tasks or ensuring trustworthiness.

A helpful analogy is to think of an LLM as a judge. The judge has extensive knowledge of laws and regulations. When faced with a new, specific case, their authority and expertise improve if they have an assistant who retrieves case files from the library containing similar rulings and procedures. In this scenario, the assistant represents the RAG functionality.

Before RAG, addressing such specialized tasks involved either fine-tuning the model—adding more data and retraining—or attempting to engineer specific prompts to guide the model’s behavior. Both methods were time-consuming and less efficient. This is where RAG comes in.

RAG

Retrieval-Augmented Generation (RAG) is an extension of LLMs. Here’s how it works: you upload your own data—this could be PDFs, PowerPoint presentations, or Word documents. Similar to how LLMs process text, these documents are divided into smaller chunks and embedded into numerical vectors. However, RAG diverges from traditional models at this point. Instead of retraining the model on this new data, the embedded vectors are stored in a vector database.

When the LLM is asked a question or tasked with generating text, it first retrieves relevant data from the vector database and uses it as context for its response. This approach allows the model to access additional data and provide answers with references.

The advantages of this method are significant:

Speed: It is much faster than retraining a model with new data.
Trust: By providing references for its responses, the model builds greater trust with users.
Customization: If you have proprietary data, such as internal processes or company-specific information, the LLM can incorporate this knowledge into its responses.

From a GDPR perspective, it’s crucial to ensure that the servers hosting your data and vectors are located within the EU. This can be a challenge, as many vector databases are hosted by U.S.-based companies. However, solutions like Azure and Supabase—which we have experience with—offer database hosting in GDPR-compliant locations such as Germany and Sweden.

Experiences

At Convai, we have spent the past few months working extensively with chatbots and RAG. Our projects have ranged widely, requiring various types of solutions. For simpler tasks, we’ve used a drag-and-drop framework called Flowise. This tool allows predefined building blocks to be arranged on a board, enabling quick and efficient solutions.

For more advanced projects that demand flexibility, we’ve turned to a Python library called LangChain, the foundation upon which Flowise is built. LangChain offers a highly customizable approach, allowing us to assemble a variety of components to process data in the way that best fits the use case. Additionally, it enables us to incorporate tailored prompts for the models when needed, adding an extra layer of precision and functionality.

Potential

As you can see, there are numerous opportunities to integrate RAG into your business. By granting an LLM access to internal documents, a chatbot can serve as a domain expert. Additionally, providing access to relevant external documents can further enhance its capabilities.

A traditional chatbot can be significantly improved by giving it access to the company’s FAQ. Beyond that, an LLM can be used to generate content for the company’s website. With tools like LangChain, the model can analyze existing content on the site, using it as a stylistic guide. This ensures that any new content the LLM generates matches the tone and format of the existing material.

If you want to use an LLM for planning purposes, you can give the model access to historical plans. Based on this data, the LLM can create new plans, such as event schedules or meeting agendas, tailored to your needs.

Usecase: TumbleLog

Mikkel — Mon, 25 Nov 2024 02:28:16 +0000

Introduction

A few weeks ago, I was contacted by my old coach, Dima Kachan, to help him out with an idea for a new training aid software.

Dima is a very successful international gymnastics coach. He has six world championship medals, across five of his athletes and two disciplines. I have been lucky enough to train with him over a period of two years (unfortunately I haven’t won him any medals) and I’ve learned a lot.

A lot of factors goes into becoming a world-class gymnast, like training intensity and recovery. One of the big factors Dima monitors, is total difficulty of skills, performed in a training session. This metric helps measure the physical demands placed on gymnasts based on how difficult skills they perform and varies depending on the training phase in a season.

Currently, Dima uses spreadsheets for tracking difficulty. Athletes manually record the number of repetitions for each skill performed during a session. However, this system is cumbersome, error-prone, and often neglected by athletes. This is where I come in.

This blog will first explain the more overall process I go through in a project like this, followed by a bit of technological decision making.

The Process

When I start new project, I use an iterative process. Rapid prototyping allows me to get a functional version into users’ hands early, ensuring the app meets their needs. Feedback is invaluable, helping me avoid spending time on unnecessary features.

The first step is to figure out the requirements. What does the app need to do? In this case, it need to replace the spreadsheet and make tracking and monitoring training, more dynamic.

After spending some time talking with Dima and looking at the existing approach, the main requirements of the app is:

Athletes should be able to track each skill they do and on what apparatus
Athletes should be able to see their difficulty for each session
Coaches should be able to see which skills the athlete has done during a session
Coaches should be able to see a weekly or monthly view of what an athlete has done
Coaches should be able to assign programs (set of skills for a session) to an athlete.

These requirements translated into the app’s core features:

Tracking: Where users choose their program and record their skills
Monitoring: Where athlete and coach can monitor completed sessions and difficulty
Programming: Where the coach can put skills into programs and assign them to athletes

With the features set and confirmed with Dima, I can start developing.

To ensure the app is useful from the start, I prioritise features in this order:

Tracking → Monitoring → Programming

This sequence allows me to collect and review training data early, gradually adding more functionality. Each feature is deployed on GitHub Pages for Dima and his athletes to test, providing immediate feedback. These insights guides refinements before I move on to the next feature.

Technical Insights

Tech Stack

My base tools of choice are Flutter and Supabase. Flutter is a cross-platform frontend framework, which allows me to make apps for both Android, iOS and web. Supabase is my backend which delivers database and authentication, it is open source and it runs on PostgreSQL.

These choices came out of necessity, as I had to implement ML models into a Flutter app, in a previous job for a medical monitoring app. I fell in love with both Flutter and Supabase and have used it since.

I wanted to try out the MVC design pattern, as it seems to be the standard in a lot of companies, but as the app grew, the unfamiliar framework got in the way of quality. The controllers got too bloated and the interplay between the different components wasn’t clear to me.

Because of this, I switch to Clean Architecture, which I’ve worked with in other projects. For me it offers more granularity, as the organisation into individual features makes a lot of sense to me and it ensured that each feature remained independent and testable, simplifying future maintenance and scalability as more users are added to the app. I make use of BLoC for state management as well, which makes my widgets very clean and simple.

Features

With the foundation of tools, architecture, and state management in place, I focus on implementing the core features that will bring the app to life. Each feature is designed to replace the existing spreadsheet workflow while prioritising ease of use and functionality for both athletes and coaches.

Tracking

The main functionality from the spreadsheet, that I replace, is tracking. It needs to be quick and simple, so the athletes wont get their training disturbed, but it just becomes a habit. In order to accomplish this, I come up with two designs. A button layout and a bar layout.

The design needs to fit in adding, subtracting, reseting and editing a large number of skills. As you can see, my solution is a toggle menu for layout and skill name format, allowing the athletes to smoothly go back and forth. This makes it possible for the athlete to customise the experience, while utilising all the features with one tap.

The functionality itself is a simple set of counters stored in BLoC and uploaded to Supabase. The challenge with this is how to format the data. Sessions and Skills are the two main models, but within them, they need to keep track of data for each piece of equipment. Although I don’t believe it’s the best solution, I’ve used a map to solve this. This keeps the data structure compact but may require additional handling for custom equipment in the future.

Session Model:

import 'package:equatable/equatable.dart';
import 'package:json_annotation/json_annotation.dart';
import 'package:tumblelog/constants.dart';

part 'session_model.g.dart';

@JsonSerializable(explicitToJson: true, fieldRename: FieldRename.snake)
class SessionModel extends Equatable {
  final String id;
  final String athleteId;
  final String athleteName;
  final DateTime date;
  final double totalDd;
  final Map<EquipmentType, double> equipmentDd;

  const SessionModel({
    required this.id,
    required this.athleteId,
    required this.date,
    this.totalDd = 0,
    this.athleteName = 'NoName',
    this.equipmentDd = defaultEquipmentDd,
  });

  // Factory method to create a SessionModel from JSON
  factory SessionModel.fromJson(Map<String, dynamic> json) =>
      _$SessionModelFromJson(json);

  // Method to convert a SessionModel to JSON
  Map<String, dynamic> toJson() => _$SessionModelToJson(this);

  @override
  List<Object> get props =>
      [id, athleteId, athleteName, totalDd, date, equipmentDd];
}

Skill Model:

import 'package:equatable/equatable.dart';
import 'package:json_annotation/json_annotation.dart';
import 'package:tumblelog/constants.dart';

part 'skill_model.g.dart';

@JsonSerializable(explicitToJson: true, fieldRename: FieldRename.snake)
class SkillModel extends Equatable {
  final String id;
  final String sessionId;
  final String name;
  final String symbol;
  final double difficulty;
  final Map<EquipmentType, int> equipmentReps;

  const SkillModel({
    required this.id,
    required this.sessionId,
    required this.name,
    required this.symbol,
    required this.difficulty,
    required this.equipmentReps,
  });

  // Factory method to create a SkillModel from JSON
  factory SkillModel.fromJson(Map<String, dynamic> json) =>
      _$SkillModelFromJson(json);

  // Method to convert a SkillModel to JSON
  Map<String, dynamic> toJson() => _$SkillModelToJson(this);

  @override
  List<Object> get props =>
      [id, sessionId, name, symbol, difficulty, equipmentReps];
}

As you can see, I use json_serializable, to automate the transformation from model to json. I do this to avoid a bunch of human errors I’ve made in the past. It adds more complexity up front, but saves me a lot of time down the line.

Monitoring

The next feature is monitoring, which lets the coaches and athletes see their progress. After doing the data ground work in the tracking feature, this is relatively straight forward. The data is just fetched an displayed nicely.

This design lets the coach get an overview of how much work the athlete has done during the week, and make decision for the upcoming training. For a more detailed view, the coach can see the individual session.

Programming

Finally there is the programming part. The goal is to give coaches the option to customise which skills an athlete needs to do in a session and assign this new program to athletes. Four new tables are added to the database:

Programs: For tracking the specific programs
SkillLibrary: A database of skills for coaches to choose between
ProgramSkills: For linking programs to skills
AthletePrograms: For linking athletes to programs

This setup makes the database flexible and easy to scale, as you can add as many links as needed. The drawback is that queries can be a bit more challenging:

final programsJson = await supabaseClient
  .from('programs')
  .select(
      '*, athlete_programs!inner(), program_skills(skill_library(*))')
  .eq('athlete_programs.athlete_id', userId);

The model can also get more bloated, but for me this is worth it, as I can automate conversions.

ProgramModel:

import 'package:equatable/equatable.dart';
import 'package:json_annotation/json_annotation.dart';
import 'package:tumblelog/core/models/skill_library_model.dart';

part 'program_model.g.dart';

@JsonSerializable(explicitToJson: true, fieldRename: FieldRename.snake)
class ProgramModel extends Equatable {
  final String id;
  final String name;
  final String? creatorId;

  @JsonKey(name: 'program_skills', fromJson: _extractSkillsFromProgramSkills)
  final List<SkillLibraryModel> skills;

  const ProgramModel({
    required this.id,
    required this.name,
    required this.creatorId,
    required this.skills,
  });

  /// Factory method to create a `ProgramModel` from JSON
  factory ProgramModel.fromJson(Map<String, dynamic> json) =>
      _$ProgramModelFromJson(json);

  /// Method to convert `ProgramModel` to JSON
  Map<String, dynamic> toJson() => _$ProgramModelToJson(this);

  /// Helper method to convert `program_skills` into a list of `SkillLibraryModel`
  static List<SkillLibraryModel> _extractSkillsFromProgramSkills(
      List<dynamic>? programSkills) {
    if (programSkills == null) return [];
    return programSkills
        .map((e) => SkillLibraryModel.fromJson(e['skill_library']))
        .toList();
  }

  /// Convert `ProgramModel` to a map for database saving
  Map<String, dynamic> toProgramsTableMap() {
    return {
      'id': id,
      'name': name,
      'creator_id': creatorId,
    };
  }

  /// Map skills to a list of `program_skills` entries
  List<Map<String, dynamic>> mapSkillsToProgramSkills(String programId) {
    return skills.map((skill) {
      return {
        'program_id': programId,
        'skill_id': skill.id,
      };
    }).toList();
  }

  @override
  List<Object?> get props => [id, name, creatorId, skills];
}

The end product of the programming pages looks like this:

That is the rough outline of the app. As you can see, I am not a design genius and have gone with either simple or default colors. I have mainly focused on building a functioning prototype.

The Outcome and Lessons Learned

Now that the project is done, it needs to get used. Every time I have implemented a new feature, I have sent it to Dima for him to test with his athletes. The feedback has been very positive, but will the app stand the test of time?

Will the design be simple enough for athletes to seamlessly integrate it into their training routines without distractions or missed entries?

As the app is being used and more data is being recorded, hopefully we will be able to see patterns in training frequency, difficulty and performance or injuries.

I might make a small workshop with the club, to see them use the app and identify pain points.

Refactoring the app to Clean Architecture was a key moment where I focused on applying principles like separation of concerns and DRY. This approach made the app more maintainable and scalable.

As the app continues to be used and evolves, I'm excited to see the insights it brings to Dima and his athletes. This project has been a significant step in my growth as a developer, and I'm looking forward to tackling even more challenging projects in the future.