DEV Community: Svitla Systems Inc.

AI is more material than it looks: 5 ways to reduce inference cost and risk

Svitla Systems Inc. — Tue, 30 Jun 2026 12:09:32 +0000

The AI ecosystem is maturing, making operational costs more visible, particularly the energy consumption of inference and the increased security risks in agentic architectures. Now they're embedded in supply chains, risk models, and healthcare, and nobody's treating them as optional.

As they scale, technical leaders must recognize AI’s tangible infrastructure and operational limitations.

Agentic AI is powerful. However, while delegating building tasks to autonomous systems, it's important to remember that underlying material limitations do not disappear but simply scale out of sight.

These hidden risks become more urgent in code review and generation. When agents write and review code, they can bypass traditional security guardrails and introduce vulnerabilities that propagate across the system at scale.

To understand how to regain control over this AI-generated technical debt, we must first debunk the myth of the 'immaterial' cloud, moving us directly into a conversation about AI's tangible realities.

I’m Patricio Gerpe, a Senior AI Engineer and consultant with global experience in AI startups, applied research, and social-impact projects. Working in both high-compute and resource-limited environments led me to focus on inference efficiency and energy-aware systems.

The industry already has emerging security frameworks such as the OWASP LLM Top 10. What is still missing is a similarly practical engineering mindset around inference efficiency and operational sustainability. In this article, we will review five practical engineering practices to reduce inference waste and help teams build AI systems that remain efficient and sustainable in production.

The reality check: the materiality of AI

For too long, AI has been framed as a weightless abstraction, but real-world deployments are tightly bound by computing capacity, energy availability, and cooling infrastructure.

Beneath the sleek APIs, there is also a very real human layer: large workforces of data labelers and content moderators who continuously correct and curate model inputs and outputs so that systems appear “seamlessly intelligent.”

Research on “ghost work” documents how this invisible labor is often outsourced to workers in the Global South under precarious conditions. Some moderation and labeling pipelines reportedly pay only a few dollars per hour.

When we scale these systems, we are expanding a global supply chain for energy, water, and, in many cases, low-cost labor. Recent analyses indicate that, in many production deployments, inference can account for a larger share of total energy consumption than one-off training runs.

At the same time, the water required to cool data centers is substantial. Studies suggest that extended interactive workloads can consume hundreds of milliliters of water per multi-turn session, depending on the model and cooling infrastructure.

Simultaneously, as we move past simple ReAct (Reason-Act) patterns into continuous cognitive loops, orchestrated in frameworks like OpenClaw (Think -> Plan -> Act -> Observe), the risk surface expands.

By executing these loops through periodic background heartbeats, agents maintain temporal persistence. This persistence changes the threat model. Vulnerabilities such as indirect prompt injection or excessive agency stop being isolated at events and become persistent operational risks. If the system is physical, and its execution loops are continuous, how should we measure its efficiency?

This brings us to an uncomfortable conversation. One metric has become increasingly common in startup AI teams: “tokens burned.”

Tracking tokens as a proxy for system productivity has become standard practice. However, interpreting increased token usage as higher productivity is risky and can be misleading. While token count reflects the amount of computing resources consumed by the system, it does not measure the actual value delivered to users or stakeholders.

As architectures become more complex, a high token count can just as easily indicate inefficient model use, uncontrolled agent loops, or redundancies as it can real work performed. We must consciously differentiate between token consumption driven by necessary inference and signaling waste or poorly constrained workflows. Are we truly measuring value created, or simply measuring compute consumption?

Consider when an agentic workflow transforms a single user request into unpredictable internal inferences that inflate token usage.

Is a rising token count a true indicator of productive computation, or is it a vanity metric that hides inefficiencies? Recognizing the problem is only half the battle. To build reliable AI, we need better architecture. From a cybernetic perspective, resilience requires feedback mechanisms and proactive limits to prevent runaway resource use.

5 ways to build resilient agents

If you are looking to engineer these boundaries and ensure the long-term viability of your agents, I suggest implementing these five technical strategies:

1. Right-size language models (RLMs)

Fundamentally, the industry still pursues trillion-parameter models, but most tasks like routing, classification, extraction, and summarization rarely require such a scale. Smaller, task-specific models, properly tuned, typically reduce latency and resource consumption, often without notable performance loss on target KPIs.

2. Token-efficient prompting

Once the model is properly sized, the next step is to reduce unnecessary token generation. True "macro" Green AI optimization, which is renewable energy and efficient cooling, is managed by cloud providers.

However, we have direct control over how prompts are constructed. Unbounded output generation wastes compute cycles. We can mitigate this by engineering prompts to explicitly request concise outputs. A relevant example is the viral community project "Caveman", which forces AI to output text without grammatical filler.

This project shows that aggressive brevity limitations can yield great reductions in token usage in suitable tasks. Rather than treating such numbers as guarantees, technical leaders should benchmark brevity strategies on their own workloads and report on the actual token and latency savings observed.

3. Caching management

Efficiency also depends on reducing redundant computation across repeated requests. High-throughput agentic loops suffer from massive memory issues if unoptimized. I particularly recommend structuring your requests to use Prompt Caching APIs (like OpenAI's native implementation).

Where available, prompt caching APIs allow you to front-load static content: system prompts, schemas, and tool definitions into a cacheable prefix. Subsequent requests that reuse the same prefix can avoid recomputing input tokens. This can reduce input token cost and improve Time-to-First-Token (TTFT) under supported conditions.

4. Sanitization management

Of course, an efficient system is useless if it isn't secure. When frameworks give an LLM direct access to tools or environments, relying solely on a system prompt to enforce safe behavior is not a good security strategy.

Treat pre- and post-inference sanitization as core requirements: validate outputs with strict JSON schemas, enforce allow lists, and apply input/output size limits. Isolate agent runtimes with dedicated VMs or containers, quotas, and network policies. The Principle of Least Privilege helps ensure sensitive systems remain protected, even if prompts are compromised.

5. Chain-of-thought management

Finally, overusing reasoning-optimized models or chain-of-thought prompts for simple or deterministic tasks is a common source of unnecessary compute consumption. However, not every decision requires probabilistic reasoning.

In many workflows, deterministic rules or heuristics are enough. In those cases, it is often more efficient to implement the logic outside the LLM and reserve inference only for tasks that genuinely require semantic interpretation. This separation keeps marginal inference costs more predictable and makes the overall decision process easier to audit.

The final word: Engineering for the long term

As AI systems mature, efficiency is becoming an engineering discipline in its own right. The conversation around AI is shifting from raw model capability toward disciplined AI engineering.

As frameworks such as the EU AI Act move toward enforcement, organizations will face increasing scrutiny over how AI systems are operated, monitored, and governed, not only what they can generate. For many teams, “Safe & Green AI” becomes a practical engineering goal: building systems that are secure by design, aligned with applicable regulations, and efficient enough to be sustainable at scale.

Ultimately, efficiency is a proxy for architectural quality. By bounding your execution environments, right-sizing your models, and prioritizing deterministic guardrails, you ensure that your AI infrastructure remains viable.

I highly encourage technical leaders to audit current AI pipelines against the OWASP LLM Top 10 and ask themselves: are we building systems we can sustain?

Operationalizing AI agents requires disciplined systems engineering and a clear understanding of infrastructure limitations. As AI systems move from experimentation to operational infrastructure, many teams discover that scaling models is easier than scaling governance, efficiency, and resilience.

Svitla Systems supports clients in assessing current AI pipelines, designing secure and efficient architectures, and implementing managed services that keep operational risk and resource consumption under control.

Whether you need help implementing Managed Services in IT or auditing your cloud deployments, Contact Svitla Systems today to explore how our experts build software that is secure by design and efficient by necessity.

Written by
Patricio Gerpe
Senior Full Stack AI Engineer

How to work with AI: From prompting to prompt thinking

Svitla Systems Inc. — Tue, 30 Jun 2026 11:59:18 +0000

AI is only as good as the instructions you give it. The tool is almost beside the point, your thinking matters more.

Today, almost every company has already deployed AI tools across their teams. Costs are lower than expected, and productivity numbers look decent on paper. But the outputs are generic. The decisions built on those outputs carry hidden risk. And nobody can quite explain why.

The gap is input.

AI does not interpret intent. It responds to structure. And when the structure is vague, the model fills the gaps with whatever is statistically plausible, whether or not it is accurate.

AI is only as good as your instructions

There is a simple truth about working with AI that most people figure out the hard way: the quality of what you get back is almost entirely determined by what you put in.

Large language models do not retrieve facts. They predict the most probable next word based on patterns in their training data. That is a huge distinction.

So, these systems can produce outputs that sound authoritative, flow logically, and cite plausible sources while being entirely wrong. MIT researchers found that AI models use 34% more confident language when generating incorrect information than when generating correct information. We should understand that the model does not know if it is wrong.

Of course, this is not a reason to avoid AI. However, it is a reason to approach it with structure and verification from the very beginning. That is where prompt thinking comes in.

What is prompt thinking?

There is a difference between typing a question into an AI tool and thinking through what you need before you type anything.

Prompt thinking means doing cognitive work first. You define the outcome you want, identify the limitations, supply the relevant context, and specify the format. Only then do you write the prompt.

For example: a vague brief to a junior colleague produces a vague deliverable. A specific one -here is the goal, here is the audience, here are the limitations, here is what success looks like - indeed produces something you can use. AI works exactly the same way.

A useful structure is Role, Context, Task, Format. Instead of:

"Give me an analysis of our Q3 performance."

Try:

"Act as a business analyst reviewing SaaS metrics. Our Q3 revenue grew 12%, but churn increased from 4% to 6.5%. Identify the three most likely causes and suggest one corrective action for each. Use plain language. Keep the response under 300 words."

The second prompt leaves little to interpretation, and that is the point.

Organizations that implement structured prompting frameworks report average productivity improvements of 67% across AI-enabled processes, compared to informal approaches that see minimal gains despite similar technology investments.

Dr. Janna Lipenkova, an AI researcher who studies how organizations build prompting into scalable practice, puts it directly: “Prompts that solve recurring problems can be the seed of reusable AI workflows. The prompts your team writes today are proofs-of-concept for the processes of tomorrow.”

Why this matters: how AI works

To understand why prompt thinking is extremely important, first of all, we need to figure out how modern AI systems generate output.

As you know, artificial intelligence has evolved over several stages. Machine learning-powered systems detect patterns in data. Deep learning extended this through layered neural networks, making it possible to process text, images, and speech.

From there, models split into two broad types: discriminative models, which classify inputs, and generative models, which create new content.

Large language models belong to the latter category. They generate text by predicting the most probable next token based on patterns learned during training. This leads to an important point: AI does not inherently evaluate truth. It generates outputs based on probability and pattern matching.

According to a study by AllAboutAI, AI hallucinations cost businesses $67.4 billion in losses. And a Deloitte survey found that 47% of enterprise AI users made at least one major decision based on content the model fabricated.

Air Canada discovered the price of AI mistake when a tribunal forced the airline to honor a bereavement discount its AI chatbot had invented and confidently told customers about.

AI fails predictably when inputs are vague or underspecified. This is also why small changes in phrasing can affect the output. The model is sensitive to structure. The more you give it, the more reliably it delivers what you need.

Prompt thinking in practice: externalizing your thinking

AI works only with the information it is given. Think of the context window as a form of short-term memory: the more relevant and well-organized your input, the more focused is the output.

Prompt thinking requires you to make your reasoning explicit. Instead of expecting the system to figure it out, you guide it step by step toward the desired outcome.

Good prompts rely on the same things that make human communication work, covering clarity, specificity, and context. The only thing you can skip with AI is empathy as the model does not need to feel understood. It just needs the right instructions.

One effective framework to simplify the process includes four components:

Role - who the AI should act as
Context - relevant background information
Task - what needs to be done
Format - how the output should look

Prompting as a process, not a one-time action

One of the most common mistakes teams make is treating prompting as a single event. Write the prompt, get the output, move on. In practice, effective prompting is iterative. You generate an output, evaluate it, refine your instructions, and repeat.

Consider how a financial analyst at a mid-size firm might work. They ask the model to summarize competitor pricing. The first output is broad and lacks the regional specificity they need.

They refine the prompt to specify the geography, the customer segment, and the pricing dimension they care about. The second output is sharper. They push further, asking the model to surface its assumptions and flag where it is uncertain. Now they have something they can verify and act on.

Each iteration reduces ambiguity. Each pass makes the output more aligned with the initial goal. The analyst is thinking through the problem more clearly because of querying it.

_The act of structuring a prompt forces you to clarify what you want in the end. The query sharpens your own thinking before the model produces anything. _

The right technique for the right task

Different prompting approaches suit different problems. None is universally correct.

Zero-shot prompting works when the task is clear and simple. Ask the model to summarize a document, reformat data, or draft a routine email. It is fast and requires no setup.
Few-shot prompting improves consistency for tasks that require a specific style or format. Provide two or three examples of the output you want, and the model adapts to that pattern.
Role-based prompting controls tone and expertise level. Telling the model to act as a senior legal reviewer produces a different output than asking it to act as a plain-language editor. Both can be useful for different audiences.
Chain-of-thought prompting asks the model to show its reasoning step by step. This is useful for complex analysis, but it is a debugging tool, not a trust signal.

Chain-of-thought: useful, but not magic

Yes, asking AI to think step by step can improve outputs because it externalizes its reasoning. This works for complex tasks, and it lets you inspect how an answer was constructed, not just what the answer is.

Lipenkova compares this to how people work through difficult problems: instead of jumping to a conclusion, you decompose the problem into substeps and address each one. For the model, this translates into more tokens and more opportunity to reason explicitly before committing to an answer.

However, if an early assumption is wrong, the rest of the reasoning can still appear coherent, built on a flawed foundation. A clean chain of reasoning built on a bad premise is still wrong.

A more reliable approach is combining structured reasoning with explicit uncertainty:

_"Show your reasoning and indicate where you might be unsure." _

Chain-of-thought improves transparency but it doesn’t affect correctness.

Thinking is not trust: why verification is a must

This is the point that saves companies real money. AI sounds 100% certain regardless of whether it is right. Stanford's Human-Centered AI Institute found that AI models hallucinate on legal queries between 69% and 88% of the time, depending on the model and the question type. Even purpose-built legal AI tools hallucinate more than 17% of the time.

Courts have imposed sanctions exceeding $10,000 in multiple cases involving AI-generated fabricated citations and by May 2025, most of those cases involved practicing lawyers, not self-represented parties who might be excused for not knowing better.

Verification at a practical level means checking facts independently, confirming that cited sources really exist, and asking the model directly what assumptions it made and where it might be wrong. A good thing to do is rephrase the same question differently, so if you get a totally different answer, be skeptical of both.

The most useful prompt you can add to any high-stakes output:

"Review your response. Identify any claims you are not confident about, any assumptions you made that I did not explicitly confirm, and any areas where a different interpretation of my question might produce a different answer."

That single follow-up prompt surfaces more useful information than most teams get from three rounds of generic questioning.

At a basic level:

Check facts, numbers, and claims independently
Verify that cited sources exist and support the argument

At a deeper level:

Ask the model to surface its assumptions
Rephrase the same question to test consistency
Compare outputs across different prompts or sessions

What good looks like in practice

A healthcare technology company using AI for clinical documentation reduced its output-review time by 40% after implementing structured prompt templates across its documentation team.

The templates specified in the clinical context, the required format, the audience, and a mandatory uncertainty flag for any claim the model was less than certain about.

An e-commerce retailer cut content production time by 73% using few-shot prompting for product descriptions but only after investing two weeks in building a validated prompt library with clear brand-voice examples. The upfront investment paid back in the first month.

A legal team at a professional services firm stopped using AI for first-draft research after a hallucinated citation made it into a client brief. They rebuilt the workflow with a verification step: every AI-generated reference gets cross-checked against an official legal database before it moves forward.

In each case, someone thought it through first and didn't just take the output at face value.

Data privacy: a practical reminder

Keep in mind that AI systems are not secure environments. Inputs can be processed or stored in ways you do not fully control. It is easy to overlook, and often costly when people do.

Avoid putting into any AI tool:

Personal data, such as names, addresses, identification numbers
Financial information
Medical records
Credentials, e.g., passwords, API keys
Confidential business data

Use incognito or temporary chat modes when you can and check your organization's data retention settings. Getting better results from AI is part of it, but so is not being careless with what you share.

What this comes down to

What this comes down to

Here is the sobering truth: most teams are getting poor results because they are treating a thinking problem like a technology problem.

AI does not rescue unclear thinking in any way but only scales it. Feed it a half-formed question and it returns a half-formed answer confidently, fluently, and fast enough that you might not notice until it is already in a deck or a brief or a decision.

The teams that have figured this out are not doing anything exotic. They just define what they need, not just what comes to mind first. They push back on the first output instead of accepting it. Over time, that habit compounds and their prompts get sharper and their workflows get tighter.

That is what prompt thinking really is. It is the discipline of bringing your best thinking to the tool instead of expecting the tool to do that work for you.

So, before the next prompt, take an extra sixty seconds. Define the outcome, set the limitations, and give the model something real to work with.

Written by
Kseniia Ryzhkova
Marketing Analyst Intern

How-to use AWS System Manager to optimize cost of Dev environment

Svitla Systems Inc. — Tue, 30 Jun 2026 11:11:54 +0000

Source code GitHub

Introduction

When we design VPC according to the best practices, all resources like EC2 instances or Lambda Functions that don't require public access should be put to private subnets. Sometimes they could require access to the Internet, for example, for integration with external services, and it is where we need to implement a NAT (Network Address Translation) solution. In AWS, NAT could be implemented using NAT gateways or NAT instances. NAT Gateway is the recommended, convenient managed service that eliminates manual work.

On the other hand, NAT Gateway is quite expensive, especially for development and testing environments that do not require permanent outbound access 24/7. For example, NAT Gateway in the eu-central-1 region costs approximately $37 per month per 1 AZ plus cost for data processed, that gives more than $70 per month even with low traffic. In addition, a NAT Gateway resource costs money even when idle, and this quickly adds up across regions and accounts.

In this post we discuss how to reduce cost for Dev or other non-critical environments by removing NAT Gateways and stopping EC2 instances and restoring it back using AWS System Manager Automation.

Task

Let’s formulate the task: to reduce cost of non-critical environments by disabling outbound Internet access when it is not needed, while keeping infrastructure consistent with IaC and avoiding manual changes.

Possible Approaches
There are several ways to address this problem. First, you should review the environment to make sure outbound Internet access via NAT is indeed required and consider other infrastructure designs. For example, if outbound access is added to provide access to public AWS services like S3, adding VPC endpoints is an alternative design.

For non-critical environments with low traffic where high availability is not a strict requirement, you may use a single NAT Gateway for subnets in several Availability Zones. Despite this approach reduces NAT hourly rates, it has the following disadvantages:

Single NAT Gateway is charged even if it is idle.
Traffic between AZs increases latency.
Traffic between AZs introduces inter-AZ data transfer charges.
Adds single point of failure - any issues with routing in the public subnet where NAT Gateway is deployed blocks Internet access for all private subnets.

Further, most development teams use IaC tools that allow control and automate infrastructure deployments. Metadata of resources deployed by IaC tool are kept in the tool state, so deployment could be broken if resources are removed and recreated manually. A close approach is to introduce a Boolean variable and create a NAT Gateway depending on the value of the variable.

It helps to set up CI/CD pipelines with different settings that at morning updates the environment, creates NAT Gateways and starts EC2 instances, and in the evening removes NAT Gateways and stops EC2 instances. From my personal experience, such an approach adds extra complexity, as it's difficult to set an optimal time for such pipelines and is hard to understand and manage. But it still can be a good option if the team needs to remove or shut down many resources for the deployments.

To simplify deployments, IaC tools may deploy NAT Gateways for environments where they are permanently used and exclude NAT Gateways from the deployments to non-critical environments. In this case, engineers may create CLI scripts to manually create NAT Gateways and remove them when the environment is not in use. This approach is also time-consuming, and often some errors may pop up; it is hard to manage manual scripts and analyze history.

An alternative to fragile, manual scripts is AWS System Manager Automation - auditable, secure, and event-driven workflows that integrate natively with AWS services. Using AWS SSM documents provides a lot of benefits:

Visual presentation of automation runbook graphs with steps, loops, branches, and failure transitions helps to understand the step sequence and simplifies further development.
Automation documents are idempotent by design, they are declarative and state-aware, so re-running them does not lead to partial or repeated execution.
Every execution is logged, timestamped, and stored in AWS.
Documents support step-level failure handling, branching, retries, and rollback logic without custom code.
Runbooks can be reused consistently across environments without copying or modifying content.

There is no built-in feature of Automation that runs runbooks on a cron expression. We may use EventBridge rules to restore the resources in the morning and reduce them in the evening, but that part is out of the post scope.

So, let’s design several AWS Systems Manager Automation documents for our task.

Solution

In the next sections, we consider the following runbooks that reduce the cost of non-critical environments:

NAT Gateways are provisioned on demand.
NAT Gateways are removed when not needed.
EC2 instances are stopped.
EC2 instances are started.

These documents manage resources in a predictable way, and don't require recreation of VPC or any other resources deployed by an IaC tool. It is not a complete set of potential reductions, so you may stop other resources like the RDS databases or create an autoscaling group that reduces the number of running instances to 0 when there are no user loads.

Background

The solution uses Change Management tools for AWS System Manager, Amazon VPC, Amazon EC2, and Terraform.

As the infrastructure baseline, the source code includes the Terraform project, which creates

a VPC with 2 public and 2 private subnets, required security groups;
IAM role for EC2 instance and IAM role that allows Automation to perform the actions;
ALB, Target group;
EC2 instances in each private subnet.

Obviously, it is not complete infrastructure, and the real infrastructure uses custom AMI, SSL certificate and HTTPS traffic only, has Route53, Web ACL, and other resources that we don’t include to keep things simple. As we discussed above, NAT Gateways are deployed for UAT/Prod environments only. For the Dev environment, NAT Gateways should be created on demand or automatically during working hours.

Initial deployment requires outbound access since the Terraform project includes EC2 instances with standard AMI and user data. For this demonstration, you need to deploy a Terraform project twice with local create_nat = true to deploy NAT Gateways and launch instances. Then revert this value back to create_nat = var.environment != "dev" and redeploy the environment.

Resource diagram is demonstrated below.

Fig. 1. Dev and UAT environments deployed by Terraform.

AWS System Manager Documents

In addition to the mentioned documents, the solution includes 2 compound documents: Reduce-Environment and Restore-Environment, which execute other documents. So, an engineer may execute these documents instead of executing each individual document. Let’s review them.

Reduce Environment

Runbook Reduce-Environment deletes NAT Gateways and stops running EC2 instances.

Workflow

Execute Delete-NATGateways runbook, with 2 retries and 10 minutes timeout. On failure, runbook exits.
Execute Stop-EC2Instances runbook, with 2 retries and 10 minutes timeout.

Number of retries and timeouts could be updated according to your experience.

Fig. 2. Reduce-Environment graph.

Parameters

Runbook has the following parameters:

VpcId (Required) - is a mandatory parameter that identifies the target VPC where resources would be reduced. If you plan to regularly run this runbook for the same VPC, you may set up the default value, and it simplifies runbook execution, either via console or CLI script.
AutomationAssumeRole (Required) - is a mandatory parameter, which provides the ARN of the role that allows Automation to perform the actions on your behalf. Individual required permissions are listed in the source files, but they could be assigned via AmazonVPCFullAccess and AmazonEC2FullAccess policies. In addition, Terraform code creates a SSMDemoAutomationRole-<> that has required permissions and could be used to execute runbooks.

Let’s note that Delete-NATGateways and Stop-EC2Instances have the same parameters.

Delete NAT Gateways

Runbook Delete-NATGateways deletes all available NAT Gateways in the given VPC, removes routes in private route tables that use them, and releases Elastic IPs. As the further development of this runbook, it may review NAT Gateways in public subnets only, or remove only those NAT Gateways, which were deployed by Add-NATGateways runbook. To implement this, Add-NATGateways runbook needs to store NAT Gateways identifiers in SSM Parameter Store parameter(s). Then, Delete-NATGateways reads these values and iterates through it.

Workflow

Search for all NAT Gateways in the given VPC.
Delete routes in private route tables that direct outbound traffic (0.0.0.0/0) through the NAT Gateway.
Remove NAT Gateways and wait until all of them are fully deleted.
Release allocated Elastic IP addresses.
How-to use AWS System Manager to optimize cost of Dev environment

Fig. 3. Delete-NATGateways graph.

Parameters

VpcId: (Required) Id of the target VPC where to delete resources.
AutomationAssumeRole: (Required) The ARN of the role that allows Automation to perform the actions on your behalf.

AutomationAssumeRole should have at least the following permissions:

ec2:DescribeRouteTables
ec2:DescribeNatGateways
ec2:DeleteRoute
ec2:DeleteNatGateway
ec2:ReleaseAddress

Stop EC2 Instances

Runbook Stop-EC2Instances stops all running EC2 instances in private subnets of the given VPC. To cover more scenarios, it could stop instances based on the name pattern or image ID.

Workflow

Describe all private subnets in the given VPC.
Search for and collect running EC2 instance IDs in those subnets.
If any, stop the instances and wait until they are fully stopped.
Stop-EC2Instances graph AWS system manager

Fig. 4. Stop-EC2Instances graph.

Parameters

VpcId: (Required) Id of the target VPC where to delete resources.
AutomationAssumeRole: (Optional) The ARN of the role that allows Automation to perform the actions on your behalf.

AutomationAssumeRole should have at least the following permissions:

ec2:DescribeSubnets
ec2:DescribeInstances
ec2:StopInstances

Restore Environment

Runbook Restore-Environment adds NAT Gateways and starts stopped EC2 instances.

Workflow

Execute Add-NATGateways runbook, with 2 retries and 15 minutes timeout. On failure, runbook exits.
Execute Start-EC2Instances runbook, with 2 retries and 10 minutes timeout.

Number of retries and timeouts could be updated according to your experience.

Fig. 5. Restore-Environment graph.

Parameters

Runbook has the following parameters:

VpcId (Required) - is a mandatory parameter that identifies the target VPC where resources would be updated. If you plan to regularly run this runbook for the same VPC, you may set the default value, that simplifies runbook’s execution - either via console or CLI script.
*ProjectTag *(Optional) - is a value for the Project tag (default value - 'blog').
*EnvironmentTag *(Optional) - is a value for the Environment tag (default value - 'dev'). ProjectTag and EnvironmentTag are provided to assign tags to the new NAT Gateways. Depending on the application, you may add other required tags and assign default values for them.
AutomationAssumeRole **(Required) - is a mandatory parameter, which provides the ARN of the role that allows Automation to perform the actions on your behalf. Individual required permissions are listed in the source files, but they could be assigned via **AmazonVPCFullAccess and AmazonEC2FullAccess policies. In addition, Terraform code creates a SSMDemoAutomationRole-<> that has required permissions and could be used to execute runbooks.

Let’s note that these parameters are passed to Add-NATGateways and Start-EC2Instances documents.

Add NAT Gateways

Runbook Add-NATGateways looks up for NAT Gateway and creates a NAT Gateway in each public subnet of the specified VPC if it doesn't exist. Then it runs a Python script that updates the private route tables to route internet-bound traffic. Public and private subnets are identified by names - they should include ‘public’ and ‘private’, respectively. In most cases, it is the correct assumption.

Workflow

Identify all public subnets in the specified VPC.
For each subnet, check if a NAT Gateway already exists.
If no NAT Gateway is found, create a new NAT Gateway and assign Elastic IP address.
Wait until NAT Gateway is fully available.
Update private route tables to direct outbound traffic through the NAT Gateway in the public subnet in the same AZ.

Fig. 6. Add-NATGateways graph.

Parameters

Runbook has the following parameters:

VpcId: (Required) Id of the target VPC where to delete resources.
ProjectTag: (Optional) Value for the Project tag (default value - ‘blog’).
EnvironmentTag: (Optional) Value for the Environment tag (default value - ‘dev’).
AutomationAssumeRole: (Optional) The ARN of the role that allows Automation to perform the actions on your behalf.

AutomationAssumeRole should have at least the following permissions:

ec2:DescribeVpcs
ec2:DescribeSubnets
ec2:DescribeTags
ec2:DescribeRouteTables
ec2:DescribeNatGateways
ec2:DescribeAddresses
ec2:AllocateAddress
ec2:CreateNatGateway
ec2:CreateRoute
ec2:ReplaceRout

Start EC2 Instances

Runbook Start-EC2Instances starts all stopped EC2 instances in private subnets in the given VPC. To cover more scenarios, it could start instances based on the name pattern or the image ID. Another improvement is to save the instance ID of stopped instances in parameters, and then start only previously stopped instances. It prevents an unexpected run of stopped instances, not included to Dev or other environments.

Workflow

Describe all private subnets in the given VPC.
Search for and collect instance IDs of stopping EC2 instances in those subnets.
If any, start the instances and wait until they become running and pass all checks.

Fig. 7. Start-EC2Instances graph.

Parameters

Runbook has the following parameters:

VpcId: (Required) Id of the target VPC where to delete resources.
AutomationAssumeRole: (Optional) The ARN of the role that allows Automation to perform the actions on your behalf.

AutomationAssumeRole should have at least the following permissions:

ec2:DescribeSubnets
ec2:DescribeInstances
ec2:StartInstances

Test execution

Let’s deploy infrastructure and execute runbooks:

Deploy a Terraform project with local create_nat = true to successfully launch instances;
Set value back to create_nat = var.environment != "dev" and redeploy environment. As a result, redeploy dev environment by Terraform without NAT Gateways. EC2 instances deployed to the private subnets run web server and home page shows offline status, so it cannot reach Internet;
Execute Reduce-Environment runbook, NAT Gateways are removed, and EC2 instances are stopped;
ALB shows “503 Service Temporarily Unavailable”, because there are no running EC2 behind;
Execute Restore-Environment runbook, and check web server;
Home page shows online status, connection is restored;
Repeat execution of Reduce-Environment and Restore-Environment runbooks, if you’d like;
Destroy all infrastructure.

Fig. 8. Tests execution.

Fig. 9. How to use AWS System Manager to optimize cost of Dev environment.

Summing up

In the post, we discussed how to reduce the operational cost of non-critical environments, considered various approaches, and reviewed their pros and cons. AWS Systems Manager Automation provides a clean and auditable way to manage expensive infrastructure components without breaking IaC workflows.

Instead of rebuilding environments, we can dynamically enable and disable resources while keeping deployments predictable and reproducible. This approach allows teams to reduce costs in non-critical environments without introducing manual scripts or complex CI/CD logic. In practice, automation becomes a controlled extension of infrastructure, not a workaround around it.

Written by
Illya Reznykov
CTO EU & Asia

Svitla Systems Acquires Australia’s Kiandra IT to Expand Global Engineering Footprint and Accelerate AI-Driven Delivery

Svitla Systems Inc. — Sat, 02 May 2026 16:24:18 +0000

Svitla Systems, a global software engineering and digital solutions company, today announced the acquisition of Kiandra IT, an Australia-based software engineering firm with more than 30 years of experience delivering solutions for highly regulated and mission-critical industries.

The acquisition marks Svitla Systems’ formal entry into the Australian market and expands its global delivery capabilities across North America and Asia-Pacific. By combining Svitla’s international scale with Kiandra IT’s deep regional expertise, the company aims to accelerate delivery of AI-enabled, low-code, and enterprise software solutions for clients worldwide.

Kiandra IT employs a team of 45 professionals across Australia and is recognized for its expertise in Microsoft .NET technologies, AI accelerated software development, and its partnership with OutSystems, a leading enterprise AI low-code platform. The company was recently named OutSystems Partner of the Year for New ARR Growth.

“Kiandra IT represents a strategic fit for Svitla Systems on multiple levels,” said Nataliya Anon, CEO and founder of Svitla Systems. “The acquisition strengthens our market entry into Australia and the Asia Pacific region, deepens our AI low‑code and Microsoft engineering capabilities, and enables us to bring AI‑enabled software engineering to market through a mature delivery model. Just as importantly, it allows us to extend Kiandra’s specialist capabilities into the US while giving Kiandra the support to scale globally.”

The acquisition strengthens Svitla’s capabilities in key areas including AI low-code and code-gen application development, Microsoft-based enterprise systems, and project-based delivery for regulated industries such as healthcare, financial services, and government.

It also enables a two-way growth strategy: expanding Kiandra IT’s specialized capabilities into the U.S. market while providing its clients access to Svitla’s global delivery network and broader technology offerings.

Cameron Brookes, Co-Founder and Chief Executive Officer of Kiandra IT, said: “Kiandra IT has built its reputation on trust, quality, and delivering outcomes in complex environments. Joining Svitla Systems allows us to preserve that approach while expanding our reach beyond Australia. Our clients gain access to global scale and advanced engineering capabilities, and our people gain the opportunity to work on larger, international programs while staying true to our delivery culture.”

Kiandra IT will continue to operate under its existing leadership team, ensuring continuity for clients and employees.

Together, Svitla Systems and Kiandra IT will deliver enhanced solutions for organizations operating in highly regulated environments, combining enterprise-grade engineering, AI-enabled delivery, and global scalability.

About Svitla Systems:

Svitla Systems is a global software engineering company specialising in digital transformation, focused on six core practices: AI, Data, Cloud, Software Engineering, Application Managed Services and Team Extension. With delivery teams across North & South America, Europe, India and Australia, Svitla partners with organisations to solve complex business challenges through technology.

About Kiandra IT:

Kiandra IT is an Australian boutique software engineering firm with over 30 years of experience delivering premium digital solutions to highly regulated industries. Known for its expertise in Microsoft .NET and OutSystems, Kiandra IT provides trusted, high‑quality software engineering services for organisations where reliability and compliance are paramount.

Becoming a Data Analyst: How I Left Medicine and Switched Careers

Svitla Systems Inc. — Sat, 02 May 2026 16:09:37 +0000

At Svitla Systems, most of our team consists of seasoned senior-level professionals, but we always have room for those who are just beginning their journey in tech. Sometimes it is a long-planned decision, but often it is a career switch that no one saw coming.

Today, we are happy to share the story of Maria Bazualdo, a Junior Data Analyst at Svitla Systems, who spent years studying dentistry and training in maxillofacial surgery before making the leap into data analytics.

Leaving a profession you have spent years building, starting from scratch, and proving yourself in a completely new field is no small feat. That is exactly why we admire Maria’s courage and decision to take a risk and make it work.

Making a Life-Changing Career Switch Into Tech

As a Junior Data Analyst at Svitla Systems, I aim to share how I switched careers from medicine to tech by finding peace in data analytics.

I wish to encourage anyone who might be on the fence or feel it’s “too late” or “too difficult” to make a switch. Regardless of career or background, everyone should pursue what they truly want. Ultimately, finding joy in your work should always be the goal.

As I share my story, I’ll also detail the skills and knowledge that made the decision a no-brainer and brought genuine fulfillment to my everyday life.

Before data, I lived and breathed medicine.

My parents were doctors, so for us, medicine was part of our identity. Our dinner table conversations centered around patients and treatments, normalizing topics that others might find a bit out of the ordinary.

Studying medicine felt like an unavoidable destination. It was ingrained so deeply in me that I never even entertained the idea of anything else. Simply put: my path was laid out.

I studied dentistry and later trained in maxillofacial surgery. I should have been happy, right? Well, not quite. Everything was great on paper, but the right things never fully clicked for me, leaving me deeply unsatisfied, anxious, and with a strong sense of failure.

As time went on, the pressure only increased. Anxiety became a routine for me. At one point, I even stopped recognizing myself since the stress had affected my health so visibly. In denial, I kept telling myself that things would get better if I pushed harder, studied more, or went further.

Spoiler alert: they didn’t.

When I decided to leave medicine, I didn’t have a plan in place. I needed a job, and fast, since money was tight, and my parents felt alienated by my choice.

Stumbling Into Data Analytics and Self-Education

I didn’t necessarily set out to become a data analyst. In fact, when I first encountered data work, I didn’t even know that what I was doing had a name.

I found a company that needed help migrating files to the cloud and organizing information so business partners could understand their numbers. I jumped at the opportunity since the position wasn’t advertised as a technical role, and I didn’t think of myself as someone “in tech.” After a quick background check, I was offered the role and accepted it because it was practical, not because I was planning a career change.

During onboarding, we were asked about our backgrounds. I tried to be as transparent as possible about my limitations so I wouldn’t set the wrong expectations. The response surprised me: my soon-to-be manager didn’t question my background or my lack of formal training. He simply asked if I wanted to try. That mattered more than he probably realized.

I started helping with simple database structures, organizing information and thinking about how data should be stored, how different pieces relate to each other, and how it could be meaningful to others reading it.

I quickly realized that, at its core, the role was about making sense of information so that others don’t have to struggle through it themselves.

I also realized that this first experience in data analytics didn’t make me confident, but it definitely made me curious.

With the newfound knowledge that I loved working with data, it was inevitable to wonder: what’s next?

After years of becoming an expert in dentistry, jumping into data analytics meant accepting that I was a beginner again. I was afraid, yes, but also excited. I began exploring online courses and certifications, studying in the same way I did in medicine: seriously, consistently, and with focus.

The difference? This time, learning felt like discovery, not pressure.

I quickly learned that Python was a must for data analytics. Intellectually, I understood its power, but emotionally…it felt abstract and overwhelming. Filled with black screens and strings of code, I couldn’t see the results of my work in a way that made sense to me, which made it harder to trust myself. At the time, I even worried that this meant I wasn’t suited for data work after all.

That all changed when I started using Power BI. I felt an immediate switch go off in my brain.

For the first time, I saw data as a visual, living being. Numbers turned into shapes, trends, and patterns…and the logic I was applying was laid bare in front of me.

I stopped feeling lost, and it revealed something important about how my brain works: visual representation helped me better consume the information.

That truth connected deeply with my past. In medicine, I had always gravitated toward the “materials and methods” sections of research papers. I cared more about understanding how conclusions were reached. I wanted to see the structure behind the results. At the time, I thought that was just a personal preference. In data, I turned it into a strength.

Learning the tools wasn’t easy, but it was grounding. Each new skill reinforced the same idea: I was capable of learning beyond the path I had been given; I just needed patience.

What a Data Analyst Actually Does

Now, when I tell people I’m dabbling in data analytics, they often think my days are filled with complex formulas and advanced programming, and that I have years of technical education…all of which are not entirely true.

I started working with a very limited definition of what the role entailed. I knew there were numbers and reports involved, but that was about it.

I soon learned what a data analyst role entails. A data analyst is responsible for transforming raw data into information that people can actually use. The analyst takes large, messy datasets and organizes them in ways that enable teams to answer key questions. As a data analyst, you’re not tasked with making decisions; you’re just providing a clear, reliable foundation on which decisions can be made.

From a technical perspective, the role requires strong skills:

**Strong knowledge of SQL is a must. **Queries are how we retrieve, combine, and clean data, shaping everything that comes afterward.
Visualization tools present data in clear, accessible ways. You turn numbers into dashboards, charts, and reports that others can quickly understand.
Programming languages are inevitable. Python, for instance, is especially designed for data preparation, automation, and complex analysis.

Of course, tools alone don’t make someone a data analyst. The role also requires key soft skills:

Analytical thinking and discipline: You need to be able to spot inconsistencies, question assumptions, and understand how small changes in logic can affect results.
Eye for detail: Accuracy matters, so it’s critical to always pay attention to how a single filter or calculation error can lead to incorrect conclusions.
Patience: Data takes time, so it’s crucial that you give yourself the grace to learn the ropes and avoid rushing for quick results that could impact the final outcomes.
Communication: You often act as the bridge between technical data and non-technical stakeholders, so you need to be ready to explain what the data shows, what it doesn’t show, and why certain conclusions are reasonable.
Curiosity: The best insights usually come from asking follow-up questions and being genuinely interested in what the data is trying to reveal.

Online courses, certifications, and practical projects go a long way when it comes to building confidence. I recommend starting with courses in data analytics fundamentals, SQL, Python, and data visualization.

Another win is to constantly practice and apply what you’ve learned to real tasks, especially in this field that rewards hands-on experience over theory alone.

Learning in a Real Environment

Working at Svitla Systems is my first serious role in tech. I don’t come from a traditional programming background, and I don’t have previous analytics roles on my resume. What I do have is curiosity, discipline, and a desire to understand how data supports outcomes.

I’m free to explore, try new things, and ask questions. I’m not expected to know everything from day one, and that rewired my brain chemistry. In fact, in stark contrast, I’m encouraged to understand why things are built the way they are. And if I happen to make mistakes? They’re treated as part of the process, not as proof that I don’t belong.

Svitla has given me the chance to gradually grow into my role rather than expecting perfection from the very beginning. That means a great deal to me and makes me feel empowered.

Today, I focus my data analytics efforts in two main areas: product and insights. Together, I have a clear picture of how data supports internal teams and clients.

On the product side, I work with tables, queries, and filters that allow account managers and other teams to retrieve the information they need efficiently.
On the insights side, I work to improve reports clients use to understand performance, trends, and potential issues. I review queries, refine logic, and carefully consider how information is presented.

For me, what I enjoy most is the thought process behind it all. I find it interesting to look at highly complex datasets and try to explore them so that people consuming the data don’t feel confused.

Another thing I love about the role is the sense of completion it offers. Projects have a start and finish line. Reports are delivered. Questions are answered. That closure brings a kind of satisfaction I hadn’t experienced before, and it allows me to move on to the next task with clarity instead of lingering stress.

If I needed to sum up my work in tech in one word, it would be balance. I’m intellectually challenged without being overwhelmed. I’m growing at a sustainable pace.

My journey in data analytics is far from over, and I feel fortunate to work with experienced analysts who proactively share their knowledge. When I run into a problem, I’m encouraged to think it through, propose an approach, and discuss it openly. Sometimes I’m right. Sometimes I’m not.

Over time, this has changed how I respond to challenges. Instead of feeling immediate anxiety when I don’t know something, I’ve learned to pause and break the problem down. I’ve learned that uncertainty means there’s something new to understand.

I’ve learned to be patient with myself. I’ve learned that progress doesn’t require constant urgency.
I can take the time to think, to ask, and to refine my work.

What This Journey Taught Me (And What I Hope To Leave You With)

For a long time, I believed that success meant endurance. It meant staying on a path because I had already invested so much in it, even when it was affecting my health and happiness. Changing direction felt irresponsible, and if I’m being honest, like I was letting my family down.

Today, success looks different to me. It means working in a role that challenges my thinking without overwhelming me. It means continuing to learn and contributing in thoughtful, reliable ways. It means being able to grow professionally while still taking care of myself.

If you are considering a career change, especially from a field that feels very far from technology, I would tell you the following: You don’t need to have everything figured out. You don’t need to feel confident from the start. What matters is knowing yourself well enough to be honest about what actually works for you.

Starting over doesn’t erase what you’ve done before. In my case, my background in medicine taught me discipline, attention to detail, and respect for complexity. Those skills followed me into data and gave me confidence.

I didn’t leave medicine to escape the hard work. I left to find a new journey that made sense to me.

And in data analytics, I finally did.

Written by Maria Bazualdo — Junior Data Analyst at Svitla Systems

No Servers, Just Vibes… AWS Serverless Music Chart App

Svitla Systems Inc. — Thu, 19 Feb 2026 21:01:54 +0000

In Jonathan Swift’s Gulliver’s Travels, giant Gulliver was defeated by tiny Lilliputians. Even though they were small, due to their planning and synchronized activities, they successfully defeated the giant. This is a perfect metaphor for modern software construction. Developing an application was like being Gulliver, as a lot had to be managed in terms of complex, weighty infrastructure, e.g., server provisioning, planning for capacity, and keeping everything up and running.

But today, with serverless architecture, the playing field has changed. Like the Lilliputians, tiny, thin scripts can be incredibly powerful when linked together as a system. Rather than focusing on servers, developers can concentrate on writing the essential logic, linking components together, and delegating the heavy lifting to cloud services.

This post presents a sample in-the-wild example, a Beatport Chart Playlist App, to demonstrate how serverless tools in AWS enable the development of a product, from idea to launch, without ever requiring a server.

Real-World Use Case: Beatport Chart Playlist App

I love music. I can’t imagine my day without that. Back in our childhood, we used to trade cassettes to get new music. Discovered new songs from TV, radio charts, etc. However, nowadays, things have become much easier. We all have access to various music platforms (YouTube Music, Apple Music, Spotify, etc.), where almost any music is available. The thing I’m struggling with is getting new music into my collection. I use the Beatport chart to discover new popular electronic music, and then I search for it on YouTube. I’m too lazy, though, to do that manually. That’s tedious, so I automated that.

Let’s take a look at the architecture. This is a music chart processing automated system that web-scrapes Beatport data, sends the scraped data to a server for completion with information pulled from YouTube Music, and then automatically produces public YouTube playlists. The automated system is developed based on an event-driven AWS serverless architecture.

What Is Beatport?

Beatport is one of the world’s largest charts for DJs, producers, and electronic music geeks (looks like I'm the last one). It allows you to purchase high-ranked tracks across various genres, including house, techno, drum & bass, and trance, with daily and weekly charts that reflect trends within the global electronic music community. DJs often use these charts as a source to find out about new releases and what’s hot in clubs and festivals.

What Is Serverless Architecture?

Technically speaking, serverless is not the same as “no server.” There are still servers, but server management falls exclusively on the cloud provider. You don’t have to write complicated code: only functions and service linking.

On the other hand, conventional methods based on EC2 instances or even containers imply continuous infrastructure upkeep. Serverless eliminates that chore and allows fast prototyping and production-ready apps.

Core elements:

Event-driven. Function execution is triggered by events such as API calls, file uploads, or schedules
Auto-scaling. Automatic adjustment of capacity based on workload
Pay-as-you-go. You pay only for the actual execution time
No server management. No patching, scaling groups, or manual deployments

Typical Serverless Architecture on AWS

AWS provides a complete ecosystem for building serverless systems. Imagine it as a miniature city, with every service a specialized guild or district:

API Gateway or App Runner, the city gate. Where travelers (clients) enter, requests are inspected and routed to the right street.
AWS Lambda, the guild of artisans. AWS Lambda forms the heart of our Lilliputian workforce: a guild of agile craftsmen that show up only when invoked. These artisans are assigned specific tasks, complete them efficiently, and then disappear once their job is finished. In real-world terms, it allows developers to run code without provisioning or managing servers. Lambda scales automatically and charges only for the compute time used. This makes it perfect for event-driven workloads, where small and well-defined functions handle data processing, API requests, and automation with minimal overhead.
EventBridge, the clocktower. The scheduler that chimes on time, retries missed beats, and records every strike in the chronicles.
S3, the granary & archives. Where each day’s harvest (raw chart data) is neatly stored, versioned, and aged into cheaper cellars.
DynamoDB, the registry hall. Every track is a citizen with a unique ID. Updates enrich records, no duplicates, no chaos.
Athena, the great library. Think of Athena as the Great Library of our Lilliputian city: a vast reading hall where scholars can open any scroll (JSON file) stored in the Archives (S3) and ask questions in the universal language of SQL. With Athena, there’s no need for complex ETL pipelines or separate databases; the data remains where it was collected, yet becomes instantly searchable. You can analyze historical charts, compare weekly movements, or identify artists that consistently appear in top positions, all without moving data or maintaining additional infrastructure.
CloudWatch, the watchtowers. Sentries observing the city: metrics, logs, and alarms across every guild.

This ecosystem is elastic, cost-efficient, and self-maintaining, where each service is a specialist contributing to a seamless system.

Data Flow

In our Lilliputian city, the data journey resembles the work of many coordinated guilds:

The scraper scouts (Lambda). At dawn each week, the scouts venture to Beatport to collect the newest Top 100 tracks and return with the harvest of data.
The archivists (S3). They carefully store every harvest in well-labeled baskets - JSON files organized by date, preserving both current and historical charts.
The messengers (SQS/SNS). Once the harvest is complete, agile couriers carry the news across the city so that other guilds know it’s time to act.
The processor scouts (Lambda). Upon receiving the news from messengers. They start looking for the new track on the streaming service.
The scholars (DynamoDB). Inside the Registry Hall, scholars catalog each track as a new citizen, verifying whether it already exists and enriching it with new details from YouTube.
The herald (EventBridge). When the last record is written and the charts are ready, the Herald rings the Clocktower bell, signaling the Playlist Guild to publish a new YouTube playlist for the people.

Through this network of tiny but skilled citizens, data flows seamlessly, from discovery to publication, without a single human having to manage a server.

S3, The Granary of Chart Data

Amazon S3 serves as the primary storage layer for raw chart data in our music processing pipeline. It provides a cost-effective, reliable, and scalable solution for storing JSON snapshots of scraped music charts, making them available for both real-time processing and future analytics through services like Amazon Athena.

Why use S3 for Chart Data Storage

S3 is ideal for this use case because:

Cost-effective. Pay only for what you use, and automatically move unused data backward
Reliable. 99.999999999% (11 9's) durability ensures data won't be lost
Analytics-ready. Direct integration with Athena, QuickSight, and other AWS analytics services
Event-driven. Native S3 events trigger downstream processing automatically

Data Structure and Organization

Our chart data follows a hierarchical structure that enables efficient querying and lifecycle management:

charts-vibe-playlists/

├── beatport/

│ ├── 2025/

│ │ ├── 08/

│ │ │ ├── 30/

│ │ │ │ └── top100-120000.json

│ │ │ └── 31/

│ │ │ └── top100-121500.json

│ │ └── 09/

│ │ └── 01/

│ │ └── top100-123000.json

The BeatportScraperFunction is responsible for collecting chart data and storing it in S3. When new files arrive in our bucket, with the help of S3 events, our workflow proceeds, triggering the next artisan to perform their task.

We add a lifecycle policy that moves data to IA/Glacier in 30/60… days and removes objects in 365 days (this is an example).

Benefits for Analytics

This S3-based approach enables powerful analytics capabilities:

Historical analysis. Track the chart evolution over time
Athena queries. SQL-based analysis without data movement
Cost optimization. Automatic lifecycle policies move old data to cheaper storage classes
Data Lake integration. Easy integration with other AWS analytics services

The stored JSON files serve as both the trigger for real-time processing and a historical record for future analysis, making S3 the perfect foundation for our music chart processing pipeline.

EventBridge, The Clocktower

EventBridge is a serverless event bus that makes it easy to connect applications using data from your own apps, integrated Software-as-a-Service (SaaS) applications, and AWS services. The solution utilizes it as a powerful and reliable scheduler to periodically trigger the BeatportScraperFunction.

Why would I even use EventBridge over a simple cron job?

You could certainly use a cron job running on an EC2 instance (you should have an EC2 instance, though) or even a Lambda artisan (easier, but it requires workarounds) – EventBridge offers several advantages for cloud-native applications:

Centralized management. Your scheduled rules are all in one place, defined as infrastructure-as-code right in our template.yaml. This makes them easily traceable, versioned, and managed.

Enhanced reliability & retries. EventBridge has built-in retry policies with exponential backoff. The failure of the target Lambda function, or its throttling, will be automatically retried by EventBridge, enhancing the resilience of our scraping process. So, if the Beatport website is down when our scouts go for the weekly hunt, they would wait till the mammoth comes back.

Deep integration & logging. It integrates well with AWS services. Every invocation attempt will be logged in CloudWatch; rest assured that your scheduled tasks ran, when they ran, and whether they succeeded. The tale of the great quest won’t be forgotten.

Decoupling. The scheduler, EventBridge, has no linkage with the artisan, Lambda. We can modify the schedule without changing the Lambda code or logging into an EC2 instance to reschedule. We could even route the same schedule to multiple squads.

DynamoDB, The Registry Hall

At the heart of our system lies Amazon DynamoDB, the single source of truth for every track. A highly performant and scalable NoSQL database ensures data integrity and enables powerful features.

Why DynamoDB?

We chose DynamoDB for several key reasons that are critical to our application:

Fast, predictable lookups. The main challenge is how to identify whether we’ve previously seen a particular track quickly. Instead of scanning the entire database for every incoming track (that could be pretty costly), we create a deterministic track_id as the partition key by hashing the track’s title and artist and updated_at as a sort key. This way, we can utilize the GetItem operation provided by DynamoDB, which provides access directly to a specified item. This is the cornerstone of our deduplication strategy.
Preventing duplicate entries. Before saving any new track from the scraped chart, we first generate its track_id and check if an item with that ID already exists. If it does, we don’t create a new record; instead, we optionally update the existing one with the latest information, such as its current chart position. If it doesn’t, we create a new record. This simple “check-then-write” logic prevents the database from getting filled with redundant data.

Centralized metadata and enrichment. With the registry hall, we get a single source of truth for all metadata that describes a track. In the case where the music service identifies a matching video for a track, it doesn’t retain that information locally but instead updates that track’s record in DynamoDB. Consequently, our data becomes further enriched, hence more useful for subsequent playlist creation. We can easily extend this to store links from Spotify, Apple Music, or any other platforms. We just need to hire some more artisans who are working with new platforms.

Athena, The Great Library

Our system scrupulously collects weekly snapshots of music charts and stores them as JSON files in an S3 bucket. This creates a valuable historical dataset. But how do we ask complex questions about this data, like "Which tracks are trending upwards?" or "Which artist had the most staying power in the top 10?" Loading terabytes of JSON into a traditional database for analysis would be slow and expensive.

This is where AWS Athena comes in. It allows you to query JSON data directly from files using SQL queries. So you could easily adopt it to execute intricate queries. The possibilities are limited by your imagination only.

Other Guilds

While our main serverless stack covers the core functionality, a few supporting AWS tools complete the picture and make the system production-ready:

CloudWatch, Monitoring & Error Logging

CloudWatch provides centralized logging for all the Lambda invocations, EventBridge events, and even S3 or DynamoDB activity.

With it, we can:

Track execution metrics, including duration, memory usage, and number of invocations.
Set up anomaly detection through alarms for high error rates or throttled invocations.
Debug distributed workflows using CloudWatch Logs Insights with log filtering on request_id across functions.

By default, AWS creates a log group for the Lambda function with an easily recognizable name:

Secrets Manager, API Tokens

Since our app interacts with external APIs, such as YouTube Music or Spotify, credential management is essential.

AWS Secrets Manager securely stores API keys and automatically rotates them as needed. To create a playlist on YouTube, we need to use a YouTube Data API key and store it. For other streaming services, it's nearly the same.

Each Lambda retrieves credentials at runtime using IAM permissions, removing any need to hardcode or commit sensitive data. This setup ensures security, maintainability, and compliance, even as we scale to multiple music providers. Despite its great benefits, we could use a parameter store to minimize costs.

SAM/LocalStack, Local Development

Building serverless apps used to mean constant deployments to AWS, but with AWS SAM CLI and LocalStack, we can develop and test the entire stack locally.

Using these tools:

Lambdas can run in Docker, mimicking the AWS runtime.
S3, SQS, and DynamoDB interactions can be tested offline.
The same template .yaml is used for both local and production environments, ensuring consistency.

This allows rapid iteration while maintaining full IaC (Infrastructure-as-Code) parity with production.

Cost Analysis

One of the biggest strengths of a serverless stack is its cost efficiency.

Let’s break down the potential monthly expenses of this project under moderate usage (e.g., one scheduled scrape per day, a few thousand invocations). AWS gracefully offers our easy-to-use calculation instrument for these purposes:

Even with continuous daily operation, this app costs less than a cup of coffee per month.

Conclusion

The serverless model changes the way you think about application development. No servers to manage, no scaling headaches. Just well-orchestrated, event-driven components, working together in a team.

Here is what the above architecture provides for a project like ChartsVibe:

Speed. We need only a few days to deliver our concept
Scalability. New computing powers just appear when we need them
Cost-effective. Pay for what runs
Maintainability. Minimize DevOps overhead

Serverless works well for startups and MVPs, but it’s also suitable for long-running products that require a steady state of reliability with minimal attention.

As a quick reference, you can find the PoC codebase in my personal repo:
https://github.com/addicted2sounds/charts-vibe

It definitely requires improvements and most likely fixes, but I’m ok with it, since it does what it should do.

With AWS, you truly get “No servers, just vibes.”

Written by
Borys Oleh
Senior Full-Stack Engineer at Svitla Systems

Developers’ Procrastination Unpack: Causes and Solution Hacks (by Nataliia Romanenko)

Svitla Systems Inc. — Tue, 21 May 2024 10:43:27 +0000

Staring at my task list for this development project, I see the usual end-of-day scene — all the little tickets checked off except for that one big feature at the bottom, which I’ve been avoiding. I tell myself, “No worries, I’ll knock it out tomorrow, swear!” But underneath, I’m frustrated that I keep putting off this complex task.

If this is relatable, then you and me both. Procrastination sneaks up on the best of us now and then. For some people, it’s occasional, but for chronic procrastinators, it becomes a habit that reduces their productivity.

So, if you’re in the procrastination boat, don’t worry. There’s a way out, and we’ll get there — maybe not today, but the journey of a thousand tasks begins with a single…oh look, funny meme. Ok, let’s start.

Why Developers Procrastinate and How They Handle It

Reasons for procrastination come in many flavors. Perfectionism, lack of priorities, or poor time management are just the ones that come to my mind first. Luckily, the psychologists have already offered some methods to handle the problem. Moreover, our team members have bravely opened up about their experience dealing with procrastination. So, let’s overview each reason and its possible solutions.

Reason #1. Perfectionism

Aiming high can be helpful, but unrealistic expectations of perfection can paralyze you. The fear that the outcome won’t meet impossibly high standards can stall starting altogether. This cycle of wanting flawlessness yet dreading mistakes hampers getting things done. A 2017 study found those prone to perfectionism were also more likely to put things off.

My procrastination is also often fueled by perfectionism. Take this article, for example — I kept putting off writing it because I was worried it wouldn’t turn out as great as I envisioned. The fear of not doing justice to our team members’ experiences also held me back. To overcome this, I’ve found that setting a strict and near deadline works for me. When I commit to delivering by a specific time, there’s no room for missing the deadline.

Productivity Hacks from the Svitla Team

Kostiantyn Bilyk, Mobile Practice Lead, Team Lead and Lead iOS/macOS Engineer: “In my case, perfectionism is a part of my personality. I always aim to do the task perfectly and in the best possible way. For sure, sometimes it causes procrastination. I start thinking that if I can’t do the task perfectly, I’d better not do it all. On the one hand, it’s a nasty treat. Yet, if we think about it from another perspective, there are some positive sides. When faced with complex tasks, I tend to put them on hold and focus on simpler ones. This way, I stay productive. I also find the 80/20 rule, known as the Pareto principle, helpful in my work. If a task is complex, I tackle its core first and then create a follow-up ticket for the remaining aspects. Surprisingly, more often than not, these follow-up tasks become unnecessary over time. Now, let’s move on to the real-life example. In 2014, I faced a challenge to develop a telemedicine video chat platform, a concept ahead of its time, given the technological constraints of 2014–2015. I treated this task with my perfectionist attitude. So, I took one week to do some research. And another week to implement my idea. By the end of the first week, I found the ideas I liked and presented them to the project manager. After their approval, I started implementing them. By mid-next week, I had finished the platform’s basic functionality — to start and end a call. After that, I presented it to the customer, who was quite satisfied. Soon after, I found out, that the other development team had been working on the task I’d done in 2 weeks, for 3–3.5 months.”

Nadiia Chumak, Senior Go Developer: “I began to notice my tendency to procrastinate since my school days, right from childhood. I understand there must always be a perfect version when doing homework or writing code. Now, I’ve gotten a bit better with that. So, when I see myself spending time on things already working fine, I try to stop and push the code. It helps me get things done faster.”

Reason #2. Absence of Instant Results

We’re used to getting things fast now. But big projects take time before you see results. We get tempted to procrastinate when we don’t get that quick feeling of finishing something. We immediately switch to more manageable things that make us feel good, like checking our phones. To fix this, set small milestones to finish the big project. Breaking it into smaller steps makes each piece feel less complicated, and finishing each step motivates you to continue.

Productivity Hacks from the Svitla Team

Andriy Arysmyatov, Senior .Net Developer: “Facing a task that initially seems manageable, but as you delve into it, the results remain elusive. It gets postponed, especially when it’s not a top priority. The cycle of delay continues until you find a moment of free time. You keep postponing, pushing it aside, and eventually, it slips from your mind. Instead of actively combating this tendency, consider breaking down the task into smaller subtasks. It’s essential not to harbor illusions about the ease of completion. Challenges arise, and acknowledging them is the first step. Planning subtasks helps tremendously. Allocate dedicated time rather than tossing the task aside. Setting specific plans and sticking to them is crucial, ensuring that the task doesn’t get lost in daily activities.”

Reason #3. Unclear Understanding of the Final Result

Fuzzy goals stall momentum. Laser focus fuels drive. Define projects concretely — what must happen and when. Broad assignments invite procrastination. Specific, scheduled tasks kindle motivation. Chop large efforts into bite-sized pieces. Rather than “finish report” in the foggy future, spot “Draft intro section Tuesday, 10 AM-12 PM” in the calendar now. Granular goals crystallize the path from concept to completion. They transform vague aspirations into timed action steps. Clarity breeds productivity. Hone objectives. Then, accomplish each scheduled session one at a time.

Productivity Hacks from the Svitla Team

Pavlo Zubenko, JS Lead: “To tackle procrastination, I’ve got a few tricks. If it’s a big task, I take a break — maybe grab a slice of pie or brew some coffee. It’s a mini reward to keep things rolling. During a hefty project, I take short breaks, get some coffee, or snack a bit to stay sharp. I ditch my phone and stick to using a notebook to dodge distractions. Magnetic doodle pads and a whiteboard notepad help me sketch out ideas and smoothly switch between different parts of the task. Taking a step back and looking at the big picture helps gain some perspective. As I near the end of a task, I follow a simple rule: no starting anything new for precisely 20 minutes. It’s a way to wrap things up”.

Yurii Stasuik, Senior Front-End Developer: “When you’re working on a product but don’t get the expected result, it messes with your motivation. Being a front-end developer is not just about coding. You also have to think about the user interface and get feedback, which is not always there. The fix? Well, maybe switching up the project is the easiest option. For me, also doing A/B tests work. This way, I get the necessary feedback, keep myself motivated, and improve the final product.”

Reason #4. Lack of Self-Discipline

Pushing through routine or tiring tasks takes self-control. When self-discipline is lacking, procrastination often happens instead. Building good daily habits boosts focus and avoids distractions. This means limiting social media, turning off notifications, and potentially using website blockers. Start each day by working on priorities, not checking emails. Taking regular breaks to rest fully recharges energy and concentration. The key is structuring consistent conditions focused on concentration until self-discipline becomes a habit.

Productivity Hacks from the Svitla Team

Yurii Stasuik, Senior Front-End Developer: “Balancing work, home stuff, family, and other tasks can be challenging, especially when working from home. I’ve established clear boundaries to combat procrastination due to a lack of self-discipline. I avoid working in bed, dedicating 3–4 hours to focused work, followed by a break. This approach helps me maintain productivity and balance within my responsibilities.”

Reason #5. Task is Too Difficult

We often put off work that seems too hard, or we don’t feel skilled enough to complete it. Tasks with unclear instructions or too many complex parts can make us freeze up and avoid starting. To fix this, break big intimidating goals into small bite-sized tasks. This outlines a path to slow but steady progress.

Look at what skills you need to develop and leave enough time for improvement. As you build momentum step-by-step, what felt impossible starts to seem within reach. Check in often to adjust expectations before frustration creeps in. Bit by bit, seeing progress dispels the notion that the level of difficulty can’t be handled. Structuring overwhelming tasks into achievable pieces avoids putting them off.

Productivity Hacks from the Svitla Team

Kristina Shatts, Senior Node.js Developer: “Currently, I rarely procrastinate due to overly complex tasks. There was a time, especially when I was a beginner, when I was afraid to work on challenging tasks, making me nervous. To tackle this, I tried to break down the tasks into smaller sub-tasks to understand them better and deal with each part separately. Seeking assistance was also important. Not just from colleagues, but also by reaching out to acquaintances, friends, and individuals knowledgeable in the field. Asking questions was something I learned not to fear. There is nothing wrong in not knowing something”.

Kostiantyn Bilyk, Mobile Practice Lead, Team Lead and Lead iOS/macOS Engineer: “In 2019, I started to lead work on the new project. Its idea seemed almost unrealistic — creating a next-gen file manager with meta-connections between files. The project involved mobile/desktop platform support, Apple sign-in, and working without servers. It seemed daunting from an engineering perspective, but the project looked interesting, long-term, and promised team expansion.

Initially, I was overwhelmed. To overcome this state, I adopted the eat-the-frog technique. Each day, I set one challenging but doable task — the “frog” — to tackle. This simple strategy helped maintain a steady workflow.

Another useful thing for me was incremental growth. Starting with basic functions, I gradually expanded the functionality. As the project grew, we added AI-driven features and integrated chat support. I needed a holistic vision, breaking the project down like a pyramid and building it up iteratively.”

Reason #6. Boring Task

Tedious tasks need a fresh take. What could make this better? Look for hidden upsides and chances to grow. Get creative — can you mix fun with work? Make dull duties more lively. Add enjoyment into tasks. Reframing can shift your mindset, fueling motivation. Find enriching bits in each chore. Tweaking your view lifts your mood, lightening the workload. Soon, getting things done flows faster, building your willpower.

Productivity Hacks from the Svitla Team

Kristina Shatts, Senior Node.js Developer: “If there’s something new and exciting, I’m all in, but the not-so-interesting stuff, like messing with database scripts, tends to make me procrastinate.

To tackle this, I’ve got this Pomodoro technique going on — work hard for 30 minutes, then take a 10-minute break to do something fun as a reward. I’m a pretty sociable person. So collaborating with colleagues on tedious tasks also works for me.

Finally, if I’m doing routine stuff the whole day, I take more time for breaks afterward — solving puzzles, enjoying some tea, or just chatting with someone to shake things up.”

Reason #7. “I’ll Do It Later”

The mindset of “I’ll do it later” leads to endless delays. Unpleasant tasks often get pushed to some vague future time to avoid doing them now. But later rarely comes, creating a cycle of stress and rushed last-minute work. To break this, firmly decide what will get done today, not someday. Tying daily goals to values, not just distant obligations, makes completing them more meaningful.

Productivity Hacks from the Svitla Team

Kristina Shatts, Senior Node.js Developer: “If there’s no deadline, I tend to postpone things. But funny enough, I do things without a hitch when there’s a deadline.

With bigger tasks, I convince myself I can knock them out quickly and decide to do them later. Yet looking back, I understand that it’s not the best approach. At first sight, the task may seem less time-consuming than it is. In this case, the best way is to look closer at the task and estimate its priority from the beginning.

Reason #8. Lack of Energy

When energy is low, big tasks can feel impossible. But waiting until we feel perfectly energized to start important work can cause endless delays. Regular routines for sleep, diet, exercise, and renewal activities help manage daily energy. To combat a lack of motivation, know when you tend to focus best and tackle critical projects.

For draining items at low-energy times, briefly work in 20–30 minute intense bursts, then wholly rest. If you often feel exhausted, carefully spend high-value energy on priorities while conserving strength to recharge. With enough built-in self-care, you store vitality instead of endlessly waiting to feel energized first before starting.

Productivity Hacks from the Svitla Team

Andriy Arysmyatov, Senior .Net Developer: “Whenever my motivation takes a hit, I find it crucial to take a break. I step away and read something, but I always try to maintain focus. Getting a good night’s sleep or walking helps recharge my energy. When I return, I make it a point to create a clear plan.

Purposeful procrastination involves taking a break to rest and returning with a fresh perspective. Switching to another task and returning later often leads to a quick resolution within 1–2–3 hours.

I also contact my team for help when needed. Sharing my feelings and challenges with colleagues can make a significant difference. Having a trustworthy manager’s support is also crucial in such situations.”

Yurii Stasuik, Senior Front-End Developer: “Procrastination hits me quite often, especially after an initial burst of energy. I’ve figured out that taking a moment to recharge and switch things up helps. I turn to things like snowboarding, hiking the mountains, or going on a trip for an energy boost.

Procrastination is less common at work, but it tends to sneak in when I’m dealing with something new or uncertain. I’ve got some tricks up my sleeve, but handling procrastination is an ongoing struggle. It’s all about finding that balance and grabbing energy from different things I enjoy doing.”

Reason #9. Mental Issues

Putting things off can sometimes come from underlying mental health issues. Symptoms like depression, anxiety, ADHD, or chronic stress drain motivation to start and stick to boring work. Without treating the root causes, just trying to push through backfires.

For example, emotional paralysis may benefit from scheduling required activities each day. Perfectionism from anxiety might require reframing unhelpful thought patterns. Those with ADHD can use external accountability tools to manage distractibility. When mental troubles obstruct average productivity, the most critical step is acknowledging the needs.

Productivity Hacks from the Svitla Team

Pavlo Zubenko, JS Lead: “As a kid, I was diagnosed with dyslexia, especially dysgraphia. It didn’t stop me from becoming a developer. Yet I have some challenges to overcome. Reading tech docs can be a bit of a struggle. The sentences don’t flow smoothly, and there’s always something interesting popping up to distract me. But, you know, you get used to it, and I’ve figured out some tricks to handle it. For instance, I found out that a power nap is really helpful — it’s a short, 15–30 minutes of sleep, after which I feel recharged and more concentrated”.

Conclusion

Understanding why we procrastinate is the first step to getting things done in the quest for productivity. Whether it’s the desire for perfection or unclear priorities, each obstacle has a solution. With awareness and solutions, we can turn procrastination into a helper, making overcoming challenges easier.

In the end, getting through a thousand tasks might take time. Still, with simple strategies and self-understanding, we’re on a journey where procrastination is a temporary sidekick, not a constant companion.

Leveraging AI in Education: Exploring Big Data and Related Applications

Svitla Systems Inc. — Fri, 22 Sep 2023 11:48:48 +0000

In recent years, technology has become increasingly vital in shaping the landscape of education. From traditional classrooms to online learning platforms, the integration of technology has revolutionized how knowledge is imparted and acquired. One of the significant advancements in this realm is the application of Artificial Intelligence (AI) and Big Data.

Artificial Intelligence has opened up new horizons in education. With machine learning algorithms and data analysis, AI-powered tools personalize education and enable tailored instruction, immediate feedback, and adaptive assessments, optimizing the learning experience for each student.

Additionally, AI-driven virtual tutors and chatbots are gaining prominence, offering round-the-clock support to learners. These intelligent systems can provide instant answers to students’ queries, offer explanations, and guide students through complex concepts, empowering them to explore and deepen their understanding.

Big Data, on the other hand, plays a crucial role in education by collecting and analyzing data like student performance metrics, feedback from assessments, and even social interactions within learning platforms, generated within the educational ecosystem.

Adapting Artificial Intelligence and Big Data to education holds immense potential as technology advances. It empowers educators to deliver personalized, adaptive, and engaging learning experiences, while providing valuable insights that shape pedagogical practices.

Thus, the growing importance of technology in education, specifically through the utilization of Artificial Intelligence and Big Data, has the potential to transform the learning landscape, optimize educational outcomes, and nurture a generation of digitally fluent and adaptable learners.

Understanding Big Data in Education

In education, the concept of Big Data refers to large volumes of structured and unstructured data generated within the educational ecosystem, including student information, academic records, assessments, social interactions, and more. Big Data in education encompasses the collection, analysis, and interpretation of vast amounts of data to derive meaningful insights and patterns that inform educational practices and policies. It involves leveraging technology and advanced analytics techniques to make sense of the diverse data points generated across various educational systems and platforms.

Big Data in education is characterized by its immense volume, variety, velocity, and veracity. It encompasses diverse types of structured and unstructured data from various sources, including learning management systems and online platforms. The velocity of data generation requires real-time processing for timely interventions and personalized support. Ensuring data veracity is crucial to maintain accuracy and integrity when working with educational data.

Benefits of Big Data in Education

Personalized learning: Analyzing Big Data enables tailored instruction and personalized learning experiences based on individual student needs, preferences, and progress. A study conducted by the Bill & Melinda Gates Foundation demonstrated that personalized learning has the potential to boost student achievement by 7% compared to traditional methods, highlighting its effectiveness.
Early intervention and support: Big Data analytics identify students at risk and enable timely interventions to provide necessary support and improve student outcomes. Numerous states, districts, and schools have successfully acquired the necessary data to monitor student progress and achievement, and some have effectively utilized it. A notable example is Fresno, where the implementation of a new data system revealed a lack of awareness among students regarding their college options. In response, the district took proactive measures by creating personalized college information packets for each senior who met the state’s college requirements. The outcome was remarkable, with a 50 percent surge in the number of students applying to California public universities.
Evidence-based decision-making: Educational institutions can make data-driven decisions by analyzing educational data, evaluating interventions, and enhancing teaching methodologies and resource allocation.
Continuous improvement: Big Data analysis provides insights to refine teaching strategies, improve curriculum effectiveness, and enhance the overall quality of education.

In conclusion, Big Data in education represents a transformative opportunity to harness the power of data analytics and gain valuable insights that shape educational practices and policies. By collecting, analyzing, and leveraging Big Data, educational institutions can unlock new possibilities for personalized learning, early intervention, evidence-based decision-making, and continuous improvement to ensure better educational outcomes for all learners.

The Svitla team has extensive experience integrating Big Data into a wide variety of business processes, including education, and using advanced analytical tools. As a result, we are skilled at developing powerful and tailored software solutions tailored for educational institutions.

The Role of AI in Analyzing Big Data

Artificial Intelligence (AI) has become a game-changer in many industries, including education. Combined with data within the educational sphere, AI algorithms can unlock powerful insights and revolutionize the way educators understand and utilize information. Here, we explore the role of AI in analyzing Big Data in the educational sector.

In education, AI spans personalized learning platforms, intelligent tutoring systems, chatbots, and data analytics tools. AI algorithms process and analyze Big Data to uncover patterns, predict outcomes, and provide valuable insights for educators.

With the increasing digitization of educational processes, enormous amounts of data are generated, including student performance metrics, learning activities, assessments, and more. AI algorithms can effectively process and analyze this data in the following ways:

Personalization: AI algorithms can analyze individual student data to identify learning preferences, strengths, and areas for improvement. By considering various factors such as learning style, pace, and content preferences, AI can personalize instruction, recommend tailored resources, and provide adaptive learning experiences that optimize student engagement and success.
Predictive analytics: AI algorithms can analyze historical educational data to make predictions about student outcomes. By identifying patterns and correlations in the data, AI can forecast potential risks, such as students at risk of dropping out or struggling with specific subjects. This enables educators to intervene early and provide targeted support, improving student retention and academic performance.
Intelligent tutoring systems: AI-powered tutoring systems leverage natural language processing and machine learning to provide interactive and personalized learning experiences. These systems can assess student knowledge, identify misconceptions, and offer targeted feedback and guidance. AI algorithms continuously adapt to student responses, refining the tutoring experience and optimizing learning outcomes.
Chatbots and virtual assistants: AI-driven chatbots and virtual assistants can enhance student support services. These intelligent systems can answer frequently asked questions, provide guidance on course selection, offer study tips, and assist with administrative tasks. By leveraging natural language processing capabilities, AI chatbots can engage in human-like conversations, improving accessibility and responsiveness for students.

Benefits of AI in Analyzing Big Data

Enhanced decision-making: AI algorithms analyze complex educational data sets and provide valuable insights to educators and administrators. These insights inform evidence-based decision-making, enabling institutions to improve teaching methodologies, curriculum design, resource allocation, and student support services.

Personalized learning: AI-powered platforms can adapt instruction based on individual student needs, preferences, and learning styles. By analyzing Big Data, AI algorithms deliver customized content, adapt the learning pace, and provide real-time feedback, promoting personalized and self-directed learning experiences.

Our client offers quality education through a unique teaching methodology based on the emotional, intellectual, and social development of each student. They provide the necessary tools and experiences needed to face the challenges of today’s changing, multicultural, and competitive society. Our client has an international presence, with offices in the United States, Mexico, Spain, and Chile, serving more than 1000 schools.

Efficient data analysis: AI algorithms can process vast amounts of data at a speed and scale beyond human capabilities. By automating data analysis, AI saves time for educators, allowing them to focus on interpreting the insights and implementing appropriate interventions effectively.

Continuous improvement: Through iterative data analysis, AI algorithms facilitate continuous improvement in educational practices. By identifying trends, evaluating interventions, and measuring student progress, AI supports educators in refining their instructional strategies, enhancing curriculum design, and maximizing learning outcomes.

In a nutshell, the role of AI in analyzing Big Data within the educational sector is transformative. AI algorithms offer:

Advanced data processing capabilities
Personalization of educational services
Predictive analytics
Intelligent tutoring systems
Data-driven decision making
Personalized support
Optimized learning experience for students

Enhancing Personalized Learning through AI

Artificial Intelligence (AI) plays a crucial role in advancing personalized learning, tailoring educational experiences, and optimizing learning outcomes. Here, we delve into how AI enhances personalized learning and explore the benefits and challenges of implementing this approach with AI and Big Data.

Personalization based on individual student needs: AI algorithms analyze vast amounts of educational data, including student performance, learning styles, preferences, and progress, to create tailored learning experiences. By considering these factors, AI can customize the content, delivery methods, and assessments to match each student’s requirements.

Our client’s product enables scalable learning by powering engaging online spaces for professional communities to share knowledge. Involved communities enable multi-stakeholder groups to quickly assemble fully tagged, searchable knowledge bases to leverage machine-aided and peer-to-peer content curation. Through a design thinking methodology, peer-to-peer learning environments, and deep analytics, the Product is revolutionizing how knowledge is shared and scaled. The platform helps organizations break down knowledge silos and create a culture of scalable learning. The Service’s Communities empower learners to become participants and producers of knowledge rather than merely consumers of content.

Benefits of AI in Personalized Learning:

Customized content: AI algorithms can curate and deliver content that aligns with the specific needs and interests of individual students. By analyzing data on student performance, AI can recommend appropriate resources, adapt learning materials, and provide targeted enrichment or remediation activities.

For instance, Svitla helped our client to create captivating online courses. The first course is a comprehensive YouTube Influencer course — the only one in the world where the world’s top influencers teach how to create viral original content, develop a massive following online, and monetize your audience.

Adaptive instruction: AI-powered systems can adapt instruction in real-time, providing immediate feedback and guidance. By monitoring student responses, AI algorithms can dynamically adjust the difficulty level, pacing, and sequencing of learning activities, ensuring optimal challenge and engagement.

Our client’s collaborative learning platform empowers organizations to design and deliver experiential learning that accelerates business performance. Global corporations, executive education providers, and training firms rely on this platform to develop high-value capabilities through coaching, mentorship, and group collaboration. Their proven approach to learning connects diverse groups of learners, mentors, and leaders in a high-impact online environment, unlocking organizations’ collective knowledge.

Intelligent feedback: AI can provide timely and constructive feedback to students, highlighting areas of improvement and suggesting personalized strategies for mastery. This continuous feedback loop fosters self-reflection and empowers students to take ownership of their learning journey.

Data-driven insights: AI algorithms analyze educational data to generate actionable insights for educators. These insights inform instructional decision-making, allowing teachers to identify learning gaps, adapt teaching strategies, and provide targeted interventions that support each student’s progress.

Unlock the transformative potential of artificial intelligence (AI) in the educational sphere with Svitla’s in-depth expertise. Svitla’s AI solutions offer boundless opportunities to revolutionize teaching, learning, and administrative processes, fostering a more engaging, personalized, and efficient educational experience.

Challenges of Implementing Personalized Learning with AI and Big Data:

Data privacy and security: It is crucial to ensure the ethical use of student data and to implement robust data protection measures.
Access and equity: It’s essential to bridge the gap between technologies and learners. For example, students from underserved communities must have equal access to AI-powered personal learning experiences.
Teacher training and support: Educators need proper training and support to effectively interpret AI-generated insights, personalize instruction, and strike a balance between technology and human interaction in the classroom.
Ethical considerations: It is vital to address biases, ensure algorithmic transparency, and maintain algorithmic accountability to build trust and maintain ethical standards.

Bottom line
The benefits of AI in personalized learning include customized content, adaptive instruction, intelligent feedback, and data-driven insights. However, challenges related to data privacy, access and equity, teacher training, and ethical considerations must be addressed to ensure the responsible and equitable implementation of personalized learning with AI. With careful planning and support, AI-powered personalized learning has the potential to revolutionize education, empowering each student to reach their full potential.

AI-Powered Educational Assistants and Virtual Teachers

Virtual assistants and chatbots have become valuable assets in the educational sphere. They employ natural language processing and machine learning algorithms to comprehend user queries, provide assistance, and facilitate seamless learning experiences.

AI-powered educational assistants and virtual teachers offer numerous benefits. First, they provide personalized support, tailoring their responses and resources to meet the specific needs of individual students. This personalized approach enhances the learning experience, catering to each student’s unique requirements.

Second, virtual assistants are available round-the-clock, ensuring 24/7 access to help and resources. Students can seek guidance at their convenience, fostering independence and self-directed learning.

Third, AI-powered educational assistants offer scalability and consistency since they handle multiple queries simultaneously, ensuring seamless support for a large number of students. Additionally, they provide consistent responses and feedback, eliminating discrepancies in instructional delivery and promoting equitable access to information.

Finally, the immediate feedback and support offered by virtual teachers and assistants are invaluable. Students receive instant feedback on their work, allowing them to identify errors and areas for improvement in real-time. This feedback loop boosts learning efficiency and facilitates student progress.

Limitations of AI-powered Solutions
It is important to recognize the limitations of AI-powered educational assistants and virtual teachers. They may struggle to replicate the human element found in traditional teacher-student interactions. The personal touch, empathy, and emotional connection that come with face-to-face interactions may be lacking in AI systems. Additionally, AI algorithms may face challenges in addressing complex problems that require critical thinking, creativity, and subjective judgment.

Data dependence and privacy concerns also arise when implementing AI-powered systems. These assistants rely on extensive student data to personalize their responses. Ensuring data privacy and security, as well as addressing ethical considerations surrounding the use of student data, are essential when integrating AI into education.

Lastly, technical limitations may hinder the effectiveness of AI-powered assistants. The accuracy of these systems in understanding and responding to complex or contextually nuanced queries can be affected. This may result in incomplete or incorrect information being provided to students.

Bottom line
AI-powered educational assistants and virtual teachers have the potential to greatly enhance education delivery. Their personalized support, 24/7 availability, scalability, and immediate feedback bring significant advantages to the learning experience. However, limitations such as the lack of human interaction, challenges in complex problem-solving, data privacy concerns, and technical constraints should be taken into consideration. Striking the right balance between leveraging the benefits of AI and maintaining the essential role of human educators is crucial for a comprehensive and successful educational experience.

Final thoughts

AI-powered educational assistants, virtual teachers, and personalized learning systems offer numerous advantages, including personalized support, scalability, 24/7 availability, immediate feedback, and data-driven insights. These technologies have the power to enhance the learning experience, bridge educational gaps, and empower both students and educators.

However, it is crucial to approach the integration of AI in education thoughtfully and responsibly. Ethical considerations, data privacy, inclusivity, and the role of human educators must be carefully addressed. Collaboration between educators, researchers, policymakers, and technology developers is key to navigating the challenges and maximizing the benefits of AI in education.

The future of education lies in striking the right balance between leveraging the power of AI technology and maintaining the essential elements of human interaction, empathy, and creativity. AI should be viewed as a powerful tool that enhances and supports the work of educators, rather than replacing them. The role of educators will evolve to include utilizing AI-generated insights, personalizing instruction, and fostering critical thinking and socio-emotional skills.

As we move forward, it is important to prioritize investments in teacher training, research, and equitable access to AI-powered educational tools. That’s how we can create a future where personalized, inclusive, and lifelong learning becomes the norm, empowering individuals to thrive in a rapidly changing world.

Svitla Systems helps clients select and implement AI and Big Data solutions that perfectly align with their unique business requirements. With in-depth knowledge and expertise of AI and Big Data technologies, our engineers seamlessly integrate these solutions for our customers.

Svitla representatives can help you explore how we can collaborate strategically to enhance your adoption of AI and Big Data solutions. Contact us and we’ll get back to you shortly.

Why Every Digital Product Needs a UX Writer — Mariia Vynogradova, Senior UX Writer at Svitla Systems

Svitla Systems Inc. — Thu, 29 Jun 2023 12:59:27 +0000

User experience (UX) is a crucial element in the design of digital products, and it should never be underestimated. A successful digital product that effectively engages its audience stands out from those that fail to provide a positive user experience. Overall, the importance of UX for digital products boils down to one thing: creating a great user experience as the foundation for creating a successful and sustainable product.

For years, Svitla Systems has been dedicating a significant amount of resources, talent, and time to perfect its UX services and delivery, engaging, harnessing, and learning from top experts in the field to ensure every project is approached with a strong UX initiative from the get-go.

For today’s piece, we held a candid interview with one of our greatest: Mariia Vynogradova, our UX Writer who’s changing the world of our clients’ digital product development.

Let’s get going!

Let’s hit the basics of who UX Writers are

The phrase ‘Who is a UX writer’ is typically a starting point for anyone seeking to understand this unique role within the field of user experience. I would best explain UX Writing as the process of creating clear, concise, and user-focused content for digital products and interfaces. The primary goal of UX writing is to enhance the user experience by providing helpful, relevant, and engaging copy that guides users through the product.

The user experience is pivotal when designing digital products because it ensures digital products are effective and efficient so users get a positive and satisfying experience. Google is suggesting you an article based on your interests? Netflix is auto-playing the next episode so you don’t need to touch the button? Spotify is wrapping up your playlists at the end of the year so you can share them on Instagram? It’s all about the user experience. By focusing on UX, you can identify and eliminate pain points in your product, streamline processes, increase customer satisfaction and loyalty, improve customer retention, and even attract new customers through word-of-mouth referrals. As they say, UX design can make or break your product and relationship with customers.

“What is UX writing and how it differs from copywriting?” is a question I wish all recruiters would research before starting to look for a UX Writer 😂. Essentially UX writing focuses on the user’s needs and goals rather than marketing or promotional messaging, simple as that. It involves writing microcopies, such as buttons, labels, tooltips, error messages, and other interface elements that help users navigate and interact with the product. UX writers work closely with designers, developers, and product managers to ensure that the copy and its tone align with the product’s strategy.

As a part of the UX team a UX Writer puts together words and creates copy for interfaces. By saying “words” I mean literally all the words that comprise a digital product. From buttons, headings, navigation hints, and more — every single word that a user sees is written by the UX Writer.

Sounds easy? Try explaining what the concept of “Cookies Allowlist” is in one sentence so it’s clear for your colleagues, your children, and your granny ;)

It took over a decade to realize that designers aren’t ready to work with words, and copywriters have a very vague concept of UX and development. But we get better at this :)

Some of the key responsibilities of a UX writer include:

Creating and editing copy: Write clear and concise copy that is easy to understand and use; the UX Writer should also edit copy to ensure that it meets the needs of the product or service.
Conducting user research: Conduct user research (with UX researchers if you are lucky enough to have them!) to gain insight into user needs and behaviors, and to ensure that content is aligned with user expectations.
Collaborating with designers, developers, product managers, support, and researchers: Ensure content is cohesive and integrated effectively into the product or service.
Creating style guides to maintain the voice and tone of a product: Create style guides and standards for a consistent voice and tone across all user-facing content.
Testing and iterating: Test, iterate, and audit content to ensure that it is clear, accessible to all users, and meets the needs of the user.

When thinking about extraordinary skills or responsibilities, I foresee:

Having empathy and a user-centric way of thinking.
Possessing knowledge of wireframing/prototyping tools.
Conducting design thinking workshops.
Being data-driven.
Proficiency in design collaboration tools (Sketch or Figma) and text collaboration tools (Notion, Confluence, etc.).
Knowing the tools to conduct UX research and analyze data is a great plus.
Sometimes localization tools are required.

When Does Your Project Need a UX Writer?

Having a UX writer is beneficial for every digital product and it brings tons of value when:

The team needs to make anything from scratch.
When an existing product or service is being rebuilt.
When a product or service is released in many markets.
Whenever there is a need to write, revise, or enhance a product’s copy.

Here are the main areas that showcase the significant contribution UX writers bring to their teams and products.

Crafting clear and consistent messaging: by using the language that resonates with the target audience, UX writers ensure users can easily understand what the digital product is about and what they can expect from it.
Developing content that facilitates user flow: UX writers help map out the user journey, creating content that guides users through the digital product with ease by ensuring information is logical and intuitive.
Enhancing accessibility: They keep accessibility in mind by using plain language, providing alternative text for images, and ensuring content is easy to navigate for users with disabilities.
Testing and refining content: UX writers work closely with designers and developers to test digital product content and refine it over time by analyzing user behavior and feedback.
Building brand awareness: By helping create content that reflects the brand’s voice and values, they ensure that messaging aligns with the brand’s identity.

Negative Consequences of Delegating UX Work To Non-UX Writers

I’ve found that delegating UX work to non-UX writers legitimately causes headaches and bad results overall. Why? Firstly, non-UX writers lack the specialized skills and knowledge required for creating effective user experiences. As UX Writers, it’s ingrained in us to have a deeper understanding of user psychology, design principles, and technical considerations, and without this expertise, non-UX writers may create content that is confusing, ineffective, or even counterproductive, leading to a poor user experience.

Secondly, I’ve seen firsthand how delegating UX work to non-UX writers leads to inconsistencies and errors in messaging, tone, and style; not to mention, they typically don’t fully understand the importance of accessibility and inclusivity in user experience design.

Why does UX work keep being delegated to non-UX writers? Well, unfortunately, I’ve come across the fact that, for organizations, it’s tricky to recognize the importance of UX. Also, organizations typically prioritize cost-cutting measures, leading them to delegate UX work to existing staff members who lack the necessary expertise.

Real-life Examples of Great UX Writing

The first product that comes to my mind is Duolingo because I’m trying to improve my German and Japanese now. The app is packed with great gamification and UX Writing, exemplifying a consistent, well-executed design strategy that is intuitive and user-friendly for anyone to use.

Next up is Canva. Upon opening the app, it asks users to learn about how they plan to use the platform and then leads them to a tutorial focused on what they actually want to do rather than taking them through everything. This is particularly great because it underlines the unique experience “especially for you.”

Also, I really appreciate the fact that the subscription stage is not annoying 😀

Next, I couldn’t leave Netflix out as it’s a perfect example of an intuitive UX. Users don’t need further guidance because the product is extremely simple. From asking for a few preferences to suggest related recommendations to its simple and funny UX wording, it’s a great UX experience all around.

Slack is also a great example of an engaging experience. From the get-go, it encourages new users to learn the product by actually using it. After signing up and asking 3 questions about your team, the app immediately starts showing users around, starting with the very first project channel.

Another cool UX aspect of Slack is its use of contextual onboarding to bring information wherever it is needed. For example, when a user wants to integrate an app with their Slack workplace, a tiny three-step user onboarding flow appears.

Last but not least, Notion. It has a three-step sign-in session, helping users personalize the workspace to match their specific needs and showing straight away that Notion is a powerful and multifunctional solution.

Instead of dropping new users into a complex dashboard, Notion introduces them to the core features with a lightweight and interactive tutorial. I especially love the brief, hands-on onboarding that shows the product’s key features, functionalities, and commands.

Another burning question that many ask is whether or not I feel threatened by AI tools such as ChatGPT. My answer is “Not now”. First of all, most of my work takes place before I even start typing. As of today, there is no AI to interview users, conduct UX research, go to meetings with product teams, or have discussions with designers and developers. Plus, AI is way too polite and in my area sometimes you need to fight to prove your opinion ;) By the way, AI gave me some pretty good ideas when I was preparing for this interview 😌

UX Writer’s top recommended reading list should include a few books:

Strategic Writing for UX by Torrey Podmajersky
Microcopy: The Complete Guide by Kinneret Yifrah
Content design by Sarah Winters
Writing Is Designing by Michael J. Metts & Andy Welfle
100 Things Every Designer Needs to Know About People by Susan Weinschenk
Essentialism: The Disciplined Pursuit of Less by Greg McKeown
Hooked: How to Build Habit-Forming Products by Nir Eyal
The Design of Everyday Things by Don Norman

As well as podcasts, here I would recommend listening to Writers in Tech; courses UX-writing Hub and for general information Google UX design.

Overall you need to read, listen, highlight, and write ideas down. See something interesting? Make a note of it. Surround yourself with words and look around to spot good and bad UX copy everywhere, you’d be surprised! It’s my biggest inspiration.

How to find a professional UX Writer for your project?

Should you be looking for a boost in your digital product’s user experience, we at Svitla Systems, have a team of UX writers who are adept at simplifying complex ideas, translating technical jargon into plain language, and creating a unified voice across all user touchpoints.

Our UX writers understand the importance of user-centric content. They are always focused on refining their work, ensuring it effectively meets user needs and enhances the overall user experience. In the realm of digital product development, the significance of UX writing can’t be overstated. It’s about crafting clear, concise, and user-focused content that helps users achieve their goals effortlessly. Whether you’re starting a new project or looking to improve an existing one, we’re here and happy to assist with your UX writing needs.

Let us demonstrate how effective UX writing can elevate your user experience to the next level!

Best Test Management Tools In 2023. Comprehensive Review. (By Sergii Vashchenko, QA Practice Lead at Svitla Systems).

Svitla Systems Inc. — Thu, 20 Apr 2023 12:57:34 +0000

Hi there! My name is Sergii Vashchenko, and I am the QA Practice Lead at Svitla Systems. I enjoy trying new tools, techniques, and theories on the projects I am a part of and would like to share my experience with the rest of our QA community.

In the last ten years, I've used many tools for test management. I already have a few favorite ones, but with 2023 making waves, I became interested in finding the right tool to suit current and modern project needs for less money.

Nominees

Nowadays, there are tons of tools for test management. I had to narrow the scope, so I focused on the ten most popular tools based on the tech reviews from technical resources and QA forums. Then, I reviewed the tools documentation and picked up the five most promising ones for my current project: TestRails, X-Ray, Zephyr Scale, AIO Tests, and QA Deputy. Many of them are some of the most prominent players on the market, and I'd be willing to bet good money that you have already worked with at least two of them. However, they have different philosophies and sets of features, and the pricing differs significantly. So it's fascinating to compare them side by side doing the same things for the same project.

Assessment Criteria

To make a comparison, I analyzed project requirements and prepared a list of crucial criteria:

UX.
The most crucial criterion for the tool is an outstanding user experience. Overall it should speed up the process of writing tests and managing test executions. Nobody likes to do unnecessary clicks or to be unable to find the required stuff quickly.
Test reports and time tracking.
Statistics and estimations are beneficial for making decisions. Therefore having good reports is essential.
Requirements traceability.
The traceability between tests and user stories will help ensure that everything necessary has been considered.
Gherkin language support.
Test cases could be written in BDD style, so the tools should support the Gherkin language.
Automation.
The integration with GitHub Actions and good documentation for the test automation API is a must.
Performance.
The app should work quickly enough or it will become an annoying thing that will quite quickly distract engineers from their work.
Price.
The tools should be relatively inexpensive for a small team. However, the team will start to grow fast in the future.

Other additional criteria we include.

Jira application.
The team prefers storing tests and other development tickets in one place, so direct test management from Jira was considered a big plus.
Test data import/export. It would be good to have the option to export the test cases and test suites into some standard formats like ".csv" or ".xml ."It will allow us to migrate the test base to a new test management tool if there is a need with minimum effort.
Free trial. Trying the app alone can provide a much more user experience than watching video demos or reading product documentation.

Test Tool’s Test Drive

The tool's test drive included the following things:

Initial tool setup
Making a test set for a new user story
Making test cycles
Making requirement traceability
Time tracking
Test reports
Executing prepared tests

On average, I've used each tool for new projects for at least one week. Below is a short overview I've prepared for each tool, where I point out my impressions of each tool and omit some commonalities. Also, please remember that pros and cons are subjective, so take them with a pinch of salt!

TestRail

Home Page: TestRail: Test Management & QA Software for Agile Teams

Version under the test: 7.5.6

Overview. TestRail is a standalone app and you must consider that it takes a few minutes to register, get an instance, set up the integration with Jira, and start writing tests. You are unable to create and keep tests directly in Jira. However, installing a free Jira plugin can enhance your features. Overall, the tool allows you to link tests to Jira tickets and monitor test status from Jira. Additionally, it is possible to open bugs directly from the TestRail tool.

The TestRail has a unique feature that others don't. I am talking about UI scripts. This feature allows you to add some additional things on your own. For example, you can make custom colors for keywords inside the test cases through reminders when some specific page/test is opened, etc. You can customize almost everything inside the app (add custom templates for test cases, a company logo for test reports, and much more). The TestRail API can be used with lots of test automation frameworks.

Pros:

Reliable, stable, and fast
Intuitive interface for tests, test execution, and test management overall
Clean and easy way to monitor the testing status
Time tracking for each test as well as for test run/test plan
50GB of free cloud storage for test data (screenshots, scripts, videos, etc.). Note: you can have more storage space, but each additional 25 GB will be billed additionally to your regular subscription.
Custom fields for tests and test templates
User variables for the test automation
Good looking reports
Lots of options for different tools integration and test automation

Cons:

Quite pricey (37$ per user monthly or 34$/user monthly if you pay for a year).
The BDD test case format was unavailable in version 7.5.6, which is the one I tested. However, the BDD test documentation is already in the app documentation. According to emails from TestRail support, the feature is being actively developed, and the option is already present for early-access users. Meanwhile, the rest of the users can use a "free-text" format for the BDD tests.

Zephyr Scale

Home Page:Zephyr Scale - Test Management for Jira

Version under the test: 9.10.0

Overview. Zephyr is probably the most popular test management tool, as it has native support for Jira. It has three different versions: Squad, Scale, and Entreprise. The difference between those versions can be a topic of a separate article, so I will mention that the Scale has the most balanced set of features that fulfill all my criteria. Also, I wouldn't say I liked the Squad. It's so limited that it is incomparable to the Zephyr Scale.

You can install Zephyr as a Jira app from the Jira marketplace and then enable Zephyr in the project settings. That's it. You are ready to start writing test cases and make test cycles and test runs with Zephyr default settings. Everything works buttery smooth out of the box. The design is minimalistic and super simple. You need to do a minimum of actions to create/run/manage tests. There is one feature related to testing sets that I actually didn't like. When you are working with test sets, you have the option to create folders and subfolders. However, when you select a folder, you will see test cases from the folder and all nested subfolders at once without grouping them by subfolders. So, you need to structure subfolders not and adequately keep test cases in the parent folder if you have any subfolders inside.

The API for test automation is good enough for integration with all popular automation tools. Now, onto the subject of pricing. It's free if up to 10 users are registered in your Jira instance. It costs 4.78 per user a month if you have more. It's way cheaper than TestRail.

Pros:

There is no need to install other plugins for Jira integration; all needed things are out of the box.
UX for tool usage is one of the best on the market.
Test tickets have their suffixes, e.g., (XXX-T1, XXX-T2), and by default, are shown only inside the Zephyr board or test runs.
Requirement management is out of the box.
A "Traceability" tab inside each test case allows you to quickly add a link to any Jira ticket (e.g., Epic, User Story, Task, or Bug).
Time tracking feature for single tests as well as for test runs.A good set of default reports and the option to make custom reports using JQL (advanced Jira queries).
Free for teams with up to 10 people

Cons:

Fewer things can be customized compared to the other competitors
Test data will consume Jira cloud storage
Not all of the keywords are supported for BDD tests

Xray for Jira

Home Page: Xray - Native Test Management for Jira

Version under the test: 6.3.2

Overview. Xray is commonly considered a highly customizable alternative to Zephyr. It allows you to create separate test-type tickets for tests, executions, and test plans to manage them the same way as with all other Jira tickets. It has a lot of customization options, but the learning curve is relatively high.

I had to read a lot of documentation and watch video demos to start working with the tool. You can set up everything for any development process, yet you must adjust many project settings. Also, it takes much more clicks to start a new test case compared with the other tools. Other than that, I had issues with slow-loading tests during the trial. However, the cached requests caused it, as everything started to work fine once I rebooted my router. I didn't have such issues with the other tools, and I hope the Xray team saw my feedback and has already fixed the problem. Pricing: it costs 10$ per month if you have up to 10 users and 4.25$ per user monthly if more users are registered in your Jira instance.

Pros:

Anything can be changed, added, or reordered.
Different types of test styles, including data-driven and fully supported BDD.
Advanced requirement traceability.
Advanced Reports / Document Composer. You can make almost any type of test document in any format you want.

Cons:

The test runs looked unusual to me.
There are no test runtime tracking options from the box. However, it can be done with a third party plugin.
The separate test-type tickets heavily spam the project, quickly increasing ticket numbers overall. As a Jira SuperAdmin, you must modify Backlog and Board query settings to hide the newly created test types, create a separate QA/QC project, and keep tests there.

AIO Tests

Home Page: All-In-One Test Management App for Jira

Version under the test: 3.1.6

Overview. AIO Tests is a Jira-native app with many installations from the Atlassian marketplace and good reviews from users. The tool has a one-click installation, and you are good to go with the app's default settings. Minimalistic but has an enjoyable interface. Everything looks logical to me. There are two types of test cases: classic and BDD. The UX for BDD cases is the best of the compared tools. The tests are created with test suffixes, the same as in Zephyr Scale, so they are not shown by default on the board or backlog. Another interesting thing is that the app doesn't allow you to add created test cases to the test sets and test cycles if they are not reviewed and approved(published). It forces you to follow the review process.

I would have preferred a better way to organize the test cycles. You can re-run tests inside a cycle several times, but only part of the cycle at a time. You can make a copy of the cycle as an option, but I used to use the cycles differently. However, the cycles in AIO Tests have a unique feature to execute them in offline mode. It's done by exporting the cycle into an excel file and then importing it back into the app.

Speaking about documentation, it's worth mentioning that it's pretty detailed and has many examples. I liked the chapter that explains how to make a beautiful dashboard with AIO widgets for thorough test process monitoring. Pricing: it's free for teams with up to 10 people but costs 350$ a month for teams with 10-50 people.

Pros:

It feels simple, but yet a powerful test management tool.
Good for BDD tests.
Free for teams with up to 10 people.

Cons:

It's pricey if you have a team with over ten people ($350/month).
Test cycles can't be re-run.
Integration with other tools is limited compared to the other competitors, but it still covers most of the popular tools and automation frameworks.

QADeputy

Home Page: QADeputy

Overview. I saw QADeputy mentioned on several websites as a good tool to try. However, I have yet to have a good experience with the tool. Jira integration allows you to see the test run status for the Jira side. There are no configuration settings at all. It looks like the company hasn't yet finished the Atlassian security self-assessment program, so the capabilities are limited. I briefly checked the test management stuff in the QADeputy instance to see what the app proposes. I wouldn't say I liked the UX, in all honesty. Whenever I tried to create a new test case or test set, I was stopped by a need to fill out some required fields that are commonly optional in most other tools by default. The BDD format is not yet supported—only free-text or step-based tests. The reports have just a few basic templates. Pricing: the tool is free for one user and costs 20$ per user/month (18$ per user in case of an annual plan).

Overall, the app needs to do much more to reach the level of other competitors mentioned in the article. The tool needs to meet my basement criteria. So, I will skip the cons and pros of this one. I hope it will improve and give it a try in a year or two with better results.

Conclusion

TestRail, Zephyr Scale, Xray, and AOI Tests are all pretty good test management tools. They have pros and cons, but any tool can be used for most projects. All of them have plenty of features needed for test management. They all have good support for the popular project management and automation tools, reports, import and export of test cases, and so on.

Speaking about my own experience and from Svitla Systems' expertise in Quality Assurance, I recommend TestRail as it still feels like a safe option. However, the UI for some of you might need to be updated. Also, if you are into BDD tests, I am sure that 'AIO Tests' is worth trying instead. Zephyr Scale is a viable alternative. It delivers the same things as TestRail and AIO Tests, with a bit different test management flow. The advantages are that it is free for small teams and not so pricy for larger teams.

Also, you can use Jira built-in things (JQL for the reports, labels, etc.) All 3 Jira-native apps (AIO, Zephyr, and Xray) will make the tests accessible for the whole development team, so developers can see which tests the QA team prepared for the features in advance.

Xray for Jira might be an option for you if:

you have a specific development process.
you have the full support of BDD format tests is a must-have.
you need to spend a bit more time on the app configuration.

As for me, QADeputy is not yet on the same level as the other competitors mentioned in the article. Still, I recommend you try it to form your own opinion.

I hope the article helped you choose a new test management tool for your projects.

May the quality be with you!

Interpreting Machine Learning models using LIME and SHAP (by Arturo Soberón, Senior Data Scientis at Svitla Systems)

Svitla Systems Inc. — Tue, 07 Feb 2023 13:09:15 +0000

You are seeing three images of a Husky dog lying on the snow. The image on the left side is incorrectly labeled as a wolf; the image in the middle has the snow cropped out but it’s correctly labeled as a dog; the image on the right has trees cropped out and the dog is once again mislabeled as a wolf instead of a dog.

Why interpreting models is important
When someone acts autonomously, it’s important to understand how and why they make decisions. How does a judge reach a decision when determining if someone’s guilty or not? How do doctors decide on a specific treatment? Why does a screenwriter decide to kill a main character at the end of a season?

In the near future, we can expect Artificial Intelligence (AI) models to take over decision-making tasks such as lawsuits, non-urgent patient care, or screenwriting. But before we get there, we need to thoroughly understand how and why a decision was reached. Unlike humans, AI models cannot be interviewed to understand the motives behind their decision-making processes, and even if we could interrogate them, we would be unable to hold them accountable for the decisions they made in the past. For these reasons, it is crucial to leverage methods that explain model predictions.

Let’s take banking for instance. Risk analysis uses AI models to assign credit scores to applicants and thus, decide whether or not to lend them money — applications with low scores are rejected and applications with high scores are accepted. These scores are generated by processing applicant data such as age, reported income, existing credits on record, and more.

Imagine if an applicant with a high score requests a very large sum of money. Bankers are known for playing it safe, and applications for large sums of money are likely to be thoroughly reviewed even if scores are high. Particularly, analysts are interested in knowing if the predicted scores from the AI model are a true reflection of the client’s ability and willingness to pay. They’re also interested in knowing if a vulnerability in the model or distortions in the data cause it to predict such a high score for the applicant. Within this context, understanding and explaining the decision-making process behind the AI model matters because there’s a lot of money on the line. Lending money to people who are unlikely to pay it back can have a serious and negative effect on financial institutions and the humans working for them.

AI models are used to predict outcomes in our everyday lives. In marketing, analysts employ user search history to recommend products to users that they are likely to buy; in the field of medicine, organ scans help predict the likelihood of a specific disease; in gaming, player attributes help match gamers with players of similar skill levels.

Regardless of the industry, analysts are interested in explaining how predictions are made by AI models to determine whether they can be trusted. The methods introduced in this article will help you disentangle the logic behind any model, whether it uses language, images, or tabular data to make predictions. Additionally, you will learn how to apply these methods using Python. The code presented in this article can also be found in this GitHub repository.

The trade-off between prediction accuracy and model interpretability
Generally speaking, complex models tend to make more accurate predictions than simpler models. However, understanding the logic behind a model’s predictions becomes increasingly difficult as we increase its complexity.

It’s easy to understand how a marginal increase in one of the input variables affects the predictions made by a linear regression model(1). Unfortunately, there are times when the relationship between the features and an outcome cannot be accurately predicted by a linear model, so we have to move on to more complex algorithms. Although the predictions made by, say, an ensemble of gradient boosted trees(2) could accurately model a nonlinear relationship, understanding why a prediction came out as it did is not as simple as before.

The accuracy gained by using an ensemble method is offset by the loss in interpretability of the new model. This is the essence of the infamous accuracy-interpretability trade-off.

What is LIME?
Not long ago, we used to think this trade-off was inevitable. Fortunately, researchers have proposed a few methods that can help us understand the logic behind the predictions made by any model, no matter how complex.

Local Interpretable Model-agnostic Explanations, also known as LIME, was proposed by Marco Tulio Ribeiro in 2016. As the name suggests, it is a procedure that uses local models to explain the predictions made by any model.

To use LIME, we need:

A model we can probe as many times as we need to; and
A model that has a predict_proba method (for compatibility with LIME).

LIME works under the assumption that the model you want to explain takes a set of features (such as a vector of numbers, a sentence, or an image) to make a prediction. It also assumes you can probe the model as often as needed. The procedure consists of fitting an interpretable model to a data set generated by slightly modifying a single observation.

The data set is created by taking all the features from a single observation and then iteratively removing some of its components to create a new set of “perturbed” observations. The model processes observations in the perturbed data set, and then, an interpretable model fits the latest data.

For example, imagine a Natural Language Processing (NLP) model that takes in a sentence and predicts its negative tone. LIME creates the perturbed data set using the procedure described above(3).

As shown in the table, removing the word “lazy” decreased the predicted negative score to zero, while all other perturbations had a negligible effect on the predictions made by the model. The idea is to fit an interpretable model (such as a decision tree or a linear regression) on the perturbed data set to understand which components influence the original predicted score most. The column “Similarity to original” is the share of words from the initial observation included in the perturbation and is used as weights to fit the simple model.

This example shows the procedure for an NLP model. In the case of tabular data, the perturbed data set is generated by slightly changing input vector values. In the case of images, chunks of the image (called “contiguous pixels”) are grayed out, and predictions are made on the remaining pixels.

As you may have noticed, following this procedure explains why the model made its prediction around the original set of inputs. In layman’s terms, this means that the explanation is valid for a single observation (i.e., a local explanation) and not for all possible observations (i.e., a global explanation).It is for this reason that the explanations made by LIME are considered to be local.

LIME in Python
We will now use LIME to explain the vaderSentiment model. First, install and import both modules:

For each text, Vader returns a dictionary with the text’s negative, positive, neutral and compound scores. Although LIME is model-agnostic (i.e., it will work on any model), it requires the model to be similar in structure to a sklearn object. In particular, the model needs to have a predict_proba method, so we will first modify Vader to make it resemble an sklearn instance.

NOTE: You can skip this step if your model already has a vectorized predict_proba method!

Let’s now make a prediction on the following text:
I HATE Mondays! I missed the bus and forgot my lunch…

If you’re in a notebook environment use explanation.show_in_notebook(). Otherwise, use explanation.as_pyplot_figure(). This is what the results look like in a Google Colab notebook:

Based on this output, you can tell the word that most influences the text’s negative score is HATE, followed by missed. The words that most heavily influence the text’s non-negative score are I and the.

What is SHAP?
SHAP was proposed in 2017 by Lundberg and Sun-In. Like LIME, it yields local explanations for predictions made by black-box models. The main difference is that SHAP uses game theory (a branch of Economics) to make local explanations.

SHAP stands for SHapley Additive exPlanations and is based on adding Shapley values together (Shapley values are solutions to cooperative games introduced by Lloyd Shapley in 1951). In short, a Shapley value represents the expected value of one player’s contribution in a game. To link game theory with machine learning, we must think of the model as the rules of the game and features as players that can either join the fun (i.e., the feature can be observed) or choose not to join (i.e., the feature cannot be observed).

To compute SHAP values, we need:

A model we can probe as many times as we need to;
A sample of observations; and
A very specific structure for the model (to be compatible with SHAP).

NOTE: With LIME, we needed one observation to generate an explanation. Because of how Shapley values are calculated, we will need a whole sample to explain a single observation using SHAP.

Suppose we have a fitted model f that uses input variables X = (X1, X2, …, Xp) to make a prediction, f(X). The goal of SHAP is to explain prediction f(X = x) by measuring how much each feature contributed to the model’s output at X = x.

SHAP in Python (linear regression example)
Calculating Shapley values is a messy process (it requires evaluating the model under many different combinations of input variables). Still, they are easy to visualize when dealing with an additive model (such as linear regression). For the sake of this introduction, imagine we are trying to find the most important variables of a linear regression model that predicts the severity of a patient’s diabetes one year after they were diagnosed.

We start by installing SHAP and importing all the dependencies we need to fit a linear regression to the diabetes data set.

Then, we fit a linear regression model to the data.

We will now draw a sample from the data (200 observations) to create a partial dependence plot for the Body Mass Index (BMI or also bmi for this article) feature. This step will help us visualize what Shapley values are and where they come from.

The vertical red line represents the difference between E[f(x) | bmi = 0.015] and the unconditional expected value E[f(x)]. This is the SHAP value for bmi when bmi = 0.015. Once we have calculated the SHAP values of all the remaining features, we can use the value represented by the red line to understand the importance of bmi in the 42nd observation.

We can explain the importance of each feature for the prediction made on the 42nd observation using a waterfall plot.

The above plot is read from the bottom up. The x-axis represents the predicted values for the target (the severity of the patient’s diabetes), and the y-axis represents each feature. We start at the unconditional expected value of the predictions made by the model in the sample (141.2). Each horizontal bar represents the SHAP value of each feature, sorted by magnitude in ascending order. This means that the least important feature for the 42nd prediction is age, followed by s4, followed by s3, and so on. Notably, the most important feature is s5, and it had a positive impact on the predicted severity of the patient’s disease. In this example, s5 represents the patient’s serum triglycerides level, which is known to increase the risk of strokes and heart disease.

As stated before, the plot starts at E[f(X)]. Note that the plot ends at f(x) = 171.3, the predicted value for the 42nd observation!

The fact that the waterfall plot starts at E[f(X)] and ends at E[f(x)] is not a coincidence. Rather, SHAP values are additive, and they always sum up to the difference between the prediction made on the observation we are trying to explain (E[f(x)]) and the unconditional expected value (E[f(X)]) (4). This is a desirable property because it allows us to interpret the sum of all SHAP values as the difference between the prediction when no features are present and the prediction when all features are present. In this context, each feature’s SHAP value represents its contribution toward the prediction made by the model.

Generally speaking, SHAP values cannot be seen in partial dependence plots when we are using nonlinear models (like neural networks). However, the intuition remains the same regardless of what model we are using: we explain the prediction made on a single observation by calculating the SHAP values of all features.

Key takeaways
AI models are used to predict our physical health, financial stability, willingness to watch a movie, and even our skill level in video games. These models are often allowed to act alone and make decisions autonomously. For this reason, explaining the logic behind models’ decision-making processes is crucial. However, we cannot sit down and interrogate an AI model, which is why methods to explain predictions are necessary.

LIME and SHAP can be used to make local explanations for any model. This means we can use either method to explain the predictions made by models that use text, images, or tabular data. On the one hand, LIME fits a simple model around a prediction to create a local explanation. On the other, SHAP uses game theory to measure the importance of each feature. In the end, both methods are used to understand how features affect a predicted outcome, though each method uses a slightly different procedure.

In practice, we can expect LIME and SHAP to yield similar results, so choosing which method to use ultimately comes down to personal preference. In any case, we can use both methods simultaneously to validate each other’s explanations. To summarize:

LIME generates a perturbed dataset to fit an explainable model, while SHAP requires an entire sample to calculate SHAP values. This means that LIME requires only one observation, while SHAP requires multiple observations.
SHAP values are relative to the average predicted value of the sample, which means that different samples will yield slightly different SHAP values.
Although LIME and SHAP are model-agnostic, both Python libraries require models to have a specific structure. LIME is slightly more flexible because it only requires a predict_proba method. Even though SHAP requires a more specific structure, the method works with the most popular AI libraries, including sklearn, xgboost, tensorflow and many others.

In Svitla, we have experienced Data Scientists and Machine Learning engineers with proven records of training and producing AI models. Our staff comes from a diverse background of computer scientists, engineers, mathematicians, and economists worldwide. We have successfully built end-to-end Artificial Intelligence solutions for some of the biggest tech, finance, and health companies. The work presented in this article is the culmination of many years of experience working with AI models and explaining their decision-making process for our clients to derive a clear understanding of the products and solutions we deliver. Reach out to one of our representatives so we can help you make the best decision for your project.

All the code used in this article can be found in this GitHub repository.

Words of caution

LIME or SHAP cannot replace performance metrics
The mean squared error, mean absolute error, area under the ROC curve, F1-score, accuracy, and other performance metrics evaluate a model’s goodness of fit. On the other hand, LIME and SHAP yield local explanations for a model’s predictions. In other words, these methods are not meant to account for the quality of a prediction, but rather, they tell us why a prediction was made.

For example, if you know that a given prediction is incorrect (i.e., a false negative), you can use either method to understand which components drove the model to classify that observation falsely. This will help you identify edge cases where your model performs poorly and can ultimately lead to better performance metrics (if you retrain your model, of course).

LIME or SHAP do not imply causality
Another important consideration to bear in mind is that your model is an approximation of the underlying data-generating process. Hence, explanations made by LIME or SHAP do not imply that those components have a causal effect on the target variable.

Explanations solely tell you why the model made a certain prediction. They do not necessarily imply that these relationships are reflected between the features and the target.

Linear regression is a simple learning algorithm that dates back to the 1800’s. It essentially finds the line that best fits a set of data to make predictions.
Tree-based gradient boosted ensembles are a series of decision trees (i.e., an ensemble) that are trained sequentially. Each tree in the ensemble focuses on the errors made by its predecessor to create an increasingly accurate model.
Natural Language Processing is a branch of Machine Learning that focuses on learning from text, voice and other forms of language-based communications.
Shapley values are based on the axioms of efficiency, symmetry, dummy and additivity. As a consequence, SHAP values are naturally additive. Winter (2022) offers a comprehensive review of the additive property of shapley values.

Communications in Remote Teams

Svitla Systems Inc. — Fri, 22 Apr 2022 12:33:18 +0000

At Svitla Systems, we know a thing or two about working remotely. For 19 years now, our employees have enjoyed the perks of a far more balanced lifestyle since we’ve encouraged remote work long before the pandemic struck.

While some organizations struggled, our rich expertise in remote work made it seamless to continue to provide ongoing, uninterrupted support for our clients, while keeping our employees safe and comfortable.

After many years of managing administrative processes for our remote employees scattered all over the world, we believe it’d be of great value to share firsthand experiences from our employees on how they’re coping with this new reality, what tools help them be more effective at their jobs even if working remotely, and some of the best practices they employ to ensure smooth and transparent communication with their teams.

Here, you’ll find real stories from Svitla Systems employees where they share their personal experiences, advice, and even some useful life hacks. Introducing our contributors, we’d like to say special thanks to Anna Bigun, Project Manager, Kostiantyn Bilyk, Team Lead and Lead iOS Software Engineer, and Dmytro Dehtiarov, Senior Software Engineer.

All three of them have convergent approaches to remote work that highlight how they’ve achieved a streamlined, transparent, and effective way of working remotely that leverages all the good bits of working from home while overcoming the challenges of missing out on face to face interactions.

Here we go!

How to manage a remote team communication

When managing communication in a remote team, one must always drive things home to what truly matters: trust and transparency. Almost all professionals understand that work is about being, well, professional, so it shouldn’t really be a problem for employers to allow employees to work from home either intermittently or always. Yet, that’s not the case still. Even with the pandemic, many executives were anxious about sending workforces home, as they feared they wouldn’t perform as well as they did if they were in an office.

Well, employees proved a lot of people wrong. In fact, studies show that working from home can increase productivity by 13%. A survey shows that 77% of people who work remotely at least a few times per month show increased productivity, with 30% doing more work in less time and 24% doing more work in the same period of time.

Other reports performed during the pandemic showed an increase in productivity by 47% since March of 2020 (compared to March and April 2019), and have deciphered when people are the most productive, which is on Tuesday, Wednesday, and Thursday between 10:30 am and 3:00 pm. Interesting, right?

With this context in mind, Anna Bigun, one of Svitla’s key Project Managers, shares the following insights about how to manage communication in remote teams successfully.

Consider time zones as they play a key role. When working with global teams, it’s important to factor time zones into your processes. If your team is all over the world, a person might not reply in your time zone, and that’s okay and to be expected. Thus, it’s critical that each member understands that team members are in different time zones and agree on common hours that are convenient for everyone to hold meetings. This rings true for clients as well. For example, it’s not uncommon to have a two-hour gap with team members working in the US as some are in Eastern Time while others are in Pacific Time. Now add Eastern European Standard Time to the mix and we got ourselves a lot of planning to do. Of course, you can almost always find that sweet spot where you all overlap even if it’s just for a couple of hours so you can communicate instantly, avoiding blockers by not having to wait for an answer just because you sent your email an hour late. This also helps boost efficiency, and productivity, maintaining a healthy work-life balance.

Choose your communication platform wisely. There are many tools and apps out there for you to communicate, whether it’s via email, Skype, Slack, or any other solution, the key is to carefully assess what will work best for the team and what will be convenient for everyone. This doesn’t mean that you have to commit to a single channel, but it is recommended that you keep it as narrow as possible because having too many options could lead to unwanted chaos and messy conversations. Keep in mind that structure is very important when working as a remote team. For example, Slack allows you to create theme-based channels and add people that are relevant. In contrast, Skype is not as friendly in that regard which is why it’s not as effective in most cases.

Rules and structure matter. Oftentimes, we see developers turn down a project because it requires too much communication. This is a downside of remote teams as remote work requires a lot of communication and calls. Thus, it’s very important to establish ground rules and regulations. For example, keeping an agenda of the questions that will be addressed during the call, the duration, and so on, goes a long way in keeping the call succinct and to the point. If time runs out, you can schedule a separate call but you respect everyone’s time for the first call.

The right methodology can be your best ally. If you are a Project Manager, you know it’s all about the methodology. Project Managers can advise clients about which methodology to use because some approaches work better for remote teams. It’s very valuable to select the right tracking tool to understand the team’s workflow and how everyone’s plate looks in terms of capacity management. Also, if some people don’t have any tasks assigned, you can plan and strategize what to do in such scenarios. When choosing the right methodology, don’t forget about documentation as it is a key element of all good projects.

Keep things social. People need people, that’s a fact. Some clients allocate budgets for team-building activities but this is not always possible. People tend to change their behavior when working online and offline, so it’s important to make the distinction and understand the context of any team gathering. For instance, online communication opens up the door to a lot of misunderstandings, so it’s recommended that you create a channel with the purpose of just communicating and having fun, leaving all work-related matters out of the conversation. As such, team members can share funny pictures, GIFs, personal news, or comment on anything they want as a means to relax, distract themselves in a good way, and strengthen team bonds.

How to manage technical processes and motivate the team

Of course, working from home comes with its challenges and it's important that organizations and individuals alike take these into consideration to be as effective as possible. For example, Stanford found that only 65% of Americans have internet fast enough to handle video calls, but the upside is that work-from-home employees are far less distracted, spending up to 30 minutes less talking about non-work topics.

One of the biggest aspects that have leaders and executives worried about is the lack of social interaction, a key factor in the success of any company. Some employees thrive on social interaction and struggle with being at home for this reason. In fact, never have we seen more organizations talking and worrying about employees’ mental health, as over time, it is bound to hinder productivity, employee satisfaction, and the overall wellbeing of the employee, which is, of course, the most important thing.

Here’s what Kostiantyn Bilyk, Team Lead and Lead iOS Software Engineer had to share about managing technical processes while simultaneously keeping the team engaged and motivated.

Hold regular, value-driven team meetings. Overall, the goal of a team is to be in constant communication to reach a common goal, and scheduling regular calls goes a long way in promptly identifying bottlenecks, giving solutions, and keeping everyone engaged in where the project is going. Through daily standup meetings, the team communicates what they’re working on, share their current workload, talk about issues they run into that are holding them back, bring up any questions, share knowledge, show results, and so much more, becoming a daily ritual where communication and knowledge transfer takes center stage.

Communicate the good and the bad. Sprint demo meetings are the place where teams show and highlight the results achieved. These meetings also serve as the stage to request feedback and talk about what was done right, what needs to be improved, and what will be done in the next sprint. Whether it’s a win or a feature that didn’t work out the way the team expected it to, there’s always a positive in all of it as the team either reinforces what they did right or takes key learning items to grow and be better. For example, developers often have meetings where the client is not invited so they can discuss internally how to best approach a feature, discuss an implementation issue, or bring up new ideas on how to develop something new. Either way, the purpose of these calls is to share knowledge and work on future tasks more assertively and effectively.

Keep the team informed. It’s good practice to keep a transparent, two-way street communication with team members so everyone feels connected, engaged, and part of the team. Even though everyone is most likely working on separate tasks, it is the sense of unity and belonging that moves a project forward. In some cases, Dev Leads hold meetings with the client once a week to discuss future tasks, share updates, and talk about the overall progress of the project. After these meetings, usually, Dev Leads brief the rest of the developers so they better understand where the project stands and where it’s headed.

Don’t treat people as employees, treat them as people. Building internal relationships with peers and colleagues is what ultimately builds engagement so everyone feels connected. Online chats about life and off-the-record topics (even memes) is a valuable start to break down barriers. Sometimes, it can be challenging to keep this type of communication going since teams mostly have interpersonal communication happen during different time zones, so it’s hard to coordinate everyone in one place at the same time.

Keep daily channels of communication. Slack has become an ally for remote teams all over the world. Teams typically have several channels open to address different topics. Before, calls were more effective, but now, it’s all about emails, instant messaging, and lightweight videoconferences.

Always account for the challenges of remote communication. One of the most challenging aspects of managing technical processes while keeping everyone engaged is to pick up on emotions. Sometimes, especially when team members are chatting, it is very complex to fully discern what a person is actually trying to say if they’re being overly polite, genuine, or sarcastic. Most prefer calls with camera on to this end so as to avoid any type of misunderstanding.

Respect everyone’s time and show empathy. Agree on specific time slots to hold meetings or schedule specific goals for a specific time so everybody knows what they should be working on and what to expect. Account for everyone’s time zones and be respectful of their time by keeping meetings strictly within the allotted time frame. It’s recommended that you keep a single channel of communication and schedule calls ahead of time and take everyone’s schedule into consideration, as much as possible.

How to be involved in communication in an effective way when you work in a big team

Overcoming communication challenges in remote teams can be challenging, but with the right team members, attitude, and tools, it becomes a breezy ride. Setting guidelines, using a standard communication tool, replacing in-person meetings with videoconferences, implementing a project management system, keeping communication channels open, giving feedback, and going that extra mile, are some of the best practices recommended to avoid most of the pitfalls of working remotely.

With the Covid-19 pandemic to consider, the shift to an online workplace is not only widespread but kind of a given in the IT industry. As many organizations are learning, managing communication flows in remote teams is a sinuous road with many twists and turns.

Keep in mind that human communication happens in bursts or periods of high activity followed by periods of little to no activity. These bursts of communication with periods of quiet in between are what make teams go round. Quiet times are used to develop ideas, focus, and carry on deep work while high-burst times focus energy, solve specific questions, and move teams to the next challenge ahead.

With this context in mind, it’s important to throw out the idea of message-based communication is asynchronous. Instead, team members should align routines and communicate in short time frames when everyone is there and can respond quickly, attentively, and effectively. That’s the secret sauce to higher performance.

These are the insights from Dmytro Dehtiarov Senior Software Engineer in regards to effective communication when working in a big team.

Learn to discern what matters. In big teams, there’s always a lot of noise across many different communication streams. To help this matter, it’s recommended that teams hold daily meetings to plan, discuss blockers, and discuss anything project-related with an open relationship with the client to discuss any concerns directly, with no middlemen involved.

Get your zen on. One of the most challenging aspects of remote team communication in big teams is technical issues. Sometimes people have slow connections, or the electricity is gone, so it’s difficult to keep the communication going when there’s a slow response time. Because team members can be in different time zones, it’s challenging to get answers in real-time, so it’s crucial that people learn to be okay with this and respect everyone’s unique circumstances.

Keep strict working hours. In the case of remote work, it’s easy to get carried away and work past your working hours. A late-night here and there never hurt anyone, but it’s important that you both respect your personal time and the personal time of others. In big teams, it’s recommended that everyone determines an overlapping time slot where almost everyone is online and available to improve response times.

Build a true partnership. Your peers and colleagues should be your partners and be there for you. If someone asks a question, the team is there for each other to search for or provide an answer, in true partnership form.

Conclusion

Remote teams are here to stay and will become the future of work. Over 40% of the global population is mobile, and highly-developed countries have an express ticket to reach 75% this year.

As entire workforces go remote, it’s critical that organizations shift to a new kind of management and leadership style that takes into account the unique challenges and benefits of working remotely across different time zones, languages, and cultural barriers.

While being spread out over the world can lead to teams feeling as if they weren’t truly a team, communication is chief in building a highly-collaborative, creative synergy amongst teams. Bridging that virtual gap can be challenging, but not impossible.

Svitla Systems has favored remote work for several years now (way before the pandemic) and is highly experienced in working with teams in different geographies, in a seamless and effective way. Our team of professionals is highly proficient at avoiding communication pitfalls and fostering strong collaboration ties with peers, partners, and clients alike.

Through a strong sense of team purpose, Svitla Systems professionals break down silos and communication barriers, with a clear sense of how their individual work fits into the bigger picture. For more information about how our team of highly-skilled and collaborative engineers can help your remote-based projects, reach out to one of our Svitla representatives and enhance your project’s communication in a virtual team setting.