DEV Community: Jorge Catriel Lopez

Navigating the Future: AI and Automation in Telecommunications

Jorge Catriel Lopez — Mon, 18 Mar 2024 19:44:24 +0000

The telecommunications industry is on the brink of a new era, shaped by AI and automation. These technologies are set to redefine service delivery, network management, and customer engagement. But what do these changes mean for the industry's future?

AI's Role in Enhancing Connectivity

AI transforms telecommunications by optimizing network operations, improving customer service through chatbots, and enhancing security. This technology is crucial for predictive maintenance, reducing downtime by anticipating and rectifying network issues before they affect users. AI's ability to manage and analyze vast datasets in real-time supports the seamless deployment of next-generation services, including 5G, enhancing connectivity across the globe.

AI technologies such as speech-to-text (STT), text-to-speech (TTS), and computer vision (CV) enhance the interaction between users and applications, while noise control and speech compression technologies ensure clear communication.

AI Virtual Assistant Technology Guide 2022

Evgeniy Krasnokutsky ・ Mar 21 '22

#ai #virtual #avatar

Automation: A New Standard for Efficiency

Telecommunications companies are leveraging automation to revolutionize their operations, from network management to customer service. This shift not only boosts operational efficiency but also significantly cuts down manual labor and operational costs. Automation allows for real-time network optimization and the automation of customer interactions, providing a more consistent and reliable service experience. This evolution is instrumental in accommodating the growing demands for telecommunications services, ensuring scalability and sustainability.

In telecommunications, Python has emerged as a powerful tool for automating tasks, including setting up webhooks for messaging services. A DEV Community guide on creating a WhatsApp bot with Python exemplifies this, providing code snippets and explanations on setting up APIs for enhanced messaging automation. This practical application demonstrates the versatility of automation in improving efficiency and responsiveness in telecommunications services.

Setting Expectations

While AI and automation promise to enhance efficiency and service quality, setting realistic expectations is crucial. These technologies also present challenges, including the need for substantial data for AI training and potential risks in data security and privacy. Furthermore, the automation-driven transformation in the workforce necessitates upskilling and reskilling initiatives to prepare employees for the future. Navigating these challenges requires a balanced approach, prioritizing technological advancements while ensuring ethical and responsible use .

For a more detailed exploration of these topics, consider reviewing the insights provided by these sources:

Adapt IT's discussion on AI's impact in telecommunications here.
Ericsson's insights into AI and its benefits here.
McKinsey's analysis on how AI is revolutionizing telco service operations here.
Intellias' overview of AI in telecommunications here.

What are your thoughts on the future of AI and automation in telecommunications? Have you experienced these changes firsthand? Share your opinions and experiences in the comments below!

Header photo by Gerard Siderius on Unsplash

Why is replicability essential?

Jorge Catriel Lopez — Tue, 05 Jan 2021 12:58:15 +0000

Introduction
Quality control in science
- Bias
- Lost data
- Fraud!
Who watches the Watchmen? A broken peer review
Our work
References

Introduction

Reproducibility of results is imperative for a sound research project. The replication of experiments is an essential part of scientific research in which significant theories are based on reproducible results (Staddon et. al. 2017), so failing to provide this results go against the very basis of what we are doing. Replication is extremely important. It means that after a study is published, other scientists can then conduct the study themselves independently and come to about the same conclusions. If a study’s results can be consistently replicated by other scientists, that means the study is more likely to be valid.

In academic environments, the reproducibility factor is often overlooked and scientist don't put enough effort into establishing essential environments and workflows early on in the project life-cycle. Readers are often expected to figure things out themselves from the minimal information in the methods section if they want to reproduce the results themselves.

Quality control in science

As we said, readers are often expected to figure things out themselves from the minimal information in the methods section if they want to reproduce the results themselves. Why is that? There are three causes that we've found mentioned in the literature (among others):

Bias

Fierce competition, strong incentives to publish, and commercial interest have inadvertently lead to both conscious and unconscious bias in the scientific literature. And, the higher the vested interest in a field, the stronger the bias is likely to be. A major cause of low reproducibility is the publication bias (when the outcome of an experiment or research study influences the decision whether to publish or otherwise distribute it) and the selection bias (the sample obtained is not representative of the population intended to be analyzed).

One cause for lack of replicability is that many scientific journals have historically had explicit policies against publishing replication studies (Mahoney 1985). Over 70% of editors from 79 social science journals said they preferred new studies over replications and over 90% said they would did not encourage the submission of replication studies (Neuliep & Crandall 1990).

Lost data

A vast amount of data is lost simply because we don’t know how to access it or is not free. As Editor-in-Chief of "Molecular Brain", Tsuyoshi Miyakawa has handled hundres of manuscripts and has made countless editorial decisions requesting that the authors provide raw data. On his article "No raw data, no science: another possible source of the reproducibility crisis", Miyakawa details how simply asking for this leads to several authors to withdraw their submissions.

Flowchart of the manuscripts handled by Tsuyoshi Miyakawa in Molecular Brain from December 2017 to September 2019

"With regard to the manuscripts that the authors withdrew independently, there are possible reasons for withdrawing after being asked to provide raw data.

One possible reason is that although they actually had the raw data, the authors were not willing to gather all the raw data and upload them. It’s also possible that authors did not disclose raw data which they could use as an exclusive source for data mining to publish additional papers later. Another possible reason is that they chose journals where the disclosure of raw data is not required at the time of publication. However, the “data mining” hypothesis is unlikely for many of the authors in the cases considered here, since most of the rejected manuscripts did not contain big data that are suitable for data mining, as most of the requested data prior to peer review were images for western blotting or for tissue staining.

Note that I asked not only for raw data but also for absolute p-values and corrections for multiple statistical tests; therefore, the possibility cannot be excluded that some of them did not wish to provide absolute p-values or to conduct corrections for multiple tests, though I do not think that these can be the primary reasons for the withdrawal. As for the ones that I rejected, it is technically possible that the insufficiency or mismatch between raw data and results are honest and careless mistakes."

Even if he's not able to verify the reason why the authors decided to withdraw their work, he proposes that a lack of raw data or data fabrication is another possible cause of irreproducibility.

Fraud!

Fraud and questionable practices (excluding studies that didn't work, gathering more data that needed, rounding errors, etc) obviously affect the reproducibility of the study. If the work is wrong, how can you get to the same results without commiting fraud yourself?

Who Watches the Watchmen? A broken peer review

After all, who's resposible for verifying other people's work? On "Scientific Knowledge and Its Social Problems", author Jerome Ravetz states that it wouldn't be feasible to erect a formal system of categories of quality for a piece of work, and train a corp of experts as asessors to operate in their framework, "for the techniques are so subtle, the appropriate criteria of adequacy and values especialized, and the materials so rapidly changing, that any fixed and formalized categories would be a blunt and obsolete instrument as soon as it were brought in to use".

This results in the fact that if there are to be truly expert assesments of quality of work, they must be made by a section of those who are actually engaged upon that work. In itself, this provides no guarantee whatever that proper assessments of quality will be made, or that they will be used appropriately for the maintenance of quality.

Our work

As we continue our work on our graduate thesis, we have accumulated a suite of scripts that need organization and cleaning. A basic R package, with a good project layout, will not only make our life easier, but also allows us to tackle our reproducibility problem. It also helps ensure the integrity of our data, makes it simpler to share our code with someone else, allows us to easily upload our code with our manuscript submission, and makes it easier to pick the project back up after a break.

This follows from several tips that we have to keep in mind:

For every result, keep track of how it was produced: we are avoiding manual changes in data, everything must be modified through a script to ensure that every change is registered.
Version control all scripts: we are currently using Git, and a private repository on Github.
For analyses that include randomness, note underlying random seeds.
Always store raw data.
Connect textual statements to underlying results: we are currently using R markdown files to write about our development and the changes we make to the data.
Provide public access to scripts, runs, and results: for now, our work and data are private, but that has to change as soon as we are done!

References

Hard Problems in Data Science: Replication
10 Rules for Creating Reproducible Results in Data Science
No raw data, no science: another possible source of the reproducibility crisis
What Can Be Done to Fix the Replication Crisis in Science?
Special Section on Replicability in Psychological Science: A Crisis of Confidence?

Original cover photo by Brandable Box on Unsplash

Discuss: What's your favourite book on Data Science?

Jorge Catriel Lopez — Thu, 17 Sep 2020 13:58:00 +0000

It could be for any number of reasons! Maybe it generated new opportunities for you, it taught you something valuable, or you just enjoyed it more than any other book. So tell me: what's your favourite book on data science?

Cover image by Enayet Raheem on Unsplash

The Lies of Productivity Porn

Jorge Catriel Lopez — Sat, 12 Sep 2020 00:11:48 +0000

If there's something we can all agree on, it's that we would love to be more productive. Just take a look at the almost ten thousand posts published under #productivity, or the 13501 books on the subject on GoodReads (as of September 2020).

We would love it, yet we aren't. Whether we find the tasks before us tedious, scary or boring, there's always something else to do. Something better. Something urgent or entertaining. Of course we know that we shouldn't, that there shouldn't be anything more important than our responsibilities. And yet, we still do it. Time and time again.

We look for guidance on the subject, diving into self-help books, courses and talks. We sit on a mountain of strategies and plans that don't amount to anything, because we are still not productive enough. And yet it feels good!

Surely learning and planning about optimizing my agenda can't be wrong, right? Learning how to better yourself can't be unproductive! But, as you read and learn and plan about this, have you actually move forward with your tasks? Or are you spending more time thinking about your schedule than doing stuff?

This is called Productivity Porn.

Original XKCD comic.

What are we talking about?

Productivity porn is just another form of procrastination. We spend (too much) time thinking about what we are gonna do, what bullet journals do we need, what new apps are out there that can help us keep track of time, and reading "Top 10 tips to avoid procrastinating (#3 will SHOCK you!)". And like any other form of procrastination it leads to poorer health, more perceived stress, and fewer wellness behaviors.

When I'm in writing mode for a novel, I get up at four a.m. and work for five to six hours. In the afternoon, I run for ten kilometers or swim for fifteen hundred meters (or do both), then I read a bit and listen to some music. I go to bed at nine p.m.
Haruki Murakami

Take a look at that, how wonderful, how inspiring. Even reading about it feels great.

So why doesn't mine look like that?!

We all want to be like Murakami or whoever else you are comparing yourself to. We tend to copy their lifestyle, but sooner or later we find out it's not the right thing for us. We are different. We have different lifestyle, thinking patterns, and responsibilities from them. You should only compare yourself to you.

Claire Major has this great article called “Productivity during a pandemic” where she talks about how the combination of events on this glorious 2020 makes it harder for people to write, even if “writing helps [them] cope with the stresses and distractions”. Add to this the (new?) peer pressure of being extra productive and not waste a single moment, and your anxiety levels will go through the roof!

(And did you even consider "who is this ‘hustle culture' actually benefiting?")

Caught in a trap, you can't walk out (yet)

Is there a way out? Well, it's a tricky subject. Can I really say yes without providing an answer that falls under the porn label?

Productivity porn betrays personal insecurity. The most productive people I know don't read these books, they don't watch these videos, they don't try a new app every month. They are far too busy getting things done to read Getting Things Done. But what they've also come to accept is that working hard is indeed, just hard work. There's no trick to it.
The Trap of Productivity Porn - Jamesbedell

We don't spend enough time doing the things we are avoiding. Well, duh! that's what got us here on the first place! But it really is that simple. There are no magic words that will make it all better. Its just work.

But that's not what this article was about: I'm not here to give you the magic answer. The point is that we don't even realize we are not being productive, because passive thinking can give us the illusion of productivity and won't even help us to build productive habits. That's it. Not a solution to the base problem, just a heads up: hey, this is not as productive as you may think it is.

What about you? What has been your experience like? I want to hear about it!

More about this

Cover photo by Carl Heyerdahl on Unsplash

Git, but not dumb down: Local branches

Jorge Catriel Lopez — Thu, 03 Sep 2020 11:34:03 +0000

Introduction

Welcome back! Be sure to check out the first past of this series before you jump into this one.

Let's start with a brief reminder of the last post:

To summarize: every time we stage a file, a blob object is created, storing the content of each file. Its name is the SHA-1 of the file's content and it’s stored under .git/objects. Whenever we commit the changes to the repository, we create two new kinds of files: a tree, which stores more information about the files changed under it and a commit, with all the information about who saved the snapshot and why (the message!).

We took a quick look at the .git/objects folder in your repository in our last post. Today, we are going to look at .git/refs and the concept of local branches.

Branching

First, we need a new repository (practice makes perfect):

Now that we have a working repository, we can introduce a little bit of theory. A branch in Git is simply a lightweight movable pointer to one of your commits. The default branch name in Git is master. As you start making commits, you’re given a master branch that points to the last commit you made. Every time you commit, the master branch pointer moves forward automatically.

Internally, a branch is just a file in your .git/refs/ folder that contains the SHA-1 id of a commit. Before we commited the new_file, this was the structure under .git:

And this happened after the commit:

Can we take a loot at that master file? Of course!

If we take a look at the log, just to be sure, we can see that it is in fact the id of the first commit that we made.

To get some content in our repository, and to check that what we just said really works, let's add another file:

HEAD

We have confirmed that the pointer "moves" whenever we make a commit, so let's bring another branch into this:

We used the git branch command to create a new branch. Can we just add commits to it?

What happened there?

We got a new file called new_branch.
We made a new commit that was supposed to be on the new branch, but!
We found out that both new_branch and master point to this last commit.

Why did master update? Creating a new branch doesn't mean you are working on it. Git utilizes a special pointer called HEAD to know in which branch you are working on. If we take a look at it, before we switched branches

we can see that it contains a reference to another reference, in this case your master ref file. We'll use the git checkout command to switch branches and add a new file to test it:

HEAD updated to the new branch after checkout. You can look at this on your log too (HEAD -> current_branch):

Merge

After we looked at how we can diverge our work, we'll take a look at merging. The git merge new_branch command will replay the changes made on the new_branch branch since it diverged from master until its current commit on top of master, and record the result in a new commit. So what happened to our ref files?

Now the both point to the same commit, and HEAD still points to master from the checkout. If we continue working on this branch, we can see in our log that new_branch will be left behind.

Workflow

Now that we introduced the basics of branching, we'll take a look at a couple of git workflows. First, you must know that there is no standardized process on how to interact with Git. You can commit after each new word on a file, or make and merge branches for every file, for all Git cares. Use it however works for you and your team.

Feature branch

The core idea behind the Feature Branch Workflow is that all feature development should take place in a dedicated branch instead of the master branch. This encapsulation makes it easy for multiple developers to work on a particular feature without disturbing the main codebase.

From Atlassian's Git Feature Branch Workflow

Its just that simple! Whenever you need to add some feature to your project, you checkout a new branch, commit anything that you need there, and then merge it into the main branch.

Gitflow

This workflow is the idea of Vincent Driessen on 2010. You really should read that post, as its really nicely explained!

Gitflow doesn’t add any new concepts or commands beyond what’s required for the previous workflow. It assigns very specific roles to different branches and defines how and when they should interact. In addition to feature branches, it uses individual branches for preparing, maintaining, and recording releases.

Future notes

After this post, the plan is to continue the branch topic with remote branches, reset, and rebasing.

I'm interested to know what do you think, if you have any comment or suggestion! Thanks for stopping by, see you again soon!

Git, but not dumb down: Introduction

Jorge Catriel Lopez — Fri, 28 Aug 2020 23:28:00 +0000

Motivation

Tutorials on git are present everywhere. So, why write another? There’s a trend I’m not a big fan of: a lot of people like to divide the content between “basic” and “advanced” git.

Listen, you don’t have to know how a motor works to drive a car. I know. But sooner rather than later, something is gonna break. And if you don’t know how the thing works, other than how to use some basic commands, odds are that you are the one who’s gonna break it and will get stuck waiting for A Professional to show up.

This post is meant to explain Git in a clean and concise way, so you can be The Professional. I’m not going to try to explain the advantages or disadvantages of using git or any other VCS. I’m also not gonna divide the content into “basic” and “advanced”. To truly understand git, you must look below the surface, at the internals.

If you want to learn about why Git is such a great tool, Attlasian wrote this fantastic article about it.

If you want to learn about the implementation of Git, the source code is available here.

There’s also this great book about it, Pro Git, that I can’t recommend enough, and this post by Nico Riedmann.

Let’s start with the basics. We’ll create a new repository on any folder on your computer, by opening up the terminal on it and executing the command git init:

When you create a new repository into a folder in your computer, your workspace is divided in three main areas:

Working directory

The folder itself, in which you will create, modify, and delete files.

Stage area or Index

All the changes tracked by git will reflect in the .git directory that the git init command created, even before you commit them. You can use this index to build up a set of changes that you want to commit together. When you create a save a snapshot, you save what is currently in the index, not what is in your working directory. How do we do this?

After making any changes to the working tree, and before running the commit command, you must use the add command to add any new or modified files to the index.

This command can be performed multiple times before a commit. It only adds the content of the specified file(s) at the time the add command is run; if you want subsequent changes included in the next commit, then you must run git add again to add the new content to the index.

Repository

The (local) repository is everything in your .git directory. Mainly what you will see in your repository are all of your snapshots of the project.

In a few paragraphs, we can see the main git workflow in action: you modify a file, prepare the changes to be saved, and save the snapshot (the current version of your project) to the repository.

As we said, the git init command has created a new folder .git in our working directory. What's inside?

As it turns out, a lot of things! We'll look at them as we need them. For now, we need content. We'll create a new file, called new_file.txt and insert some text to it:

The new_file file has been modified, but its not yet staged. We can see this if we take a look at the git status command:

Git knows of the existence of the file in the working directory, but it is not tracking any change on it. We can make it do so by staging it:

Does this mean that git has stored the file in its history? Not yet. If we take a look at the output of git log, which shows all the commits made, we see that there's no snapshot of the project stored yet.

But! Changes have been made to the .git folder:

We have a new folder and file under .git/objects/, with a long and weird name. We only have one file, so this must be the snapshot, right? And if we look at what's inside, we should get a single sentence ("This is a new file"):

Clearly, it didn't work, as that is a binary file. And it gets worse! Look at what happens when we save the snapshot of our project to the repository:

The plot thickens! We have two new folders and files under .git/objects/ now. Let's take a moment to see what's going on here.

Git internals: How is this data saved?

After we added the new_file object to the repository, Git created a file under the objects folder with a weird name. This file is named with the SHA-1 checksum of the content and its header. The subdirectory is named with the first 2 characters of the SHA-1, and the filename is the remaining 38 characters.

Can we look at what's inside? Of course. Git has the cat-file command to inspect it:

Great! So the CONTENT of the file is stored in the repository. This is important, as a lot of people think that git stores only the diff between files.

So, let's say I create a repository with a 50gb in it. And let's say that I modified it three times. Does this mean that I'll have a 150gb size repository? No. Git has a compression layer that we could talk about later.

Ok, git stored the content of new_file, but just that and nothing else, not even the name of it!. What about the other two files that were created after the commit?

The repository’s content is stored as tree and blob objects, with trees corresponding to directory entries and blobs corresponding more or less to file contents. A single tree object contains one or more entries, each of which is the SHA-1 hash of a blob or subtree with its associated mode, type, and filename.

Now we have that extra information we needed about the file. But, git is a collaboration tool, and as in any collaboration tool, we need some information about the user. We don’t have any information about who saved the snapshots, when they were saved, or why they were saved, yet. This is the basic information that the commit object stores for you.

To summarize: every time we stage a file, a blob object is created, storing the content of each file. Its name is the SHA-1 of the file's content and it’s stored under .git/objects. Whenever we commit the changes to the repository, we create two new kinds of files: a tree, which stores more information about the files changed under it and a commit, with all the information about who saved the snapshot and why (the message!).

What’s next?

We’ll stop here for now, and we’ll get into branches on the next post. Let me know of any suggestions you've got, or if I made any mistake, as this is my first post. I hope you enjoyed it!

Original cover photo by Emile Perron on Unsplash.

DEV Community: Jorge Catriel Lopez

Navigating the Future: AI and Automation in Telecommunications

AI's Role in Enhancing Connectivity

AI Virtual Assistant Technology Guide 2022

Evgeniy Krasnokutsky ・ Mar 21 '22

Automation: A New Standard for Efficiency

Setting Expectations

Why is replicability essential?

Table of Contents

Introduction

Quality control in science

Bias

Lost data

Fraud!

Who Watches the Watchmen? A broken peer review

Our work

References

Discuss: What's your favourite book on Data Science?

The Lies of Productivity Porn

What are we talking about?

Caught in a trap, you can't walk out (yet)

What about you? What has been your experience like? I want to hear about it!

More about this

Git, but not dumb down: Local branches

Introduction

Branching

HEAD

Merge

Workflow

Feature branch

Gitflow

Future notes

Git, but not dumb down: Introduction

Motivation

Working directory

Stage area or Index

Repository

Git internals: How is this data saved?

What’s next?