DEV Community: aletelecom

GPON Network visualizations — Part 1

aletelecom — Sat, 04 Jan 2025 14:19:22 +0000

This is the first of a short series of visualizations I’ve done in order to identify some “difficult to spot” or “never thought of” conditions on GPON networks, which of course could be generalized to any network NEs as long as it makes sense.

These visualizations are done using the Python language, and some of the most popular data analysis tools available, like Pandas, Matplotlib, Seaborn, and so on.

Today’s visualization

The visualization that I’m showing you today is a annotated “heatmap” that would allow you to “see” the distribution of the average of the client quantity in each PON port as if you’d be seeing the OLT from the front:

An example of the visualization we are introducing today

So, each column represents a slot of the OLT, each row represents a PON port, the number inside each cell is the average of the quantity of client in that PON port, and the color of each cell is a visual cue of how much “filled” the OLT is.

You could put this visualization in a dashboard, so by just applying a filter, you could check individual OLTs, or filter by country regions, states, network clusters, the possibilities are endless.

Lack of public data

It is necessary that I address the fact that it would be nearly impossible to find a public dataset that contains the kind of information that I would need to make these visualizations (and the reason should be obvious to you), so in order to overcome this difficulty I will make a synthetic dataset, that, in no way pretends to be a reflection of real data, nor it will have a statistical support whatsoever, and the reason for that should be also obvious for you (if I ever state that the average of the number of customers in a OLT is X, that could mean that I am projecting some inside and private knowledge that I have into the data), so I just will work with standard max quantities and KPIs for this technology.

Now with that out of the way we can start to build our synthetic dataset, and because this is a GPON visualization, I will make some synthetic OLT information with the following schema:

# Make all the imports needed
import pandas as pd
from itertools import product
from random import randint, choice
import matplotlib.pyplot as plt
import seaborn as sns


df = pd.DataFrame(data=[[1,2,3,'model_A'],[5,6,7,'model_B']], columns=[['slot', 'port', 'client_qty', 'model']])
df

Example of the data schema we want to create

The data can come in any form: csv, xlsx, etc., and it should consist of at least the columns showed in the previous dataframe: ‘slot’, ‘port’, ‘client_qty’, ‘model’, and hopefully each column name is self explanatory.

Now, to the creation.

I have decided to create synthetic OLTs with synthetic information, and to do that we will define a class named “OLT”, and then instantiate it several times. The next code cell will do the first part:

class OLT:
    """ Define a class OLT that have the following attributes:
        - An "int" Board quantity (slot_qty) which represent the maximun slot count that the OLT has.
        - An "int" Port quantity per slot (port_per_slot) which represent the maximun port count per slot that the OLT has.
        - An "int" Maximun client quantity per pon port (max_client_qty) which represent the maximun client count per port per slot that the OLT has. """

    def __init__(self, slot_qty=16, port_per_slot=16):
        self.slot_qty = slot_qty
        self.port_per_slot = port_per_slot

    # Then we define a method that will create and populate each pon port of the OLT instance with a client quantity
    def create_olt_and_populate(self, max_client_qty):
        # We assign the model of the OLT at random, during the creation
        models = ['Model_A', 'Model_B', 'Model_C', 'Model_D']
        model = choice(models)
        # We define a list as population, to add to it the information as tuples (slot, port, client_qty, model)
        population = []
        slot_port_tuples = [(slot, port) for slot, port in product(range(1,self.slot_qty+1), range(1,self.port_per_slot+1))]

        for tupla in slot_port_tuples:
            population.append(tupla + (randint(0,max_client_qty),model))

        # Then we create and return a Pandas DataFrame, in which each row represents "a PON port" with its information
        df = pd.DataFrame(population, columns=['slot', 'port', 'client_qty', 'model'])

        return df

We check that the class outputs the desired information by instantiate one OLT:

olt = OLT()
df = olt.create_olt_and_populate(max_client_qty=64)
df

Create the visualization

In order for this visualization to be more accurate with reality, we need create several OLTs instances, giving the impression that you have a lot of NEs in the network, so this time we create 150 OLTs:

olt_qty = 150
max_client_qty = 64
concat_df = pd.concat([OLT().create_olt_and_populate(max_client_qty) for olt in range(olt_qty)])
concat_df

That easily we created a synthetic dataset of 38.400 PON ports with random client quantity for each one of it.

Now the steps to create the visualization:

First, we need to group the data by “port”, “slot”, and “model”, and calculate the average of the client quantity at the same time using Pandas “groupby” method
Then, we create several groups of olts by filtering them by model.
We then create a second list, now of pivot tables, that allow us to create some sort of matrices in which each element represents a PON port (called “olt_list” in the code)
And finally we plot the “heatmap” to represent an OLT:

plt.rcParams.update({'font.size':14, 'figure.figsize': (25,20)})

grouped_olts = concat_df.groupby(['port', 'slot', 'model'])['client_qty'].mean().reset_index()
grouped_olts.sort_values(by='model', inplace=True)

models = list(concat_df['model'].unique())
olts = []
for model in models:
    o =  grouped_olts[grouped_olts['model']==model]
    olts.append(o)

olt_list = []
for olt in olts:
    individual = olt.pivot(index='port', columns='slot', values='client_qty')
    olt_list.append(individual)


fig, axes = plt.subplots(2, 2, figsize=(20,15))
for (i, axs, olt, modelo) in zip(range(4), axes.flatten(), olt_list, models):
    sns.heatmap(olt, vmin=0, vmax=64, cmap='RdYlGn_r', linewidths=0.5, annot=True, ax=axs)
    axs.set_title(modelo, fontsize=20)
fig.tight_layout(pad=2)
plt.show()

Visualization of all the models in the dataset

Closing comments

Is evident that the average value of the client quantity per PON port in this dataset obeys the “random.randint” function, but if you use it with your data it will surely reflect your networks KPIs.
This visualization could give you unexpected insights because normally we don’t plot this kind of KPI in this format, for example:
You could see if overall your NEs have good distributed loads of clients among all the boards.

You could make this visualization for every POP (point of presence) in your network, and check if particular POP is unbalanced,

And following the previous point, you could use this visualization even per OLT, and check if a particular OLT is very loaded, and triggering a hypothetical client migration.

This visualization can be generalized to represent networking routers, switches, optical network equipment, etc., as long as the information is treated as showed.
This visualization can also be about any KPI of the NE:
You could use the temperature of the PON port.

You could use the average/max/min PON port throughput or ocupation.

You could put this visualization in a dashboard, so by just applying a filter, you could check individual OLTs, or filter by country regions, states, network clusters, the possibilities are limitless.
I hope this was useful for you, and that you are looking forward for the next visualization!

You can check this code in a Jupyter Notebook in my Github:

aletelecom / GPON-Network-Visualization

GPON Network Visualization

This repository contains a collection of visualizations that represent the key performance indicators (KPIs) of GPON (Gigabit Passive Optical Network) and FTTx (Fiber to the x) networks.

Visualizations

1. OLT Heatmap

The OLT Heatmap is a visualization that resembles the front panel of an OLT (Optical Line Terminal) and allows you to represent multiple KPIs simultaneously. The heatmap is color-coded, with each color representing a different KPI. This visualization is useful for quickly identifying areas of the network that may require attention.

2. PON Port Occupancy Scatter Plot

The PON Port Occupancy Scatter Plot is a visualization that shows the relationship between the occupation of the PON (Passive Optical Network) port and the number of clients within the same PON port. This visualization can help identify congested areas of the network and can inform capacity planning decisions.

3. Average Client Speed Bubble Plot

The Average Client Speed…

View on GitHub

About that time I failed the “Foobar” Google (secret) coding challenge (why, and what I learned from it).

aletelecom — Sat, 04 Jan 2025 13:50:46 +0000

Early this year, like many other days during the past year and a half, I was “googling” for some answer to a question that would allow me to continue with the task I had at hand at the time, and the question was:

“How to interpret the AFC, and PAFC plots?”

As the usual Google interface to show the answers in a computer screen took place, another and different “animation” unfolded:

“A theatrical like, yet pixelated animation disrupting the search results page was happening, the screen split in two, and the results page gave its position to a black screen, with some green text on it”

For a few seconds my brain couldn’t process what I was looking at, I thought it was a new “Doodle” about Star Wars, or Star Trek, and I even opened a new tab, in order to continue with my search, but as I was clicking the “new tab” button, with the corner of my eye I distinguished an intriguing word from the rest:

“Challenge” — in green color, pixelated art, and console text font.

I immediately went back to the page, and read through the whole text, this art style, and space comic book narrative hit the geek in me as I was reading it, and in a sort of mixed reactions (mostly disbelief), I finished reading it, and could not assimilate it, you have to understand that I was deeply concentrated on what I was doing, and this sudden and quick turn of events was not in my radar, I stayed astonished for a few seconds.

As unfortunately as it is, I didn’t took a print screen of it, but the short narrative paraphrased went something like this:

“We know the kinds like you, because we are like you, people who love challenges, would you like to take a challenge from us? By doing so you would help the galactic bunnies get rid of their problems”

After the aforementioned seconds of disbelief of what I was reading was implying (that Google was giving ME a challenge) I quickly signed up for the challenge.

Part of the interface when you accept the challenge

Now, lets address the issues I wrote on the title…

Part of the disbelief I wrote about in the previous section comes from the fact that I am not a professional programmer, nor I have a background in it, I am an Electronic Engineer, with a major in Telecommunications, and I have been working in telco for the past 16 years.

So why was I looking for the “interpretation of AFC and PACF plots”?

Well, because finishing 2019 I decided to shift careers paths, I decided to enter the wonderful world of Data Science, and in 2021 I enrolled in a Masters Degree in Data Science, and as part of an assignment I needed to interpret the meaning of the aforementioned plots, and like for the past eight months (at the moment of receiving the challenge) I have been searching in Google for many topics related with Data Science, coding, algorithms, Python, and all the skills, tools, papers, and the like needed to get better at it.

The “why” part

I failed at the coding challenge.

I was not prepared, nor I had the background to tackle it at the moment that I received it, but I committed yet another mistake, I took it any ways, without further improving my coding skills. I should have tried and improve my coding skills before hand.

The “what I learned” part

Get ready for challenges. Unless you’re master in the area of expertise surrounding the challenge, you need to prepare, and if like me, you’re not a daily coder (or a professional at the topic at hand), you need to prepare even more, research, read, and above all, because we’re talking about coding, you need to code, and a lot, if you want to start to tackle this kind of challenges the proper way.

Closing comments

Receiving this challenge has been the most rewarding thing that have happened to me professionally in the last year, and I learned A LOT! about coding while doing it, and I won’t lie to you, I wish I could have that opportunity again.

I reached the third challenge of the second level, so I guess I didn’t do that bad, did I?

Part of the interface that shows your progression