DEV Community: Zephaniah Joshua

Interaction Between FTP Server and client

Zephaniah Joshua — Sat, 09 Jan 2021 18:07:07 +0000

Photo by panumas nikhomkhai from Pexels

The client sends a Passive[PASV] commands to the server which invokes passive modes
The server responds with a 227 code with IP address and port of data socket

The client sends a retrieve[RETR] command plus, the name of the file to be retrieved
If the client is authorised to access the file the server sends a 150 code[request is successful and the file is about to be transferred]

Now to establish the connection to be used connection the client sends an SYC[synchronise] command
The server replies with an acknowledgement and so does the client

The server sends part of the file and the client acknowledges it and this continues until the whole file is received.

Thanks for reading please leave comments of suggestions or contact me @black_strok3

Detecting missing pixels in an Image

Zephaniah Joshua — Fri, 08 Jan 2021 14:03:21 +0000

Today we will be talking about finding missing data from the image we sent over the network. From our previous example of sending an image over a network, we are going to check if any pixels were lost during the transfer

First off we will try to map out an algorithm for detecting an error in the image file.

Receive a pixel
Store position of the pixel
Receive the next
Check if the pixels are in the correct order i.e pixel1, pixel2
If they are not then there is a missing pixel
Get the position of the missing pixel
Continue this 6 steps repeatedly until the image is sent
Send a request for missing pixels
Open image
And insert missing pixels into their position
Because we are transferring images on the same machine I induced an error by skipping any pixel with an odd Y value.

This is the code that sends the image

from socket import *
from pickle import *
from PIL import Image
import time

#create a socket

sockfd = socket(AF_INET, SOCK_DGRAM)

#open image
image = Image.open("100kb.png")

#get image size
width, height = image.size
print(image.size)

for x in range(width):
    for y in range(height):

        #get the position in the two dimensional array
        try:
            pos = (x, y)
            rgba = image.getpixel(pos)

            message = (pos, rgba)
            if y % 2 == 0:
                msg = dumps(message)
                sockfd.sendto(msg, ("127.0.0.1", 9916))
            #time.sleep(0.001)
        except:
            print(1)
message = "done"
msg = dumps(message)
sockfd.sendto(msg, ("127.0.0.1", 9916))

def send_missing_pixel():
    while True:
        msg, cli_addr = sockfd.recvfrom(1024)
        pos = loads(msg)
        if pos == "done":
            break
        rgba = image.getpixel(pos)
        data = (pos, rgba)
        data = dumps(data)
        sockfd.sendto(data, cli_addr)
        print("eggs")


send_missing_pixel()

The first image sent

Code to receive the image and ask for missing pixels

from socket import *
from pickle import *
from PIL import Image 
import time



#viewer = ImageViewer()
sockfd = socket(AF_INET, SOCK_DGRAM)
sockfd.bind(("127.0.0.1", 9916))

height = 170
width = 272
isize = (height, width)
#image = open("image2.png", "x")
image4 = Image.new("RGBA", size=(width, height))
pixels = image4.load()
#the following variables for storing positions of missing pixels
error_pos = []
pos1 = None
x = []
y = []

def get_pixel_data():
    j = 0
    x1 = []
    e_count = 0
    #erro counter
    error_count = 0
    while True:
        j = j + 1
        msg, cli_addr = sockfd.recvfrom(1024)
        #decoding 
        message = loads(msg)
        if message == "done":
            image4.save("image3.png")
            break
        pos = message[0]
        rgba = message[1]
        #print(pos)
        #setting pixels into image file
        pixels[pos[0], pos[1]] = rgba
        #stroing postitions to check if there is a missing pixel
        xy = (pos[0], pos[1]) 
        #print(xy)
        x1.append(xy)
        try:
            s = j - 1
            #print(x1[_s])
            if len(x1) > 1:
                #checking difference betwwen previous coordinates and present coordinates 
                xnums = x1[s][0] - x1[s-1][0]
                ynums = x1[s][1] - x1[s-1][1]
                #this will check if the difference Y returns a negative
                #that means a new X value and recalculate the Y value
                if ynums < 0:
                    ynums = height - x1[s-1][1] + x1[s][1]
                #iterate through the difference and create a list of missing pixel positions
                if ynums > 1:
                    for i in range(1, ynums):
                        if xnums > 1:

                            un = x1[s][0] + i
                            x.append(un)                        
                        if xnums <= 1:
                            e = x1[s][0]
                            x.append(e)
                        un = x1[s][1] + i
                        y.append(un)
                        _c = (x[e_count], y[e_count])
                        error_pos.append(_c)
                        e_count = e_count + 1
        except:
            pass

    q = len(error_pos)
    j1 = width * height
    #print(q , e_count)
    if e_count == q:
        #print(123)
        get_missing_data(cli_addr)


def get_missing_data(cli_addr):
    _x = len(error_pos)
image5 = Image.open("image3.png")
    for i in range(_x):
        e_pos = error_pos[i]
        msg = dumps(e_pos)
        sockfd.sendto(msg, cli_addr)
        #time.sleep(0.001)
        msg, cli_addr = sockfd.recvfrom(1024)
        #decoding 
        message = loads(msg)
        pos = message[0]
        rgba = message[1]
        pixels = image5.load()
        pixels[pos[0], pos[1]] = rgba
        print(pixels[pos[0], pos[1]])
    image5.save("image3.png")
    x = "done"
    msg = dumps(x)
    sockfd.sendto(msg, cli_addr)

get_pixel_data()

Halfway into correcting image

complete Image

Photo by Umberto on Unsplash

leave a comment, suggestion or contact me @black_strok3

Transfering Bitmap Images over a network

Zephaniah Joshua — Tue, 05 Jan 2021 12:46:38 +0000

Photo by Sora Shimazaki from Pexels

An image is made up of pixels(tiny dots of colours). Images can be stored digitally as bitmaps.

A bitmap (Bit-Map) is a mapping of a domain to bits. In our case, the domain is an array of pixels, here we map a single bit to a pixel.
Pix-map is a map of pixels which maps multiple bits to a pixel.

Bitmaps use the 24bit or 32bit colour depth and use the RGB(255, 255, 255) colour system each of the value in the RGB colour system is stored as a byte(8 bits) an extra 8bits might be used to show transparency of the image

R =8bits
G=8bits
B=8bits #24bit
A=8bits #32bit #extra transparency bit

Storing images in bitmap format will make them very large so images are mostly stored in compressed file format
.PNG:: the lossless compression. It compresses data by using pattern tin pixel colour to reduce the amount of data stored. readmore.
.jpg:: the lossy compression. It compresses data by getting rid o colours that the human eyes will not pick. read more.

Now we are going to write a program to send an image over a network. So we are going to send an image over a network using UDP and we will also use pickle for serialisation

Fist of we create a socket sockfd = socket(AF_INET, SOCK_DGRAM)
Open the image to be sent image = Image.open("image.png")
Get size of the image i.e width and heightwidth, height = image.size
Iterate through the whole image (the reason why we have two loops because the image is in a two-dimensional array form)
Get RGB value of every pixel in the imagergba = image.getpixel(pos)
Change it to a transferable form using “dump” from the pixel module msg = dumps(message)
Sending it sockfd.sendto(msg, ("127.0.0.1", 9999))

image we are sending

from socket import *
from pickle import *
from PIL import Image
import time

#create a socket

sockfd = socket(AF_INET, SOCK_DGRAM)

#open image
image = Image.open("100kb.png")

#get image size
width, height = image.size
print(image.size)

for x in range(width):
    for y in range(height):

        #get the position in the two dimensional array
        try:
            pos = (x, y)
            rgba = image.getpixel(pos)

            message = (pos, rgba)
            msg = dumps(message)
            sockfd.sendto(msg, ("127.0.0.1", 9999))
            time.sleep(0.001)
        except:
            print(1)#just a check

Now to receive the image…
Create a socket
Bind socket
Create an image file to save data sent image4 = Image.new("RGBA", size=(width, height))
Create a function to receive data and write it into the image file

Here I already knew the width and height but we can also send that over the network before we start sending the pixel data

from socket import *
from pickle import *
from PIL import Image 



#viewer = ImageViewer()
sockfd = socket(AF_INET, SOCK_DGRAM)
sockfd.bind(("127.0.0.1", 9999))

height = 170
width = 272
isize = (height, width)
#image = open("image2.png", "x")
image4 = Image.new("RGBA", size=(width, height))
pixels = image4.load()

def get_pixel_data():
    while True:
        msg, cli_addr = sockfd.recvfrom(1024)
        #decoding 
        message = loads(msg)
        pos = message[0]
        rgba = message[1]
        print(pos)
        pixels[pos[0], pos[1]] = rgba
        #print(pixels[pos[0], pos[1]])
        if pos[0] + 1 == width and pos[1] + 1 == height:    
            image4.save("image4.png")
            break
#calling my function
get_pixel_data()

image we received

This was fun to do hope you learnt from my write up, please leave a comment or contact me @black_stroke

Serialisation

Zephaniah Joshua — Mon, 04 Jan 2021 08:58:59 +0000

Previously we talked about parsing data so it could be sent over a network and we used an example code in python. parsing is a simple form of serialisation.

Serialisation is converting data into a stream of bytes so that it can be stored or transmitted.

Remember our example information that was to be sent over the internet that has two data types an integer and a string

data1 = “hello” #string
data2 = 123  #interger

Now we can use pickle a serialisation module in python(FYI: marshal, JSON are other python serialisation modules ) to convert this data to bytes and send them over the network.
NOTE: any data type can be serialised with pickle

import pickle  

data1 = “hello” #string
data2 = 123  #interger

#creating our message as a tuple
message = (data1, data2)

#converting data to a stream of bytes with pickle
msg = pickle.dumps(message)
print(msg)

When this message is received it needs to be deserialised

import pickle 

#deserialising
message = pickle.loads(msg)


data1 = message[0]
data2 = message[1]

print(data1)
print(data2)

Warning as stated in the pickle documentation deserialisation(unpickling) can be exploited by crafting data to cause arbitrary code execution.

Hope this gives you a basic understanding on serialisation and how to use the pickle python module to do it. Please do leave suggestions or questions in the comment box or contact me @black_strok3

Photo by Neo from Pexels

Parsing Data

Zephaniah Joshua — Sun, 03 Jan 2021 13:48:45 +0000

Yesterday I wrote about the difference between UDP and TCP. Today we will be talking about sending information that contains different data types over a network by parsing.

A string of data can easily be sent over a network because it can easily be encoded as bytes but the same cannot be said for a large amount of data that contains different data types e.g strings, integers, float, tuples, dictionaries etc.

For example, if we have data that contains a string, date, a tuple as information we need to send over a network we might need to encompass all this data into a single message convert to bytes and send over the network. On receiving the data the server will separate the data back into its individual components and decode it.
The process of separating data back to its components is called parsing.

First step is converting other data types to string e.g Converting an integer to a string str(123)
Encompassing converted data into a single variable with a separator like “,” to separate the different data.

data1 = “hello”
data2 = 123

#convert the integer to string
data2 = str(data2)
#encompassing data into single variable
message = data1 + “,” data2

Encode the message message.encode() and send socket.send(msg).

The server has received the message and will now parse it

#recieve and decode message
message = socket.recv(1024)
msg = message.decode()

#split the message using the split function in python
#remember that we used “, ” to separate data
parts = msg.split(“,”)
#data1 is a string
data1 = str(parts[0])
#data2 is an int
data2 = int(parts[1])

So basically parsing is changing all data to be sent into a uniform data type(in our case a string) sending it as a single message and on reception separating it back to its components and returning them to there original data types.

Hopes this gives you a basic understanding of parsing. Feel free to ask questions or leave suggestions in the comment section or contact me @black_strok3.

image

Difference between UDP and TCP (example code)

Zephaniah Joshua — Sat, 02 Jan 2021 16:50:30 +0000

Hi everybody this is my first post. I started the 100DaysofCodechallenge so I decided to write about what I learn each day.

Today I learnt about the difference between UDP and TCP and used it in the client/server model python code.

UDP(User datagram protocol):

In simple terms UDP is a very fast and not-so-reliable communications protocol and does not need to make connections before sending data.

TCP(Transfer control protocol):

On the other hand TCP makes connections before data is sent, is a reliable transmissions protocol, not as fast UDP.

I will use a simple python server code to show the differences

-A simple server using TCP connection

from socket import * 

#create socket
sockfd = socket(AF_INET, SOCK_STREAM)
#bind socket
sockfd.bind(("127.0.0.1", 9999))
#listen for connections
sockfd.listen()

#accept connection
newsockfd, address = sockfd.accept() 

#recieve message
response = newsockfd.recv(1024)

#send message
message="this message is for you"
newsockfd.send(msg)

-A simple server using UDP connection

#create socket
sockfd = socket(AF_INET, SOCK_DGRAM)

#bind socket
sockfd.bind(("127.0.0.1", 9999))

#recv message
data, client_addr = sockfd.recvfrom(1024)

#send message
message = "yo, yo"
msg = message.encode()
sockfd.sendto(msg, client_addr)

We can clearly see that the UDP code is shorter than the TCP.This is because:

1.TCP connection listen sockfd.listen()for connections on it socket while UDP just doesn't do that(if data is sent and server is not up it is simply lost)

2.TCP accepts connectionnewsockfd, address = sockfd.accept()from the client before exchanging information while UDP just accepts information and stores data, client_addr = sockfd.recvfrom(1024)the IP address from which it received information

3.The receiving function. TCP uses recv(1024) to recv data "1024" specify the amount of data
while UDP uses recvfrom(1024)basically because the address from which the data was received also needs to be store

4.The send function. TCP uses the send() because it does not need to specify the address to which the data needs to be sent
because a connection has already been established between client and server
while the UDP uses sendto()which needs to sppecify the address to which data will be sent.

FTP, SSH, email use TCP.
Streaming media, online gaming use UDP.

Hope this little information gives you a basic understanding of TCP and UDP feel free to live your thoughts in the comments or contact me at black_strok3