DEV Community: Shoeb Ahmed

HireLink Contacts 10 December 2025 Embedded System(Electronics)

Shoeb Ahmed — Thu, 11 Dec 2025 06:12:39 +0000

please click on this and buy this free of cost: https://shoeb370.gumroad.com/l/gjwcr

Embedded System curated HR list 5 December 2025

Shoeb Ahmed — Wed, 10 Dec 2025 05:55:17 +0000

visit to download: --> https://shoeb370.gumroad.com/l/bgefy

Run Python code on AWS Batch.

Shoeb Ahmed — Fri, 12 Aug 2022 09:23:48 +0000

Run Python code on AWS Batch.

I have published 3 articles on how to run our basic python code on AWS batch, it is a step-by-step approach to how to create python code and then how to run our python code on Docker, and how to upload containerized solution on AWS Repository.

After that, how to run the containerize solution on AWS batch.

And here is the flow Diagram of AWS Batch

Run a Python code on AWS Batch Part — 3: Creation of Job Queues, Job definitions and Run Jobs.

Shoeb Ahmed — Fri, 12 Aug 2022 09:22:49 +0000

Run a Python code on AWS Batch Part — 3: Creation of Job Queues, Job definitions and Run Jobs.

In the previous articles, we saw how to create a container and upload the container to AWS repositories. if you didn’t visit that article please click on the below name:

In this session, we are going to run the python code:

Create a Job Queues
Create a Job definitions
Create a Jobs
Run a Jobs

Let’s start

Create a Job Queues

First, click on the Job Queues on AWS Dashboard and then click on “Create”.

And I am going to give the name in the Job queue name “test-queue-batch-v1” and you can give the name of your own interest. The priority I am going to give is 1000 because Job queues with a higher integer value for priority are given preference for computing environments.

I am going to select the Compute Environment which is necessary that the container will run this Compute Environment. In my case, I have given the Compute Environment name as “demo-batch-python-v1”. Please click on the radio button, make sure the radio button is in blue colour.

And click on create and we will see a “VALID” message on the status. And green colour banner will come on the top side of the page.

Create a Job definitions

If we see the left-hand side of the navigation bar of AWS Batch Dashboard we see the Job definitions click on that and click on the “Create” button just like we did on Job queues.

After clicking the “Create” button, it will enter the form in which we need to fill in certain parameters for job definitions. And I am taking the name “test-job-def-v1” and I am taking the Execution timeout of the code near about 1 hour, if our code execution time will go beyond 1 hour it will terminate the execution of the code.

Now we have to select EC2 for platform compatibility or you can choose Fargate if you are familiar with that.

After that, we are going to give you your Docker Container location where you kept it. You can see that in the below image I stored my Container in sider Amazon ECR and copy that public ECR which is a green colour mark in that image

And paste that link inside the image field.

After setting up all the job configurations, and then click on create button.

Now our next task is to run that Image file i.e. Docker Container on AWS Batch by creating a job.

Now click on Jobs **on the left-hand side of the navigation bar. And now click on **Submit new job

After clicking on Submit new job, we will be redirected to the form page and we have to fill in some detail step by step.
Inside the **Name **field, we can add job name of our own, and we already created the job definition and job queue.

If we select the **job definition **all the fields are automatically filled by **AWS **itself.

And same for **job configuration **is filled automatically, we just need to verify.

After that click on submit. And after that go to Dashboard and we can see our task is submitted.

We can see different columns in the above images which show the status of our AWS batch jobs or we can say that different stages of the Batch jobs.

Stages ranges from Submitted -> Runnable -> Starting ->Running -> Succeeded/Failed.

All the stages are completed automatically, we can observe the output in Succeeded/ Failed columns.

In my scenario, my job was executed properly and I can see the result by clicking on the number which is **“2”, **I had run that same job previously, that’s why it is showing 2 otherwise it will show 1, and if your job will fail, it will show 1 in Failed columns.

In the above image in my scenario, I will click on 2 and it will be redirected to the list of jobs which is succeeded.

I am going to click on **“my-job-v1”. **Because we had run the job right now.

And it will move to the Job Information **page. And we see the **Log stream name, **below that there is link is provided please click on that it contains the output. That links will get to the **AWS Cloudwatch.

What is AWS Cloudwatch? In my words, I can say that it is used to view the output of any services like AWS Batch, AWS Lambda etc.

And you can get more detail by just clicking on **AWS Cloudwatch.**

And we can see the above Image, i.e. the output of the jobs.

And we have successfully run the AWS Batch.

Run a Python code on AWS Batch Part — 2: Uploading Data to ECR and Creation of Computing Environment.

Shoeb Ahmed — Tue, 19 Jul 2022 07:25:41 +0000

Run a Python code on AWS Batch Part — 2: Uploading Data to ECR and Creation of Computing Environment.

In the previous article, I posted that how can we create and run a simple python script. And how to create a simple Docker File and Docker container that runs on a local system.

In this article, we are going for the AWS Batch side. For running an AWS Batch we require an Image File (Docker Container) and Computing Environment.

Create a repository in “Amazon Elastic Container Registry”
Push the Docker Container into the repository.
Creating a “Compute Environment” in the AWS batch.

For running a Docker Container on AWS Batch we need to store it on the AWS platform, we are going to use **“Amazon Elastic Container Registry”, **where we can store our Docker Container. Search **Amazon Elastic Container Registry **and we see the dashboard of ECR.

Click “Get Started”. After that, we will visit to create a repository page.

I am going to use public because we are in the learning phase.

And give the name of the repository which is a compulsory field.

I am selecting all **“Content types” **and you can choose your content types.

And then I am going to click **“Create repository”. **After that our page will redirect to the repository list their we can see our new repository list.

Now we click on our repository name and after that, we will click on “View push commands”.

After clicking on that button we will see a list of push commands we need to follow all the commands.

So, I am going to execute the command line by line in the command prompt.

Make sure that your current working directory is should be where you saved your python script and Docker file also.

Now we first going to copy the first command.

And we are getting the “Login Succeeded”. If you are not getting login succeeded please try to configure the AWS CLI again.

Now I am going for the second command.

Third command.

Now the last command will push your image into repositories.

We will see in the above screenshot that there is an image whose name is ‘latest’.

Now we will move into AWS Batch, first, we will create “Compute Environments”.

Click on compute environment on the left-hand side of the AWS Batch dashboard.

Click on “Create” on the right-hand side of Compute Environments.

Fill in the details given below:

And in the Instance configuration, I am using Spot.

For **Networking, **I am using the same VPC Id which I had used in AWS Redshift and the link is here: https://medium.com/codex/aws-redshift-connects-with-python-part-1-setup-a-redshift-connection-with-python-b9f6a1fa49f0

After that click on Create Compute Environment. And we will see the status of the computing environment as shown in the image which is valid.

We will see the creation of Job Queue and Job Definition in the next article which is part 3.

Run a Python code on AWS Batch Part — 1: Creation of Python Script and Docker Container.

Shoeb Ahmed — Tue, 19 Jul 2022 07:24:53 +0000

Run a Python code on AWS Batch Part — 1: Creation of Python Script and Docker Container.

A prerequisite for running python code on AWS Batch is that you already have Python and Docker installed on your system. No matter whether you know Docker or not, our goal is to run the python code in AWS Batch.

We will start with running the basic python code and that will print messages.

Steps are given below:

Create a python script.
Create a Docker File.
Create Docker Container.
Run Docker Container.

First, start writing the python script.

The output of the above code looks like the one given below here.

The next step is to create a Docker File. In that docker file, we need to insert a particular command which I had given below here. Make sure that the file we are creating is the name should be “Dockerfile”. *And the python script which I had made the filename I have taken is *“myfunction.py”.

The command for Docker is given below:

If you have external packages just like I have taken pandas, same way you can also add the command line which is mentioned on line number 5 in the above code.

RUN pip install pandas numpy more-packages-you-can-add-with-whitespace-seperated

Till now we had created a Docker file and Python script, our next goal is to create a Docker Container.

Before creating we need to set up our AWS CLI because we need to upload the Docker file and Python script on the AWS platform.

Make sure you had already installed AWS CLI from these links and click on that links: https://awscli.amazonaws.com/AWSCLIV2.msi

Now we will configure the AWS CLI by running a certain command.

Make sure you have an AWS ACCESS KEY ID and AWS SECRET KEY by visiting the Security Credential page.

If you want to generate that key please visit my old article: https://medium.com/codex/aws-s3-with-python-by-shoeb-ahmed-923c1cf47110

After getting the key let’s jump onto the AWS CLI and Open the command prompt.

Open CMD
Type “aws” and it will show some message as shown below that means AWS CLI is installed on your system:

Now type “aws configure” on the command prompt:

Enter your access key and press Enter button.
Enter your secret key and press Enter button.
After that, it will ask for the region and default output format we need to keep as it is blank and presses Enter.

Once AWS CLI is done. Now we move to the creation of a container.

Make sure the docker file and python file are in the same locations.

I open cmd on that location and type:

docker builds -t sample-aws-code .

Please give whitespace and dot after “sample-aws-code”.

In the above screenshot, we saw that our Image file is built.

We will test that image file in the local system by running the command:

docker run sample-aws-code

So this is the output of the Sample code.

I will publish part 2 which consists of AWS Batch setup and run.

AWS Redshift connects with Python Part 2 — Python code to connect AWS Redshift

Shoeb Ahmed — Tue, 05 Jul 2022 12:55:22 +0000

AWS Redshift connects with Python Part 2 — Python code to connect AWS Redshift

In the Part 1 session, we set up the AWS Redshift account to connect to the outer world.

In this session, I will demonstrate some of the code that will be set up and connected to a python world to AWS Redshift.

And my main goal is to upload a pandas DataFrame in AWS Redshift.

Before going to the coding part let's bump into the Query Editor V2.

First logged in AWS Redshift Session or we can go to Redshift Dashboard.

After visiting the Dashboard page we will see the list of clusters on the CLuster Overview Session.

Now we see that I have 2 clusters created one is active which is “Available” and another one is inactive which is “Paused”. I am going to use redshift-cluster-3.

I will also show you how to activate or we can say how to make redshift-cluster-2 Available.

Click “redshift-cluster-2” name or your cluster name, and it will redirect to the cluster page, click on “Action”, and it will show a drop-down menu.

Inside the menu, you will see “Resume”, please click on that. After clicking it takes some more minutes to activate the cluster. So, you will see the status of the cluster on the AWS Redshift cluster dashboard and the message will be Resume Modifying and after some, it will become Available.

Once our cluster is “Available”, now, let's start working with python code.

We are going to install a redshift connector by running the command given below.

pip install redshift-connector

For connecting AWS Redshift via python we require certain credentials like Hostname, Database Name, Port Number, User and Password.

If we are inside our cluster side, we go to Properties, we will see the credentials we are looking for.

Now we are going to run the query in python to create a table on Redshift.

Before that let's create a sample DataFrames

Now we are going to create a table inside the AWS Redshift cluster.

We will see the table is created.

Now we are going to insert DataFrame inside the table.

Now in the query editor, we will see that our table is created and filled with data inside the table.

This whole sample demonstrates how we can create an AWS Redshift cluster and how can I insert pandas DataFrames.

We can upload n number of Data inside DataFrame in Redshift cluster.

Web Scraping the Phillips India website — Scraping the Headphone Category.

Shoeb Ahmed — Mon, 04 Jul 2022 11:15:07 +0000

Web Scraping the Phillips India website — Scraping the Headphone Category.

Here I will scrape all the content on the Phillips website. And that content will be the Headphones category. The website is Phillips's India website.

Usually, web scraping is part of the Data Science/ Data Analyst job, which is the Data Generation or Data Aggregator.

Here we are going step by step approach, to how we do scraping the website, and the technology we are going to use is, Python, Selenium and Beautiful Soup.

First thing is that we are going to install some packages, which we type the certain command on cmd or anaconda prompt.

pip install lxml
pip install selenium
pip install beautifulsoup4
pip install pandas
pip install webdriver-manager
pip install regex

First, we are going to import the packages we required.

What I will do, I visit directly to that website which contains all the headphones and which contain all the types of headphones. So that is easy for us to try not to do much to automate the website.

I have mentioned below that how our website looks and you can visit to have look at how this website looks for you. https://www.philips.co.in/c-m-so/headphones/latest#

First, what we do will scrape all the data present inside the small container or we can say a small box. I am going to extract the Model number, Category name, and Overall Ratings.

Line number 19 on the above code is to collect the links of the product. By collecting the links of the product to visit the product website and to collect the data like technical specifications in detailed format.

Now finally we scrape all the data, and end result is that we are going to save the data in CSV format.

Inside data look like the above image.

That file will be saved as “Final_Headphones.csv”

The entire code is shown below:

That’s all about scraping the data.

Thank you.

AWS Redshift connects with Python Part 1 — Setup a Redshift connection with Python.

Shoeb Ahmed — Mon, 04 Jul 2022 11:13:41 +0000

AWS Redshift connects with Python Part 1 — Setup a Redshift connection with Python.

Welcome to the Amazon Redshift Cluster Management Guide. Amazon Redshift is a fully managed, petabyte-scale data warehouse service in the cloud. You can start with just a few hundred gigabytes of data and scale to a petabyte or more.

You can get more details on the websites: https://docs.aws.amazon.com/redshift/latest/mgmt/welcome.html

We are going to set up from the AWS side by step by step approach.

I hope you guys have created the AWS account, we logged in to the AWS Redshift account. The page looks like shown below here.

Above page which is the AWS Redshift page, on that page their is an option called “Create cluster”. We have to click on it and proceed to further.

After clicking on “Create Cluster” we need to give the name Cluster identifier, I have given “redshift-cluster-v1” and after that clicking on the Free trial option. The red colour box I have shown above is on the right-hand side.

After that, we scroll down to see that we have to give the Database name, port number and password. But I chose to keep the default as it is. But we have to give our password.

After entering the Admin user name and password, click on Create cluster.

After clicking “Create cluster” the page will automatically navigate to the cluster homepage and it will show the status of your cluster and it takes time.

The above image shows the status of the cluster, and the status is “Modifying Creating”.

After the cluster has been created, the status will show “Available”.

Now our cluster is created with some demo data inside the databases.

After that, we have to make changes in the VPC Security group to communicate between Python and redshift.

First, we must click our redshift cluster name, go down and click on the “Properties” tab.

Inside the “Properties” tab we will see, “Network and Security settings”, click on the links which are present below “VPC security group” which is in the blue box mentioned in the below diagram.

After clicking that blue link we will redirect to the Security Group page. And we have to search for “Inbound rules”

And Inbound rules are present at bottom of the screen, the green box which I had shown in the above image.

After clicking the “Inbound rules” we will see the list of various inbound rules and we click on “Edit Inbound rules”. And we will change some of the properties.

After that click on the “Add rules”, which will prompt new data and we need to modify it.

We need to modify the fields mentioned below Image and which are inside the green box.

By setting the VPC security group, we will connect AWS Redshift to python.

Now, only one thing remaining is that we have to make our cluster publicly accessible. So, we come to cluster home and click on the check box of the cluster.

After done selecting your choice of the cluster, then go to Action, locate Modify publicly accessible setting. A prompt window will arise and we have to make it enable.

Click “Enable” and save changes. Now our Redshift is ready for python.

I will publish the code of python in the next article part 2.

My Coding Vlog — Day 1

Shoeb Ahmed — Thu, 16 Jun 2022 07:20:54 +0000

My Coding Vlog — Day 1

Here I am going to share my experience as a coding vlog.

https://youtu.be/BexDAEtjKNg

I have shared a youtube link which contains the code output.

That video or I can say that I have again started with the basics of python and the famous books Automate the Boring Stuff with Python: Practical Programming for Total Beginners
my_coding_vlog/day_01.ipynb at main · shoeb370/my_coding_vlog

AWS S3 with python by Shoeb Ahmed

Shoeb Ahmed — Thu, 16 Jun 2022 07:18:59 +0000

AWS S3 with python by Shoeb Ahmed

Amazon Web Services is an online platform that provides scalable and cost-effective cloud computing solutions.

Here we are going to explore all the methods (functions) present in boto3 packages, and we are going to S3 bucket resources.

And I am going to skip lots of theoretical concepts and jump step by step.

Make sure we have an AWS account and a root user.

Once you logged in, we just need an Access key to get a connection from AWS and python.

What we need to do, is go to the home console and see on the top right corner, click on your username as shown below:

After clicking the drop-down click on security credentials.

After clicking on security credentials, we will redirect to the security page, and click on the Access Keys.

After clicking the Access keys, click on “Create New Access Key”, and a pop-up window comes up, and click on “Download key files”.

Excel sheet file will be downloaded, what we have to do is that we are going to copy the AWS Access Key ID and AWS Secret Key ID.

My goal is to explore all the functions (methods) in S3 Bucket.

Let’s start coding.

And though I am going to do any task with Pandas DataFrame only. If you want any other file format, please go to my profile and find my email id and message me or write a comment.

After running the code you will observe that your new bucket is created in S3.

Now what we are going to do, we are going to upload the csv file in S3 bucket.

First we will create a simple DataFrame

Now we are going to upload a CSV file which is stored in the local system, without creating a folder inside the bucket.

Now we are going to create a folder inside the bucket, inside that folder we are going to upload the code.

If you have a bytes-like file (i.e. binary file) or stream file so we can use uploadfileobj functions.

We can also upload by using the PutObject request, you must have the s3:PutObject in your IAM permissions.

Getting NSE data using python.

Shoeb Ahmed — Tue, 22 Mar 2022 17:20:59 +0000

Getting NSE data using python.

If anyone like coding as a hobby or they like to explore more using code, python is the best example from my point of view.

I am going to demonstrate how can we extract simple stock market data using python. So, we are going to extract the NSE (National Stock Exchange of India Ltd). So this is just a sample code of how can you use it in your application or you can explore more from this blog.

pip install nsepy

Now I am going to extract the share of “ICICI Bank” data of 30 days ago of the current date.

After running the above code you will get the data in CSV format as shown below but the date and data will be variant.

You can explore more on this topic on: https://nsepy.xyz/

Any queries drop me a question on my LinkedIn Account page.
Shoeb Ahmed - Trainee Software Engineer - Aress Software | LinkedIn

Now for option and Bank Nifty

stock_opt = get_history(symbol="SBIN",
                        start=date(2015,1,1),
                        end=date(2015,1,10),
                        option_type="CE",
                        strike_price=300,
                        expiry_date=date(2015,1,29))

nifty_next50 = get_history(symbol="NIFTY NEXT 50",
                            start=date(2015,1,1),
                            end=date(2015,1,10),
                            index=True)

Bank_Nifty = get_history(symbol="BANKNIFTY",
                            start=date(2015,1,1),
                            end=date(2015,1,10),
                            index=True)