DEV Community: Arman Idrisi

Extracting Data from npmjs User Profile with Python and BeautifulSoup

Arman Idrisi — Sat, 10 Jun 2023 12:52:54 +0000

Introduction

Learn how to extract data from an npmjs user profile using Python and BeautifulSoup. This tutorial will guide you through the process of fetching and parsing HTML content to extract information such as the user's profile image, username, name, social links, total number of packages, and details of the latest packages published by the user.

Prerequisites

Before we begin, make sure you have the following:

Python installed on your machine
The BeautifulSoup library installed (pip install beautifulsoup4)
The requests library installed (pip install requests)

Getting Started

Let's start by importing the necessary libraries and defining a function to extract the text from HTML elements:

from bs4 import BeautifulSoup
import requests

def extract_text(element):
    return element.get_text().strip() if element else ''

Fetching the User Profile

The first step is to fetch the HTML content of the user's profile page. We will prompt the user to enter the username and construct the URL accordingly:

user = input("> Enter username: ")
url = f"https://www.npmjs.com/~{user}"
response = requests.get(url)
html = response.text

Parsing the HTML

Next, we need to parse the HTML content using BeautifulSoup:

soup = BeautifulSoup(html, "html.parser")

Extracting the User's Profile Image

Let's start by extracting the user's profile image URL. We can identify the relevant HTML element using CSS selectors and retrieve the src attribute:

img_element = soup.select_one("div._73a8e6f0 a img")
img = "https://npmjs.com" + img_element.get("src") if img_element else "NA"

Extracting the Username and Name

We can extract the username and name in a similar manner. Identify the respective HTML elements and extract their text content:

username_element = soup.select_one("h2.b219ea1a")
username = extract_text(username_element) if username_element else "NA"

name_element = soup.select_one("div._73a8e6f0 div.eaac77a6")
name = extract_text(name_element) if name_element else "NA"

Extracting Social Links

To extract the social links, we need to identify the relevant HTML elements and retrieve the href attribute of the associated <a> tags:

social_elements = soup.select("ul._07eda527 li._43cef18c a._00cd8e7e")
social = [e.get("href") for e in social_elements]

Extracting the Total Number of Packages

We can extract the total number of packages by identifying the corresponding HTML element and extracting its text content:

total_packages_element = soup.select_one("div#tabpanel-packages h2.f3f8c3f4 span.c5c8a11c")
total_packages = extract_text(total_packages_element) if total_packages_element else "NA"

Extracting Details of Latest Packages

Finally, we can extract the titles, descriptions, and

published information of the latest packages published by the user. We can iterate over the relevant HTML elements and extract the desired information:

package_elements = soup.select("div._0897331b ul._0897331b li._2309b204")
packages = []
for element in package_elements:
    title_element = element.select_one("h3.db7ee1ac")
    description_element = element.select_one("p._8fbbd57d")
    published_element = element.select_one("span._66c2abad")

    package = {
        'title': extract_text(title_element),
        'description': extract_text(description_element),
        'published': extract_text(published_element)
    }
    packages.append(package)

Creating the Data Dictionary

Finally, let's create a dictionary containing all the extracted data:

data = {
    'image': img,
    'username': username,
    'name': name,
    'social': social,
    'total_packages': total_packages,
    'latest_packages': packages
}

Printing the Extracted Data

To verify that the data extraction process is working correctly, we can print the data dictionary:

print(data)

Conclusion

In this tutorial, you learned how to extract data from an npmjs user profile using Python and BeautifulSoup. We covered the steps involved in fetching the HTML content, parsing it using BeautifulSoup, and extracting various pieces of information such as the user's profile image, username, name, social links, total number of packages, and details of the latest packages published by the user. This knowledge can be applied to similar scenarios where you need to scrape data from websites for analysis or other purposes.

I hope you found this tutorial helpful! If you have any questions or feedback, please leave a comment below. Happy coding!

You can find source code here

How to create a random quotes generator api in NodeJS

Arman Idrisi — Wed, 19 Apr 2023 07:49:06 +0000

How To Create Random Quote Generator API In Node.js Express

Are you interested in creating API's using Node.js and Express? If yes, then you've come to the right place! In this post, we'll walk you through the steps to create a simple yet powerful quote generator API.

Prerequisites

Before we dive into the code, let's take a quick look at what we'll be using:

Node.js - an open-source, cross-platform JavaScript runtime environment that allows developers to build fast and scalable network applications.
Express - a popular framework for building web applications in Node.js.
node-quotegen - a simple package that provides a collection of quotes that you can use to generate random quotes.

With these tools in hand, let's get started!

Step 1: Create a new Node.js project

To create a new Node.js project, open your terminal and run the following command:

mkdir quote-generator-api
cd quote-generator-api
npm init -y

Step 2: Install Express and node-quotegen

Next, let's install the required packages using the following command:

npm install express node-quotegen

Step 3: Create the quote generator API

Create a new file called app.js and add the following code:

const express = require("express");
const { getQuote } = require("node-quotegen");

const app = express();

// Define a GET route to generate random quote
app.get("/api/quote", (req, res) => {
  const quote = getQuote();
  res.json({ quote });
});

// Define a GET route to generate quote from a specific category
app.get("/api/quote/:category", (req, res) => {
  const quote = getQuote(req.params.category);
  res.json({ quote });
});

// Start the server
app.listen(3000, () => {
  console.log("Server listening on port 3000");
});

Let's take a look at what's going on here:

First, we import the required modules - Express and node-quotegen.

We create a new Express app and define two routes:

A GET route that generates a random quote using getQuote() function and sends it back as a JSON response.
A GET route that generates a quote from a specific category using the getQuote() function and sends it back as a JSON response.

Finally, we start the server and listen on port 3000.

Step 4: Test the API

To test the API, run the following command in your terminal:

node app.js

This will start the server, and you should see the message "Server listening on port 3000".

Next, open your web browser and navigate to http://localhost:3000/api/quote. You should see a JSON response with a random quote.

To generate a quote from a specific category, navigate to http://localhost:3000/api/quote/motivational (replace "motivational" with any other category). You should see a JSON response with a quote from that category.

And that's it! You've successfully created a random quote generator API using Node.js and Express. You can now use this API to generate quotes for your web or mobile applications.

Top 5 Ways to Deploy Static Sites for Free

Arman Idrisi — Sat, 15 Apr 2023 18:45:18 +0000

Top 5 Ways to Deploy Static Sites for Free

In this post, I will cover the top 5 ways to deploy static sites for free and also provide a brief explanation of static vs dynamic sites.

Static vs Dynamic Sites

Before diving into the top ways to deploy static sites, let's first define the difference between static and dynamic sites.

Static sites are web pages that are pre-built and served as-is to the user. They are typically made up of HTML, CSS, and JavaScript files, and do not require server-side processing. In other words, static sites do not interact with databases or generate content dynamically. Static sites are easy to build and maintain, load quickly, and are more secure since they do not have a backend.

On the other hand, dynamic sites are web pages that are generated on the fly, often using server-side scripting languages like PHP or Python. These sites are more complex and require a backend to handle database interactions, content generation, and other server-side processing. Dynamic sites are more flexible and can offer more features, but are also more complicated to build and maintain.

Top 5 Ways to Deploy Static Sites for Free

GitHub Pages: GitHub Pages is a free hosting service offered by GitHub that allows you to host static sites directly from a GitHub repository. Simply create a repository and add your HTML, CSS, and JavaScript files to it, and GitHub will automatically deploy your site. GitHub Pages also offers a custom domain feature, so you can use your own domain name.
Netlify: Netlify is a popular platform for hosting static sites. It offers a free plan that includes features like continuous deployment, custom domains, and SSL certificates. Netlify also offers a variety of integrations with popular static site generators like Hugo, Jekyll, and Gatsby.
Surge: Surge is a simple and fast platform for hosting static sites. It offers a free plan that includes custom domains, SSL certificates, and unlimited projects. Surge also has a command-line interface that makes deploying your site quick and easy.
Firebase Hosting: Firebase Hosting is a platform offered by Google that allows you to host static sites as well as dynamic sites built using Firebase. It offers a free plan that includes features like SSL certificates, custom domains, and automatic scaling. Firebase Hosting also integrates with other Firebase services like Authentication and Cloud Functions.
GitLab Pages: GitLab Pages is a free hosting service offered by GitLab that allows you to host static sites directly from a GitLab repository. Like GitHub Pages, simply create a repository and add your files to it, and GitLab will automatically deploy your site. GitLab Pages also offers custom domains and HTTPS support.

In conclusion, deploying a static site has never been easier thanks to the many free hosting options available. Each of the platforms listed above offers its own unique features and benefits, so choose the one that best fits your needs. And remember, static sites are a great option for simple projects that do not require server-side processing, and can be hosted for free with these platforms.

Step By Step Guide To Become Frontend Developer

Arman Idrisi — Thu, 13 Apr 2023 08:22:05 +0000

Step-by-Step Guide to Becoming a Frontend Developer

If you're interested in becoming a frontend developer, you're in luck! With the growing demand for web development, there are plenty of resources available to help you get started on this exciting career path. In this post, we'll walk you through a step-by-step guide to becoming a frontend developer, starting from scratch.

1. Learn the basics of HTML, CSS, and JavaScript

HTML, CSS, and JavaScript are the three essential building blocks of any website. To become a frontend developer, you'll need to have a solid understanding of these technologies. HTML is used to create the structure and content of a website, CSS is used for styling and layout, and JavaScript is used for dynamic functionality and interactivity. You can start learning these languages by taking online courses, reading tutorials, and practicing on your own.

2. Build your own projects

Once you've learned the basics of HTML, CSS, and JavaScript, start building your own projects. This is where you'll really start to solidify your knowledge and gain experience. Start with simple projects, like building a basic website or a form, and gradually work your way up to more complex projects. You can also contribute to open-source projects to gain experience and build your portfolio.

3. Learn a frontend framework

Frontend frameworks like React, Angular, and Vue.js have become increasingly popular in recent years. They make it easier to build complex web applications by providing a set of tools and pre-built components. Once you have a solid understanding of HTML, CSS, and JavaScript, start learning a frontend framework. React is a good place to start as it has a large community and plenty of resources available online.

4. Learn responsive design

Responsive design is essential in today's mobile-first world. It involves designing websites that can adapt to different screen sizes and devices. Learn how to use CSS media queries and other responsive design techniques to create websites that look great on any device.

5. Learn version control

Version control allows you to keep track of changes to your code and collaborate with other developers. Git is the most popular version control system and is widely used in the industry. Learn how to use Git and Github to manage your code and collaborate with others.

6. Learn a CSS framework

CSS frameworks like Bootstrap, Foundation, and Materialize provide pre-built CSS styles and components that you can use to speed up your development process. They also help ensure consistency and responsiveness across different devices. Learn how to use a CSS framework to streamline your development process and create professional-looking websites.

7. Learn a CSS preprocessor

CSS preprocessors like Sass and Less extend the functionality of CSS by providing features like variables, mixins, and functions. They make it easier to write and maintain CSS code and can save you a lot of time in the long run. Learn how to use a CSS preprocessor to enhance your CSS skills and improve your workflow.

Average salary of a front-end dev

Entry-level frontend developer: $50,000 - $70,000 USD per year
Mid-level frontend developer: $75,000 - $100,000 USD per year
Senior frontend developer: $100,000 - $130,000 USD per year
Lead frontend developer: $130,000 - $150,000+ USD per year

It's worth noting that salaries can vary based on factors such as location, company size, industry, and individual skills and experience.

Conclusion

Becoming a frontend developer requires a combination of technical skills, creativity, and dedication. By following these steps and putting in the time and effort, you can start your journey towards becoming a successful frontend developer. Remember to keep learning, practicing, and building your portfolio to showcase your skills to potential employers. Good luck!

Common mistakes and how to avoid them when learning to code

Arman Idrisi — Mon, 10 Apr 2023 16:08:42 +0000

Common Mistakes and How to Avoid Them When Learning to Code

Learning to code can be a rewarding experience, but it can also be frustrating and challenging at times. If you're new to coding, it's common to make mistakes along the way. In this article, we'll discuss some common mistakes that people make when learning to code and how to avoid them.

Mistake #1: Not Having a Plan

One of the biggest mistakes people make when learning to code is not having a plan. Without a clear plan, it's easy to become overwhelmed and confused. When starting your coding journey, it's important to set clear goals and objectives and create a learning plan that includes what to learn, how to learn it, and when to learn it.

How to avoid this mistake: Start by defining your goals and objectives. Do you want to build websites, develop mobile apps, or work on artificial intelligence? Once you have a clear idea of what you want to achieve, create a learning plan that includes the topics you need to learn, the resources you'll use, and the timeline for learning each topic.

Mistake #2: Trying to Learn Too Much Too Quickly

Another common mistake people make when learning to code is trying to learn too much too quickly. It's important to take your time and focus on mastering one concept at a time. Trying to learn too much too quickly can lead to burnout and frustration.

How to avoid this mistake: Start by focusing on the basics. Learn the fundamentals of coding before moving on to more complex topics. Practice coding every day and build your skills gradually. Don't try to learn everything at once.

Mistake #3: Not Practicing Enough

Practice is key when it comes to learning to code. Not practicing enough is a common mistake that can lead to slow progress. It's important to set aside time each day to practice coding, even if it's just for a few minutes.

How to avoid this mistake: Schedule regular practice sessions and stick to them. Set aside at least 30 minutes each day to practice coding. Build coding projects and practice coding challenges to improve your skills.

Mistake #4: Not Seeking Help When Needed

Another common mistake people make when learning to code is not seeking help when needed. It's okay to ask for help when you're stuck or don't understand something. There are many online resources, forums, and communities where you can get help and support.

How to avoid this mistake: Don't be afraid to ask for help when you need it. Join online communities and forums where you can ask questions and get help from experienced coders. Find a mentor or join a coding bootcamp to get personalized help and support.

Mistake #5: Not Taking Breaks

Learning to code can be intense, and it's important to take breaks to avoid burnout. Many beginners make the mistake of spending too much time coding without taking breaks, which can lead to frustration and fatigue.

How to avoid this mistake: Take regular breaks and give yourself time to rest and recharge. Take a walk, do some stretching exercises, or meditate to clear your mind and reduce stress.

Mistake #6: Not Using the Right Resources

Using the right resources is essential when learning to code. Many beginners make the mistake of using outdated or unreliable resources, which can lead to confusion and frustration.

How to avoid this mistake: Use reputable resources that are up-to-date and reliable. Look for online tutorials, coding courses, and books that are recommended by experienced coders. Join online communities and forums to get recommendations and advice from other coders.

Mistake #7: Not Building Projects

Finally, not building projects is another common mistake that people make when learning to code. It's important to apply what you've learned by building projects and practicing your skills.
How to avoid this mistake: Build coding projects to practice your skills and apply what you've learned. Start with simple projects and gradually build up to more complex ones. Create a portfolio of your projects to showcase your skills to potential employers.

The importance of version control and how to use Git for your projects

Arman Idrisi — Mon, 10 Apr 2023 15:54:38 +0000

What Is Version Control

Version control is an essential aspect of software development. It allows developers to keep track of changes made to code over time, collaborate with team members on projects, and easily revert back to previous versions if necessary. In this article, we will explore the importance of version control and how to use Git, one of the most popular version control systems, for your projects.

Why Is Version Control Important?

Version control helps developers maintain a history of changes made to code. This is important because it allows developers to track down bugs and understand how and when changes were made to the code. It also enables developers to work collaboratively on a project by allowing multiple people to work on the same codebase simultaneously.

Another significant advantage of version control is the ability to easily revert back to a previous version of code. This can be helpful in case of a bug or error introduced in a recent change or if you need to return to an earlier version of the code for any reason. Without version control, it would be difficult to roll back to a previous version of the code without manually undoing all of the changes made in subsequent versions.

How to Use Git for Your Projects

Git is a popular version control system that allows developers to easily track changes made to code over time. Here are the steps to getting started with Git for your projects:

Install Git: The first step is to install Git on your local machine. You can download Git from the official website and follow the installation instructions.
Create a Git Repository: Once Git is installed, the next step is to create a Git repository for your project. You can do this by navigating to the project directory on your local machine and running the following command in the terminal:

git init

This will initialize a new Git repository in the current directory.

Add Files to the Repository: The next step is to add the files in your project to the Git repository. You can do this by running the following command:

git add .

This will add all of the files in the current directory to the Git repository.

Commit Changes: After adding files to the Git repository, the next step is to commit changes. This is done by running the following command:

git commit -m "Commit message"

The commit message should be a brief description of the changes made in this commit.

Push Changes to Remote Repository: Finally, you can push your changes to a remote repository, such as GitHub or GitLab. This is done by running the following command:

git push origin <branch name>

This will push the changes made to the local repository to the remote repository.

Conclusion

Version control is an essential tool for software development, and Git is one of the most popular version control systems used by developers today. It allows developers to keep track of changes made to code over time, collaborate with team members on projects, and easily revert back to previous versions if necessary. By following the steps outlined in this article, you can get started with Git and begin using version control for your projects.

Python Project - How To Create A Quote Generator In Python

Arman Idrisi — Sat, 08 Apr 2023 18:56:28 +0000

Python is a versatile programming language that can be used to build a wide range of projects, from small scripts to large-scale applications. If you're new to Python, starting with a small project is a great way to get hands-on experience with the language and learn its core concepts. In this post, we'll explore the simple python project which can generate the random quotes.

Get Started

Installation

In Your Terminal

pip install pyquotegen

import module in your project

import pyquotegen

Code For Get A Single Quote

q=pyquotegen.get_quote()
print(q)

Code For Get The Quote By Category

pyquotegen.get_quote("motivational") #it will give you a random quote on motivational category

Creating A Full Project Which Can Take Category From Input And Gives The Quote.

#import module
import pyquotegen

#gets the category from user input 
c = input("Enter The Category: ")

#Get The Quote
q=pyquotegen.get_quote(c)
print(q)

Check More About Pyquotegen On Github Click Here

Please Star Us On Github

Thanks For Reading

Best Practices for Writing Clean Code

Arman Idrisi — Fri, 07 Apr 2023 15:05:48 +0000

Writing clean code is essential for any software developer. It not only makes your code more readable and maintainable but also helps in reducing the chance of errors and bugs. Here are some best practices that you can follow to write clean code:

Best Practices for Writing Clean Code

Writing clean code is essential for any software developer. Clean code not only makes it easier to maintain and debug your code, but it also makes your code more efficient and readable. In this post, we will discuss some best practices for writing clean code.

Use Descriptive Names

Using descriptive names for variables, functions, and classes can make your code more readable and understandable. Descriptive names can also help in reducing the amount of time you spend on debugging and maintenance. Avoid using single-letter names or abbreviations unless they are widely understood.

Keep Functions and Methods Small

Functions and methods should be small and do one thing. This helps in making your code more modular, easier to understand, and less error-prone. Aim for functions and methods that are no longer than 20 lines of code.

Comment Your Code

Comments can help in explaining the intent behind your code and can also make it easier to understand and maintain. However, over-commenting can also make your code cluttered and difficult to read. Aim for clear and concise comments that add value to your code.

Use White Space Effectively

Using white space effectively can make your code more readable and easier to follow. Use blank lines to separate logical sections of your code and to break up long blocks of code. Indent your code consistently and use proper spacing around operators and parentheses.

Write Unit Tests

Writing unit tests is a crucial aspect of writing clean code. Unit tests ensure that your code is working as expected and helps in identifying and fixing any issues before they reach production. By following a test-driven development approach, you can write your tests before you write your code, which can help you in writing cleaner and more efficient code.

Conclusion

In conclusion, writing clean code is essential for any software developer. By following these best practices, you can ensure that your code is not only efficient and readable, but also maintainable and easy to debug.