DEV Community

SjxSubham
SjxSubham

Posted on

7 6 6 4 5

MyGithub scrap datas from Your Github account with a new format

This is a submission for the Bright Data Web Scraping Challenge: Scrape Data from Complex, Interactive Websites

What I Built

MyGithub: Explore GitHub in a Whole New Way!

Overview

MyGithub is a web application that brings the power of GitHub to your fingertips. Whether you’re a developer, a curious learner, or a seasoned open-source enthusiast, MyGithub offers a seamless experience for exploring user profiles, repositories, and more.

Features

1. Authentication & Authorization:

  • Secure login using Passport.js with GitHub Auth.
  • User sessions and tokens handled behind the scenes. ### 2. GitHub API Integration:
  • Fetch user profiles and repositories directly from GitHub.
  • Real-time data sync with GitHub.

3. Client-Side Repo Filtering:

  • Filter repositories right on the client side.
  • Find relevant repos faster.

4. Robust Error Handling:

  • Comprehensive error handling on both server and client sides.
  • No more mysterious bugs!

5. User Interaction:

  • Like profiles, sign up, and log in, see the users along with date who liked ur profile.
  • Clone repositories.
  • Explore popular repos by username.
  • Explore any users profile git usage, repos by language sorting options
  • profile linking option
  • Explore popular repos of Popular Languages

Demo

Check out MyGithub here: MyGithub App

How I Used Bright Data

So basically i have made a web scrapper for github profile scrapping on which we can get user repos as well as profile usage, repos by language along with other options for scrapping by using bright side organisations scrapping api token so write a ahort description for dev community post how i used their github profile scrapping API here is the bright side api link https://brightdata.com/cp/setting/users
How I Built a GitHub Profile Scraper Using BrightData API
I've recently developed a web scraper designed to extract comprehensive GitHub profiles, including user repositories, profile usage, and repository details categorized by language. To achieve this, I leveraged the powerful scraping capabilities provided by the BrightData API.

By using BrightData's GitHub profile scraping API token, I was able to seamlessly fetch user data and repositories. This integration allows for efficient and organized data collection, making it easier to analyze and present GitHub profiles.

The scraper supports multiple options for custom data extraction, ensuring flexibility and accuracy. This tool is particularly useful for developers and organizations looking to gather insights from GitHub profiles for analysis, reporting, or improving development strategies

Repo Link - https://github.com/SjxSubham/MyGithub

Billboard image

The fastest way to detect downtimes

Join Vercel, CrowdStrike, and thousands of other teams that trust Checkly to streamline monitoring.

Get started now

Top comments (0)

Billboard image

Try REST API Generation for Snowflake

DevOps for Private APIs. Automate the building, securing, and documenting of internal/private REST APIs with built-in enterprise security on bare-metal, VMs, or containers.

  • Auto-generated live APIs mapped from Snowflake database schema
  • Interactive Swagger API documentation
  • Scripting engine to customize your API
  • Built-in role-based access control

Learn more

👋 Kindness is contagious

Engage with a sea of insights in this enlightening article, highly esteemed within the encouraging DEV Community. Programmers of every skill level are invited to participate and enrich our shared knowledge.

A simple "thank you" can uplift someone's spirits. Express your appreciation in the comments section!

On DEV, sharing knowledge smooths our journey and strengthens our community bonds. Found this useful? A brief thank you to the author can mean a lot.

Okay