DEV Community πŸ‘©β€πŸ’»πŸ‘¨β€πŸ’»

Cover image for 🌍 Where Are DEV Users Coming From?
Boris Jamot ✊ /
Boris Jamot ✊ /

Posted on • Updated on

🌍 Where Are DEV Users Coming From?

As you may already know, dev.to has an API to fetch articles and users.

Let's give it a try with my favorite HTTP client, httpie:

http https://dev.to/api/users/1
Enter fullscreen mode Exit fullscreen mode
{
    "github_username": "benhalpern",
    "id": 1,
    "joined_at": "Dec 27, 2015",
    "location": "Brooklyn, NY",
    "name": "Ben Halpern",
    "profile_image": "https://res.cloudinary.com/practicaldev/image/fetch/s--DOU9qJSH--/c_fill,f_auto,fl_progressive,h_320,q_auto,w_320/https://thepracticaldev.s3.amazonaws.com/uploads/user/profile_image/1/f451a206-11c8-4e3d-8936-143d0a7e65bb.png",
    "summary": "A Canadian software developer who thinks he’s funny.",
    "twitter_username": "bendhalpern",
    "type_of": "user",
    "username": "ben",
    "website_url": "http://benhalpern.com"
}
Enter fullscreen mode Exit fullscreen mode

What I first noticed is that id seems to be auto-incremented.
After some tries, I quickly found out the total number of users on dev.to: around 119000.
It gives me the idea to scrape all the users public data and try to play with it.

Let's try with this command:

for i in {1..119000}; do http https://dev.to/api/users/$i >> users.dev.to.json && echo "," >> users.dev.to.json && echo "https://dev.to/api/users/$i"; done
Enter fullscreen mode Exit fullscreen mode

After a few hours running in the background, I finally got all the users in my JSON file.

I ended up by importing the JSON file in a mongodb collection using NoSQLBooster.

There isn't many information in the users' profile, but two of them worth having a look at them: joined_at and location. It could be very interesting to have a diagram showing the progression of registrations through time. Maybe I'll do it some day, but for now, let's compute some statistics on users' location.

I used two queries to compute the statistics. This one helped me to get the big trends in people's location:

db.getCollection("users.dev.to").aggregate(
    {"$group": {_id: "$location", count:{$sum:1}}}
).sort("-count")
Enter fullscreen mode Exit fullscreen mode

Then, I use another query to get the count of people per location:

db.getCollection("users.dev.to").find({"location": /\bdublin\b/i})
Enter fullscreen mode Exit fullscreen mode

The conclusions are the following:

Most of the users (90%) don't fill their location.
16.3% of the users who filled their location are from USA.
5.8% are from India.
4.9% are from UK.
4% are from Japan.
3.8% are from Germany.

Let's have a look at the full statistics below:

# country region or city #region or city #country %
1 USA 1850 16.32%
Incl New York 205
Incl CA 194
Incl NY 147
Incl San Francisco 137
Incl TX 119
Incl Seattle 117
Incl Los Angeles 107
Incl Chicago 101
Incl Washington 81
Incl FL 78
Incl PA 68
Incl MA 67
Incl AZ 38
Incl MI 34
Incl OH 30
2 India 660 5.82%
Incl Bangalore 107
Incl Delhi 103
3 UK 551 4.86%
Incl London 258
Incl England 35
Incl Scotland 33
Incl Wales 7
4 Japan 451 3.98%
Incl Tokyo 242
5 Germany 428 3.77%
Incl Berlin 128
6 Brazil 301 2.65%
7 France 294 2.59%
Incl Paris 98
8 Canada 257 2.27%
Incl Toronto 107
9 Nigeria 187 1.65%
Incl Lagos 97
10 Netherlands 154 1.36%
Incl Amsterdam 62
11 Spain 110 0.97%
12 Argentina 109 0.96%
13 Italy 102 0.90%
14 Indonesia 92 0.81%
15 Russia 89 0.78%
16 Mexico 86 0.76%
17 Philippines 85 0.75%
18 Poland 81 0.71%
19 Ukraine 75 0.66%
20 Portugal 63 0.56%
21 South Africa 57 0.50%
22 Pakistan 56 0.49%
23 Turkey 55 0.49%
24 Belgium 53 0.47%
25 Egypt 52 0.46%
26 Bangladesh 50 0.44%
27 Colombia 46 0.41%
28 Romania 45 0.40%
29 Kenya 45 0.40%
30 Switzerland 45 0.40%
31 Austria 43 0.38%
32 China 40 0.35%
33 Norway 38 0.34%
34 Ireland 33 0.29%
Total main locations (listed above) 6683 58.94%
Other locations (unlisted) 4656 41.06%
Total locations 11339 100.00%
<null> 90260 80.80%
<empty> 10108 9.05%
Total 111707 100%

Note: these numbers are one month old, but they give a good overview of the trends.

Thanks for reading!

Top comments (52)

Collapse
 
ben profile image
Ben Halpern

Cool!!

Collapse
 
yaser profile image
Yaser Al-Najjar

I guess you might want to add throttling on the API based on IP ;)

Collapse
 
ben profile image
Ben Halpern

Yes, but it’s also served from our CDN so it’s not a resource-gobbling endpoint.

 
biros profile image
Boris Jamot ✊ / • Edited on

😱

Collapse
 
matteojoliveau profile image
Matteo Joliveau

Although cool in and of itself, this also highlights why using autoincremented ids is not a good idea when it comes to protecting users privacy (this is not the case of course since I don't see any critical data in the JSON response).

I always use UUIDs for exactly that reason, and also because it helps when building distributed systems (id clashing is less probable with them)

Collapse
 
biros profile image
Boris Jamot ✊ /

I'm sorry, my english level is quite low. Should I have written "scrape" ?

Collapse
 
link2twenty profile image
Andrew Bone

I wouldn't say your English is bad, your writing is better than a lot of native speakers πŸ˜‰

Collapse
 
stefandjokic profile image
Stefan ĐokiΔ‡

At the time of posting this, DEV has 119,105 registered users. This is so cool 😁
dev.to/api/users/119105

Collapse
 
nomangul profile image
Noman Gul
Collapse
 
chiangs profile image
Stephen Chiang

What about me? I'm in Norway...says so on my profile :P

Collapse
 
biros profile image
Boris Jamot ✊ /

Norway: 33 users.

Collapse
 
biros profile image
Boris Jamot ✊ /

38 with Oslo.

Collapse
 
chiangs profile image
Stephen Chiang

i'm sure there's quite a few from Denmark as well :)

Thread Thread
 
biros profile image
Boris Jamot ✊ /

Sure, but there are a lot of other countries I didn't put in the list because their count of users is low.

Thread Thread
 
chiangs profile image
Stephen Chiang

Ah I see, I just assumed it was a complete list because of this line:

Let's have a look at the full statistics below:

And to be fair 38 is more than Ireland ;)

Thread Thread
 
biros profile image
Boris Jamot ✊ /

In fact, Norway and Denmark are in "other locations".

Collapse
 
cosminpopescu14 profile image
Cosmin Popescu

I am from Romania. :D

Collapse
 
biros profile image
Boris Jamot ✊ /

Romania: 45 (Incl Bucharest)

Collapse
 
elmuerte profile image
Michiel Hendriks • Edited on

Why?
I've been on the internet since 1997. I have never bothered about location, age, sex, species. (Although secretly I would be excited to converse with a cat.)

Collapse
 
lugodan profile image
Dan Lugo

1 of 57 from South Africa!

Collapse
 
xelaflash profile image
xelaflash

Nice, it shows that this is a small world!
I'm πŸ‡«πŸ‡·, but living in Barbados πŸ‡§πŸ‡§, i would be interested in knowing if i'm the only one from there? πŸ€”

Collapse
 
biros profile image
Boris Jamot ✊ /
db.getCollection("users.dev.to").find({"location": /\bbarbados\b/i})
{
    "_id" : ObjectId("5bdb776102a47d33d338dcad"),
    "type_of" : "user",
    "id" : 46999,
    "username" : "xelaflash",
    "name" : "xelaflash",
    "summary" : "I'm Alex, i've learned coding in 2018, i'm now willing to start my freelance career.\r\nHere to learn and grow... and learn but mostly learn 😁",
    "twitter_username" : "GhostXela",
    "github_username" : "Xelaflash",
    "website_url" : "",
    "location" : "Barbados",
    "joined_at" : "Nov 27, 2017",
    "profile_image" : "https://res.cloudinary.com/practicaldev/image/fetch/s--czVpA4Fy--/c_fill,f_auto,fl_progressive,h_320,q_auto,w_320/https://thepracticaldev.s3.amazonaws.com/uploads/user/profile_image/46999/256a85b6-8a6d-44c2-b7c6-f7cee7aca32f.png"
}
Collapse
 
biros profile image
Boris Jamot ✊ /

I'll have a look at your query this evening.

Collapse
 
marcorichetta profile image
Rich

πŸ‡¦πŸ‡· πŸ’ͺ

Collapse
 
ryuuji159 profile image
Daniel CortΓ©s

Looks like I didn't fill my location, add one for chile

Collapse
 
simo97 profile image
ADONIS SIMO

Seem like there is not a lot of people from Cameroon....πŸ‡¨πŸ‡²πŸ‡¨πŸ‡²πŸ‡¨πŸ‡²

Collapse
 
venoel profile image
venoel

I even did not know here is registration.

Collapse
 
sduduzog profile image
Beautus S Gumede

I'm probably responsible for ~10% of users from South Africa. And might grow 😌

Collapse
 
biros profile image
Boris Jamot ✊ /

South Africa: 72 (Incl Cape town, Johannesburg)

Some comments may only be visible to logged-in visitors. Sign in to view all comments.

Become a ModeratorHey! We would love your help!

Fill out this survey and help us moderate our community by becoming a tag moderator here at DEV.