loading...
Cover image for ๐ŸŒ Where Are DEV Users Coming From?

๐ŸŒ Where Are DEV Users Coming From?

biros profile image Boris Jamot โœŠ / ใƒปUpdated on ใƒป3 min read

As you may already know, dev.to has an API to fetch articles and users.

Let's give it a try with my favorite HTTP client, httpie:

http https://dev.to/api/users/1
{
    "github_username": "benhalpern",
    "id": 1,
    "joined_at": "Dec 27, 2015",
    "location": "Brooklyn, NY",
    "name": "Ben Halpern",
    "profile_image": "https://res.cloudinary.com/practicaldev/image/fetch/s--DOU9qJSH--/c_fill,f_auto,fl_progressive,h_320,q_auto,w_320/https://thepracticaldev.s3.amazonaws.com/uploads/user/profile_image/1/f451a206-11c8-4e3d-8936-143d0a7e65bb.png",
    "summary": "A Canadian software developer who thinks heโ€™s funny.",
    "twitter_username": "bendhalpern",
    "type_of": "user",
    "username": "ben",
    "website_url": "http://benhalpern.com"
}

What I first noticed is that id seems to be auto-incremented.
After some tries, I quickly found out the total number of users on dev.to: around 119000.
It gives me the idea to scrape all the users public data and try to play with it.

Let's try with this command:

for i in {1..119000}; do http https://dev.to/api/users/$i >> users.dev.to.json && echo "," >> users.dev.to.json && echo "https://dev.to/api/users/$i"; done

After a few hours running in the background, I finally got all the users in my JSON file.

I ended up by importing the JSON file in a mongodb collection using NoSQLBooster.

There isn't many information in the users' profile, but two of them worth having a look at them: joined_at and location. It could be very interesting to have a diagram showing the progression of registrations through time. Maybe I'll do it some day, but for now, let's compute some statistics on users' location.

I used two queries to compute the statistics. This one helped me to get the big trends in people's location:

db.getCollection("users.dev.to").aggregate(
    {"$group": {_id: "$location", count:{$sum:1}}}
).sort("-count")

Then, I use another query to get the count of people per location:

db.getCollection("users.dev.to").find({"location": /\bdublin\b/i})

The conclusions are the following:

Most of the users (90%) don't fill their location.
16.3% of the users who filled their location are from USA.
5.8% are from India.
4.9% are from UK.
4% are from Japan.
3.8% are from Germany.

Let's have a look at the full statistics below:

# country region or city #region or city #country %
1 USA 1850 16.32%
Incl New York 205
Incl CA 194
Incl NY 147
Incl San Francisco 137
Incl TX 119
Incl Seattle 117
Incl Los Angeles 107
Incl Chicago 101
Incl Washington 81
Incl FL 78
Incl PA 68
Incl MA 67
Incl AZ 38
Incl MI 34
Incl OH 30
2 India 660 5.82%
Incl Bangalore 107
Incl Delhi 103
3 UK 551 4.86%
Incl London 258
Incl England 35
Incl Scotland 33
Incl Wales 7
4 Japan 451 3.98%
Incl Tokyo 242
5 Germany 428 3.77%
Incl Berlin 128
6 Brazil 301 2.65%
7 France 294 2.59%
Incl Paris 98
8 Canada 257 2.27%
Incl Toronto 107
9 Nigeria 187 1.65%
Incl Lagos 97
10 Netherlands 154 1.36%
Incl Amsterdam 62
11 Spain 110 0.97%
12 Argentina 109 0.96%
13 Italy 102 0.90%
14 Indonesia 92 0.81%
15 Russia 89 0.78%
16 Mexico 86 0.76%
17 Philippines 85 0.75%
18 Poland 81 0.71%
19 Ukraine 75 0.66%
20 Portugal 63 0.56%
21 South Africa 57 0.50%
22 Pakistan 56 0.49%
23 Turkey 55 0.49%
24 Belgium 53 0.47%
25 Egypt 52 0.46%
26 Bangladesh 50 0.44%
27 Colombia 46 0.41%
28 Romania 45 0.40%
29 Kenya 45 0.40%
30 Switzerland 45 0.40%
31 Austria 43 0.38%
32 China 40 0.35%
33 Norway 38 0.34%
34 Ireland 33 0.29%
Total main locations (listed above) 6683 58.94%
Other locations (unlisted) 4656 41.06%
Total locations 11339 100.00%
<null> 90260 80.80%
<empty> 10108 9.05%
Total 111707 100%

Note: these numbers are one month old, but they give a good overview of the trends.

Thanks for reading!

Discussion

pic
Editor guide
Collapse
ben profile image
Collapse
yaser profile image
Yaser Al-Najjar

I guess you might want to add throttling on the API based on IP ;)

Collapse
ben profile image
Ben Halpern

Yes, but itโ€™s also served from our CDN so itโ€™s not a resource-gobbling endpoint.

biros profile image
Collapse
matteojoliveau profile image
Matteo Joliveau

Although cool in and of itself, this also highlights why using autoincremented ids is not a good idea when it comes to protecting users privacy (this is not the case of course since I don't see any critical data in the JSON response).

I always use UUIDs for exactly that reason, and also because it helps when building distributed systems (id clashing is less probable with them)

Collapse
biros profile image
Boris Jamot โœŠ / Author

I'm sorry, my english level is quite low. Should I have written "scrape" ?

Collapse
link2twenty profile image
Andrew Bone

I wouldn't say your English is bad, your writing is better than a lot of native speakers ๐Ÿ˜‰

Collapse
stefandjokic profile image
Stefan ฤokiฤ‡

At the time of posting this, DEV has 119,105 registered users. This is so cool ๐Ÿ˜
dev.to/api/users/119105

Collapse
nomangul profile image
Collapse
chiangs profile image
Stephen E. Chiang

What about me? I'm in Norway...says so on my profile :P

Collapse
biros profile image
Collapse
biros profile image
Collapse
chiangs profile image
Stephen E. Chiang

i'm sure there's quite a few from Denmark as well :)

Thread Thread
biros profile image
Boris Jamot โœŠ / Author

Sure, but there are a lot of other countries I didn't put in the list because their count of users is low.

Thread Thread
chiangs profile image
Stephen E. Chiang

Ah I see, I just assumed it was a complete list because of this line:

Let's have a look at the full statistics below:

And to be fair 38 is more than Ireland ;)

Thread Thread
biros profile image
Boris Jamot โœŠ / Author

In fact, Norway and Denmark are in "other locations".

Collapse
elmuerte profile image
Michiel Hendriks

Why?
I've been on the internet since 1997. I have never bothered about location, age, sex, species. (Although secretly I would be excited to converse with a cat.)

Collapse
nathi360 profile image
Nkosinathi Mothoa

1 of 57 from South Africa!

Collapse
xelaflash profile image
xelaflash

Nice, it shows that this is a small world!
I'm ๐Ÿ‡ซ๐Ÿ‡ท, but living in Barbados ๐Ÿ‡ง๐Ÿ‡ง, i would be interested in knowing if i'm the only one from there? ๐Ÿค”

Collapse
biros profile image
Boris Jamot โœŠ / Author
db.getCollection("users.dev.to").find({"location": /\bbarbados\b/i})
{
    "_id" : ObjectId("5bdb776102a47d33d338dcad"),
    "type_of" : "user",
    "id" : 46999,
    "username" : "xelaflash",
    "name" : "xelaflash",
    "summary" : "I'm Alex, i've learned coding in 2018, i'm now willing to start my freelance career.\r\nHere to learn and grow... and learn but mostly learn ๐Ÿ˜",
    "twitter_username" : "GhostXela",
    "github_username" : "Xelaflash",
    "website_url" : "",
    "location" : "Barbados",
    "joined_at" : "Nov 27, 2017",
    "profile_image" : "https://res.cloudinary.com/practicaldev/image/fetch/s--czVpA4Fy--/c_fill,f_auto,fl_progressive,h_320,q_auto,w_320/https://thepracticaldev.s3.amazonaws.com/uploads/user/profile_image/46999/256a85b6-8a6d-44c2-b7c6-f7cee7aca32f.png"
}
Collapse
biros profile image
Boris Jamot โœŠ / Author

I'll have a look at your query this evening.

Collapse
cosminpopescu14 profile image
Cosmin Popescu

I am from Romania. :D

Collapse
biros profile image
Boris Jamot โœŠ / Author

Romania: 45 (Incl Bucharest)

Collapse
marcorichetta profile image
Rich

๐Ÿ‡ฆ๐Ÿ‡ท ๐Ÿ’ช

Collapse
ryuuji159 profile image
Daniel Cortรฉs

Looks like I didn't fill my location, add one for chile

Collapse
simo97 profile image
ADONIS SIMO

Seem like there is not a lot of people from Cameroon....๐Ÿ‡จ๐Ÿ‡ฒ๐Ÿ‡จ๐Ÿ‡ฒ๐Ÿ‡จ๐Ÿ‡ฒ

Collapse
venoel profile image
venoel

I even did not know here is registration.

Collapse
sduduzog profile image
Beautus S Gumede

I'm probably responsible for ~10% of users from South Africa. And might grow ๐Ÿ˜Œ

Collapse
biros profile image
Boris Jamot โœŠ / Author

South Africa: 72 (Incl Cape town, Johannesburg)

Collapse
subsr97 profile image
Subramanian ๐Ÿ˜Ž

A bit late.
I'm from India! ๐Ÿ‡ฎ๐Ÿ‡ณ

Collapse
aligoren profile image
Ali GOREN

I'm from Turkey.

Collapse
purwnt profile image
Purwanto

Good Job, I'm from Indonesia

Collapse
ash1eyish profile image
Ashley Maria

"Full" statistics

Collapse
nans profile image
Nans Dumortier

It might be interesting to see how the proportion of the different nationalities evolve with time!
Maybe one day we'll spot that dev.to is booming in an unexpected country!
Cheers from France ๐Ÿ‡ซ๐Ÿ‡ท :)

Collapse
awwsmm profile image
Andrew (he/him)

Where my other 32 Irish at? ๐Ÿ‡ฎ๐Ÿ‡ช

Collapse
aibolik profile image
Aibol Kussain

Oh, that's interesting, how many arr there people from Kazakhstan and Belarus?) Just curious

Collapse
biros profile image
Boris Jamot โœŠ / Author
  • Kazakhstan: 6 (Incl Almaty)
  • Belarus: 26 (Incl Minsk)
Collapse
juanfrank77 profile image
Juan F Gonzalez

Latam represent!

Collapse
ryuheechul profile image
Heechul Ryu

No Koreans? :(

Collapse
bennypowers profile image
Benny Powers ๐Ÿ‡ฎ๐Ÿ‡ฑ๐Ÿ‡จ๐Ÿ‡ฆ

ื™ืฉืจืืœ ืืคื™ืœื• ืœื ื”ื•ืคื™ืข ื‘ืจืฉื™ืžื”! ื™ืœืœื” ื—ื‘ืจ''ื” ืงืฆืช ื’ืื•ื•ื” ืœื ื™ื”ืจื•ื’ ืืชื›ื.

Collapse
biros profile image
Boris Jamot โœŠ / Author

ืจืง ืฉืœื•ืฉื™ื ื•ืฉืœื•ืฉื” ืžืฉืชืžืฉื™ื ืžื’ื™ืขื™ื ืžื™ืฉืจืืœ

Collapse
bennypowers profile image
Benny Powers ๐Ÿ‡ฎ๐Ÿ‡ฑ๐Ÿ‡จ๐Ÿ‡ฆ

ื–ื” ืขื•ื•ืœื” ื—ื™ื™ื‘ื™ื ืœืฉืคืจ ืืช ื”ืžืฆื‘. ื’ื ื‘ืืจืฅ ืžื’ื™ืขื” ืœื”ื dev.to!

Thread Thread
bengreenberg profile image
Ben Greenberg

ืกื‘ื‘ื”! ื ืชื—ื™ืœ ืœื›ืชื•ื‘ ื›ืžื” ืคื•ืกื˜ื™ื ื‘ื‘ืœื•ื’ ื‘ืขื‘ืจื™ืช

Thread Thread
bengreenberg profile image
Ben Greenberg

ืื‘ืœ, ืื ื™ ืจืง ืจื•ืื” ืืช ื–ื” ืžื™ืžื™ืŸ ืœืฉืžืืœ ืชืžื™ื›ื” ื‘ืฉืคื” ืœื ื’ื“ื•ืœ ื›ืืŸ.

Thread Thread
bennypowers profile image
Benny Powers ๐Ÿ‡ฎ๐Ÿ‡ฑ๐Ÿ‡จ๐Ÿ‡ฆ

ืชืจื’ื•ื ื—ื•ืคืฉื™: PR in flight?

@ben How can we improve support for RTL languages on dev.to?

Thread Thread
bengreenberg profile image
Ben Greenberg

Formatting punctuation marks not to automatically be placed at the beginning of a sentence in a RTL language would be a big help.

Thread Thread