Tackling Clickbait on DEV: Our Strategy and Actions
Recently, we have implemented measures to address the issue of clickbait on DEV. This post aims to explain our approach and the rationale behind it.
Defining Clickbait
Clickbait typically refers to content with certain patterns in titles, such as listicles or sensationalist headlines. While listicles, for instance, have gained traction for driving engagement on DEV, they sometimes lead to an imbalance, as illustrated by community initiatives like this Chrome add-on:
Our Approach
Emphasis on Mitigation Over Strict Filtering
Instead of implementing rigid filtering, we've opted for a more nuanced approach. The reasons are as follows:
- Balance in Content: Not all listicles are negative; they can be informative. However, they are often disproportionately rewarded due to our engagement-based system.
- Diverse Nature of Clickbait: Clickbait comes in many forms, and a rigid filter might not effectively catch them all.
- Subjectivity Issues: The subjective nature of determining what constitutes clickbait makes strict filtering impractical.
AI and Human Judgment in Scoring
We introduced a clickbait_score field in our app, ranging from 0.0 to 1.0.
Add articles clickbait_score as factor in final feed ordering
#20493
What type of PR is this? (check all applicable)
- [ ] Refactor
- [x] Feature
- [ ] Bug Fix
- [ ] Optimization
- [ ] Documentation Update
Description
This adds a new field allowing a new factor in final feed ordering. It adds the A/B tests to examine its impact on community use, and adds an API endpoint to allow labeling experimentation outside of core and the allowance of third party labeling and toolking to be created by any OSS Forem admins.
Related Tickets & Documents
QA Instructions, Screenshots, Recordings
Please replace this line with instructions on how to test your changes, a note on the devices and browsers this has been tested on, as well as any relevant images for UI changes.
Added/updated tests?
We encourage you to keep the code coverage percentage at 80% and above.
- [x] Yes
This score isn't determined solely by algorithms. We utilize AI along with human judgment to assign scores to posts, recognizing the subjective and nuanced nature of the task.
Implementing Through A/B Testing
We are carefully integrating the clickbait score. As part of our ongoing A/B tests, users may experience varying levels of clickbait mitigation. This data will help us refine our approach in the coming weeks.
Looking Ahead: Customizable User Settings
While we are working on a default setting for managing clickbait, we recognize the need for personalization. Future updates may include user-specific settings to tailor feed preferences based on clickbait scores.
We will be sharing more in the near future about a variety of strategies we are undertaking to improve the core experience and help separate signal from noise on DEV.
Happy coding ❤️
Top comments (26)
Great job DEV team! My add-on is no longer needed, which is great (I also think that not all listicles are bad). Thanks for all of your hard work.
Thank you!
But with this in mind, I'd love for us to expose some native functionality people could plug in to to refine their feed.
Will get back to you.
Great work DEV team! This will help make the community better.
I totally agree with your comment @uliyahoo!
Amazing work!
great work..
I love it; we can focus on high-quality articles instead of listicles.
One thing that is not being solved yet is nonrelevant articles that are being posted by bots (non-dev related)
There's a bunch of it on the latest feed, and that massive amount of spam causes good articles not to get visibility.
Maybe some kind of
tech_scorecan be assigned, saying how technical the post is.Of course, it's completely legitimate and desirable to have non-tech posts here, often under #watercooler or similar tags, but it might be some indication.
I'm happy that the DEV team is addressing clickbait posts, but I would like it even better if you considered another metric as well. The number of engagements that a post gets in a short amount of time should be considered I think. At the very least to flag a post for further review. Trigger happy trolls often create posts that they know are very controversial and thus produce a large amount of response in a short time. Currently such posts can reside in the top 1 on DEV posts for weeks before getting removed due to significant negativity in the commentary all the while moderators get flooded with reports from people who want to misuse moderation to silence the other side of the argument. Instead of muting/banning the people, consider removing posts that trigger such commentary from the listings early, by checking for number of interactions in a 2 hour period for example.
I thank you. My blood pressure thanks you.
haha, so true
I would make sure to watch out that you don't change the platform too much for the sake of a small but loud minority position.
It's easy to hate on listicles, but they are by no means low-effort content and they provide value to a lot of users - hence why they get engagement and dominate top feeds.
Click-ability is also a natural push in any content feed. You can spend weeks working on high quality content, but if you don't optimize its click ability - no one will see it or know about it. This is by no means unique to dev.to, it's a feature of every content ecosystem.
Greater user customization and more sophisticated filtering could be great, just don't be overly reactive to users who may not represent the best interests of the platform.
Yup, that is definitely the ethos we're trying to follow here. Our implementation is not an all-or-nothing adjustment — just a cue in an increasingly intelligent algorithm. As much as we respect the filter extension, it is definitely not the direction we'd have gone. I think we are hitting the right tone here and a/b testing everything.
But thank you for the feedback, it is definitely valuable to hear this, definitely agree.
Good job, keep up the good work making this platform more welcoming and with high-quality content.
Ben, what is your definition of "clickbait'?
What I've noticed for quite some time now is the huge amount of spam accounts, which are completely unrelated to web/software dev - so I was wondering if you have a strategy in place to combat that (but, from what I can see it seems that such accounts are being actively removed, so that's a good thing).
Keep up the good work!
This is a really cool initiative. I think this will make the feed much healthier and we will have higher quality content on the platform. 💪
There is limited visibility, though, on how things are being filtered without having to dive into the algo, IMO. I have recently posted a post with a non-trivial amount of work (dev.to/quine/5-open-source-tools-f...), and I believe I have been penalised by the algo (around 5% of the impressions I normally get after 24 hours). I am not sure why, and I'm a bit bummed about this. 🙃
I really appreciate the way you face such problems.
Hope for the best.
Great post. Strategy 2 really shocked me!
Just kidding. Please don't ban me :D
Nice work team!
Some comments may only be visible to logged-in visitors. Sign in to view all comments.