Hello, wonderful DEV community!
My name is Amy, and I’m a product manager here at Forem. I started my journey here this past August so if we haven...
For further actions, you may consider blocking this person and/or reporting abuse
Yes! I was waiting for this.
Those "bad" posts gathering instant reactions and being visible for a long period to gather even more reactions to become even more visible .. are annoying. I have even seen people who intentionally create alt accounts to have that first boost and make sure their posts are visible on the top and if you check them, it's only promotional content.
I would also love if we can give more weight to the "rate quality" feature given to trusted users. I know we might have some abuse here as well but it would be insignificant compared to gaming the reactions especially that trusted users are manually selected by the DEV team.
I believe this is already the case - if I'm remembering correctly the Thumbs Up/Down by Trusted users are about 10x as valuable as a standard user's reaction. (This is based on a discussion in the connect mod chat so I can't go back to confirm this 😅)
Considering that a user can perform 3 reactions and that you can add reactions to your own post, 10x is still low. With 2 extra accounts you can get 9 reactions immediately so you are immune to the first vote.
Based on my experience, If I am late to downvote a bad post before 10/15 reactions then it's too late unless more trusted users do it before that post gather more reactions.
Once you have your first boost, you are on the top and more reactions will come easily so it become more difficult to fight. Bad posts can easily reach 100+ reactions in a few hours.
@afif This type of fine-tuning is exactly what @jeremyf and I have in mind as potential candidates for future experimentations. We believe this is just a start for many iterations to come. I am logging it here in an issue to make sure we don't miss it when we are ready for the next round of experimentation.
Thanks for your feedback!
It will be interesting to see whether this works, I hope it does.
Being the cynic that I am I think this is the wrong move, however.
While reaction farming is a big problem every site faces, the reduced relevance of votes and comments will have a side effect,
Posts that are high quality, but from lesser known authors will not get the boost they deserve if people find their article useful and upvote it and comment to show their support.
The problem (as I see it) is the tag system and always has been as far as I can tell.
If you write an article just add the #javascript tag and you will double your views for example. Albeit a mod may come and remove the tag but I have gamed the system by adding a JS example that doesn't add much to the article just so I can include the tag and it worked.
That is where I would reduce the relevance.
As for tackling farming reactions, having a hidden system that decides whether or not to count a vote for the algorithm, based on account age or some other factor, would be great.
So you have a public likes counter and then you have a separate one for the algorithm that only counts votes and comments that are from accounts xxxx days older or with a total of yyyy votes.
Don't get me wrong, I have seen some big improvements in the feed since moderators seemed to have been more involved and some of the experiments you have run have been great.
The only other thing that would be interesting to run an experiment on would be "DEV picks".
Similar to the "Top 7 posts" weekly update but actually incorporated into the feed (increased visibility and longevity perhaps?) so there is a place we can see posts that have been hand picked by the dev team and the mods. These are likely to be very high quality and hopefully quite diverse in nature.
Anyway those are my 2 cents and I am always impressed by the work the team at FOREM and DEV do to try and always improve. ❤🦄
Just to make sure I understand you correctly (you know, I am still rather new here 😉), when tagging a post with
JavaScript, you will double the views because there are a lot of people following this tag. Correct? I do see how people can abuse the tags and hence reduce the relevance here. This is going to be an interesting problem to solve.I have also gathered your suggestions on feed experimentation here for review at a later time.
Thanks so much for the write-up! I love your candid feedback.
Yes that is correct.
But the problem isn’t abusing the tags as such it is more that you have to abuse the tags if you want to get an article to show up enough to get traction (if that makes sense?)
I Love the steps the team are taking, I don’t envy you all as trying to solve biases and abuse in a feed is not an easy task! ❤️
Ah. I see. This is one of the discovery problems we are facing for less known tags (i.e. with fewer followers), similar to new authors having a hard time getting their posts surfaced, albeit of high quality.
Thanks for the extra context. It helps me see what problems you are facing with the feed as an author. 🥰
All I have to say now is welcome to DEV and as your first 2 official posts / features these are great (despite the criticism! 🤣) and super positive steps!
I look forward to seeing what you tackle next / the outcome of this feed experiment! ❤
These are awesome news that the DEV team is working to fix the feed 👍
I've posted regularly for 2 years and really love the platform and the community 💖
It is worth noting, of course, that the extra weight (previously) put on reactions is not totally unfounded. After all, it is one of the main metrics for user engagement.
The "too-many-listicles" phenomenon only occurs simply because listicles are inherently designed to be bookmarked (i.e. one of the three reaction types that DEV considers). Hence, we end up in a situation where in addition to the hearts and unicorns a listicle receives, the extra bookmarks greatly amplify what would have otherwise been a regular article.
However, the same argument can be said about regular articles, tutorials, and visualizations. That is, they can also "game the Algorithm" by being designed to be bookmarked—intentionally or otherwise. But, what separates listicles from regular articles is that they are more easily automated than full articles, guides, etc. And that is exactly why the feed becomes bloated. To make a long story short, automation and bookmarks make a perfect storm.
Unfortunately, I have no recommendations for addressing this at the moment. It's really tricky because—as mentioned earlier—certain patterns are just inherently gameable.
Anyway, with that said, I just wanted to bring some additional perspective to the table. Looking forward to hearing about the proposed solutions! 🚀
This is a great insight! 🤔 We should test treating bookmarks differently from other reactions. I am logging it here for review.
Thanks for the great suggestion!
It's great that you are trying to optimise this feature.
Since anyone could make alt accounts to give an initial boost to the post, these are my suggestions:
Also, IMO, algorithm doesn't need to be publicly known, so it would be less abused. It's important that it works good and feedback is collected from users.
Oh I love that last concept! The predefined preferences would be great for people running different instances of Forem!
The low effort part is a fair point! (I would love to see what would happen if you could only upvote 3/5 articles a day for example, that would be super interesting to see how people behaved when their votes were "valuable"!)
One idea could be time on page / scroll percentage to assess article quality / engagement? I know that is getting complex though (but one positive is it can nearly all be done client side with little performance impact)!
The only other idea I have to throw in the pot (and I have no idea if it is a good one) is to weight comments with a certain length / word count (so the "thank you" comments get ignored) more heavily.
Then the number of comments on an article that meet this criteria are then used as a weighting factor.
Yet again this is just based on my usage of the site where I look for posts with a lot of comments as they tend to either be really good or really controversial (and most of the DEV team know I am the one who loves to be involved in controversial stuff on here 🤣).
In all seriousness, looking at number of comments tend to lead me to some cracking content (or at least to engaged authors who respond), so it may be another lever to add after all of your planned experiments?
Anyway I have taken up plenty of your time I am just enjoying the complexity of the task you have undertaken as it is so nuanced!
No problem, I know how tough something like this can be to balance when there are so many variations of tag preferences etc.
Just a personal preference but relevancy first felt better (if that is the difference) and you might have slightly overcooked the pulling back of reactions (I could be entirely wrong, a sample size of one is not very useful I know!) as that seems to have 0 bearing now?
But I would imagine the end goal is to see which feed produces more balanced engagement and if this works then that is great!
Hope the experiments continue! Can't wait for the write up on the results! ❤
Hey Jeremy, quick heads up I think I am one of the ones (currently, not sure if you randomise at a certain time frame) using the new feed.
I think there is a tweak needed somewhere as my feed is pretty much in date order now like "latest" but with selected articles.
I hope that is useful feedback (if you weren't already aware) as it is burying things of value.
Also to save you having to look it up I have no preferred tags selected, just in case it only affects people without any feed preferences.
Super, thanks for those links, I knew I have seen that somewhere in the codebase but couldn't find it again!
The bit I didn't get is does that config mean that reactions have a slightly negative impact of 0.9988 if there are 0,1,2 or 3 reactions and then fall back to 1 for more than 3 reactions?
So at this stage essentially reactions are completely pointless (as I can give myself 3 reactions) to let you fine tune the other parts of your experiment? (which makes sense, A / B test one thing at a time so you know which lever pulls the feed which way!)
It is super interesting as where my gut leads me is that reactions are one of the more important metrics but the way they are gathered / assessed is the bit that needs addressing (I mentioned only counting votes from accounts that meet a certain criteria such as certain age or total number of likes for example), but often you can think one thing and find it has a completely unintended side effect!
The way you have the levers set is smart, wish I was the one playing with them as I would cause havoc going to the extremes 🤣!
Love it. The idea of making the feed more relevant. I'm far less interested in popular than I am in relevant and interesting to me. Twitter, Facebook, Instagram are all for the popularity junkies.
I see some rather poor quality Dev articles on my feed at times but mostly just let them slide on by. Seeing less of those would please me.
Above all I don't need quantity either or volume in preference to quality. I don't mind a feed that delivers more slowly, the right stuff. I do read som top quality stuff here too. And I bookmark it for future reference. A lot in fact!
Good job to those that worked on this. It's always great to improve algorithms.
That said, I don't use the algorithm-driven feed; I use the latest feed that shows all the articles in the order they were posted, so instead of actually dealing with the low quality articles this seems to me like a way to sweep them under the proverbial rug where most users simply won't see them. So I have to ask, is anything being done to actually deal with low quality articles and those that post them?