As a lot of us know, AI has become a increasingly bigger problem on Dev.to. Whether it's spam or listicles, we've all likely seen AI generated content in one form or the other. Some of the more active Dev.to moderators, myself included, have discussed this and see it as an important issue, but because a lot of that discussion takes place behind closed doors (in other words, the Dev.to moderator discord), I wanted to compile a short post of thoughts and ideas that brings a bit of transparency to the ongoing conversation about AI generated content.
Complaints about AI generated content
It feels like so many of the posts are clout-chasing LLM-generated crap.
-An anonymous moderator
It's not exactly DEV's fault, the internet is mostly AI-generated content and listicles these days.
-An anonymous moderator
I used to read some good stuff and it’s been non stop LLM pushing and listicle style for a while. I admittedly have stopped marking low quality and have abandoned even attempting to read Dev for the past couple months.
-An anonymous moderator
As for my own thoughts, AI has helped low quality content become much more popular. Low quality content used to be pretty easy to spot, but it's harder to filter through low quality content now because 90% of is AI generated.
So what should we do?
That last part was a little bit nihilistic, so I'd like to bring some ideas about how to deal with AI generated content from our moderators to the table. I'd also like to hear from you guys (the comments, that is), so if you have any ideas please feel free to comment them!
... maybe an automatic quality checker would be nice on DEV, like an API that after posting an article checks for the chances of it being AI-generated ...
-An anonymous moderator
... (how about we) force new accounts to wait a {periodOfTime} before they can post content on Dev. This eliminates a large chunk of the regurgitated AI post and run for clicks.
-An anonymous moderator
Have we considered having a "listicle" type, and then letting folks filter those out?
-An anonymous moderator
I think that there are two stages to this idea: suppress publishing of bad written / poor AI content; encourage people to get back to read good content on DEV; ..." Side note: "this idea" pertains to the idea of an automatic quality checker and the potential side effects of it.
-An anonymous moderator
I'd like to acknowledge one potential issue: false positives with AI detectors. This could certainly be an issue, but the amount of false positives would adamantly be negligible. Additionally, these false positives could simply be appealed, and a human could look over the post to clarify that it isn't AI generated.
To conclude...
What do you think about these ideas? As I previously mentioned, if you have any yourself, please feel free to say something. AI generated content is a community-wide issue, and it should be handled as such.
A side note for the people that I've quoted: If you would like to be credited (i.e. your username instead of "An anonymous moderator"), just let me know. I figured it would be better to air on the safe side and not involve anyone who didn't want to be involved.
Top comments (49)
Since I’ve followed the conversation from the start (includes the Dev.to Discord moderation server). I do wanna add some more detail on the "automatic quality checker".
My thoughts were a "trustScore" let’s say between 0-100. This score is then given depending on certain checks, now whoever uses the API could choose when and what to do, is a disclaimer good enough or completely removing the content, if the score is let’s say 90 or above.
Any thoughts?
I like that. I think platforms like GTPZero already kind of do that, but one could certainly expand on that to include other factors (not sure what factors, but still). You could also consider forcing a post that meets your "TrustScore" requirement to have a little badge at the top that says something along the lines of "Hey! This post is very likely to be AI generated" (unless the author appeals, of course).
Anyways, a lot of this is still "theory", and implementing it and putting it into practice may be a pretty big challenge -- not that I would know where to start.
This way of handling AI-generated content should be quite fair, and if we add another level of validation, it could be the best way to handle the situation, in my opinion. I don't think one way of validating is good enough, as there's a chance of incorrectly flagging content.
In my opinion, if the "trustScore" is above, let's say, 90, then decrease the reach of the post in the feed. If the post has that higher trust score with "x" number of moderators flagging it as AI content using a "Flag to Admins 🚩" button, then completely remove the post after informing the user that the post will be removed from DEV in "x" days and suspend the user for "x" days.
To date, I have come across various completely AI-written blogs that are just a good sight away from being noticed. We definitely cannot depend on a single tool like GPTZero to flag AI-generated content, as I have seen it flagging completely human-written content with slightly "better grammar words" 🤷 as AI-generated.
An article can be "AI-assisted" rather than "AI-generated," so we need to take care of that as well.
I definitely don't seem to find this approach any good as @anmolbaranwal has already said it above, many senior devs just joined might be willing to write here. But seeing this timeout sort of thing is already super disappointing, and they might end up moving away from DEV platform itself.
I think I saw a similar post from @thepracticaldev a few months ago. I would definitely love to know the team's thoughts on this.
I appreciate your mention of considering multiple different factors before labeling a post as AI. It's just a generally good idea.
As for the difference between AI assisted and AI generated articles.... I'm really not sure what to say. There are certainly situations where AI assisted content is completely fine (see @syakirurahman 's comment), but there are also situations where I feel that AI assisted content leads to lazy writing.
Anyways, I know there's more that I could talk about regarding your comment, but I'm tired and it's late. Thank you for the input and ideas!
How are you
currently, we are going to build new coin marketplace similar pump.fun or base.fun.
If you have good experience with MERN Stack, I want to collaborate with you
Exactly useless.
And you’re totally right, but this is just the tip of the iceberg.
Here I do actually have some ideas on how to solve that problem. But I like to keep it a secret for now 😅 not that it’s groundbreaking.
Let's unfold the secret 😁
How are you
currently, we are going to build new coin marketplace similar pump.fun or base.fun.
If you have good experience with MERN Stack, I want to collaborate with you
How are you
currently, we are going to build new coin marketplace similar pump.fun or base.fun.
If you have good experience with MERN Stack, I want to collaborate with you
Hey @bluedream0801, I'd love to collaborate! I couldn't find your email on your profile, but feel free to reach out to me at shrijal.acharya@gmail.com.
Okay, Could you tell me how long you have experience with MERN Stack and smart contract?
You can't just remove content because it is AI generated for a few reasons:
The problem we need to address is not AI generated content. The problem we need to address is low quality content.
Just because AI makes it easier to generate junk quickly doesn't mean we should ban AI, it means we should be more strict on junk (since more junk will be coming).
Another important point - not all listicles are bad. While they are often shallow and wastes of time, there are good listicles.
I've written a post about that here:
🚨 🚀 25 Reasons (You MUST Know!!) Why *Some* “Listicles” Are BAD for dev.to 🤯 👿 🚨
Best Codes ・ Dec 11 '23
Interesting to see everybody's thoughts!
"Better" than human written content is an opinion (in my opinion :D). I personally like seeing people's writing style. I enjoy seeing the imperfections and little punctuation errors. I enjoy seeing blogs written in a narrative style, a formal tone, etc. I don't log on to Dev.to every day hoping to read a blog that is perfect (in line with the standard conventions of modern English at that is) because it was AI generated. That being said, there's nothing wrong with chasing perfection. But that's just me :p. To each their own.
AI detectors may not be perfect, but that's why an appeal system would be a good idea. I'd have to do some research myself, but I would personally expect them to be accurate 80% of the time. That's just a random figure though. I should actually test it at some point.
And to jump back to my first paragraph, AI assisted content is fine, sure, but to what extent? I don't have a problem with someone using Grammarly or something similar to check if their punctuation is correct, but I do have a problem with someone pasting a rough draft into ChatGPT and telling it to write a final draft based off of their writing.
Well, most of what you said is a matter of preferences. I don't see why you would have a problem with passing a draft through GPT. Maybe it conflicts with your preferences, but that isn't a reason to remove it - other people might enjoy it.
How are you
currently, we are going to build new coin marketplace similar pump.fun or base.fun.
If you have good experience with MERN Stack, I want to collaborate with you
There have been concerns about low-quality and beginner content before generative AI became as popular and powerful as it is now. There is an unclear line between letting AI write articles and using AI to assist (including Grammarly for the English language and Copilot, tabnine, or Jetbrains AI for code), and people have written annoying listicles without the help of AI in the past.
I am not sure if I am a moderator. I see a "moderate post" button that I use sometimes to give extra up- or downvotes, but I haven't joined any moderator chat so far.
DEV's openness is both a great strength and a great weakness. Communities with a strict gateway don't always achieve their goals, or at least that's open to discussion, as some see StackOverflow as elitist in a positive way, others see "toxic gatekeeping" and still a lot of inappropriate or low-quality content. I prefer the DEV way. Even if it's full of content that I don't find interesting, there are tags, and mentions, and search engines. So I will keep contributing and reading, although not as frequently as I used to in the beginning. But it's not only AI related.
I see your point. Even I'm not sure where I stand on the line of AI assistance. There are a lot of pros and cons and whatnot, and on top of that it's hard to figure out if something was AI generated, or if it was just AI assisted.
Also, I appreciate DEVs method of getting moderators, specifically because of what you mentioned (the fact that it's not very prone to gatekeeping). It also tends to bring in a much more diverse group of people, which is pretty awesome.
And a side note, if you can access the little "thumbs up and thumbs down" buttons, you are a moderator. DEVs monthly emails should include a Discord invite link, but if you can't find it, feel free to DM me at
kurealnumon Discord, and I can invite you myself.How are you
currently, we are going to build new coin marketplace similar pump.fun or base.fun.
If you have good experience with MERN Stack, I want to collaborate with you
How are you
currently, we are going to build new coin marketplace similar pump.fun or base.fun.
If you have good experience with MERN Stack, I want to collaborate with you
Sure.
I want to make something clear yk.
Listicles might be hated but I don't use AI to write even a single word. I do take reference from the project websites, if that is AI generated.. then I can’t control that.
if you would see my first few articles, I used to write with AI but I stopped after seeing how terrible it sounds.
😂 I don't even know why people would use AI to create content. Writing is only good as long as there are emotions and AI completely destroys that yk.
I'm aware some people hate listicles too much. I will attach a snapshot of sublime text and the lines that you see here are open source projects, each are different. I sometimes have to explore 60+ projects in order to pick maybe 17 or so... just saying that I'm not someone who would use AI to ever write my content.
there are many factors involved. let's suppose this:
Imagine a senior technical writer (a good one) wants to try writing on devto, but they can’t because of this. I’ve seen many writers get a top 7 badge with their first article, so this could slightly restrict things.
Anyway, no solution will be perfect, but we can at least reduce the problem as much as possible. I'm part of the mod community so I've been following the discussions there...
To be completely clear, this post wasn't targeted at you at all. I know that writers such as yourself put a lot of time and effort into the posts that you make, and even though I tend to appreciate stuff that comes in less of a "listicle" style format, I don't think that your listicle style articles are bad in any way (and certainly not AI generated).
I do see your point about technical writers being discouraged by a timeout period, but I would hope that with proper explanation (i.e. the timeout says "Here's why we have a timeout on your account; it's not personal!" instead of "You've been timed out for a day") those writers would be understanding and patient. But yea, who knows :p
How are you
currently, we are going to build new coin marketplace similar pump.fun or base.fun.
If you have good experience with MERN Stack, I want to collaborate with you
How are you
currently, we are going to build new coin marketplace similar pump.fun or base.fun.
If you have good experience with MERN Stack, I want to collaborate with you
It's very easy to detect AI content with tools like gptzero, it should be implemented in DEV before posting.
Posts with AI should first inform the person that they include AI, and then they will be ranked lower on the website.
Medium.com has a strict rule that AI content will not be eligible for payments.
Many people use ChatGPT to structure the content. The thing that they don't know is that after they do that, the post will look like AI content and will be detected as AI content.
This sounds like a fine idea to me. The only hassle there is ensuring that GPTZero is accurate 99% of the time. Or as I've mentioned previously, have some sort of appeal system.
Great.
How are you
currently, we are going to build new coin marketplace similar pump.fun or base.fun.
If you have good experience with MERN Stack, I want to collaborate with you
I definitely think that the line to be crossed should be: Is the content giving a new added value? Is it not deep frying other existing contents?
Maybe i am biased but if a score is going to be assigned to the article, i think should be more related to that, rather if the structure of the content has been defined by en LLM (Because the content could be insightful and valuable anyways).
(In my mind i am thinking like asking each LLM the % of that content is known to them from an specific article and which part is fresh and new. But in a more sophisticated way).
I am new to dev.to btw, so maybe is not a strong opinion though :)
Welcome to Dev.to!
I see your point, but if AI generated content is still insightful and valuable, why don't I just go ask ChatGPT myself? I've mentioned it a whole bunch across Dev.to and this post, but I don't want to read something that a bot wrote, I want to read what you (or whoever's article I'm reading) wrote!
How are you
currently, we are going to build new coin marketplace similar pump.fun or base.fun.
If you have good experience with MERN Stack, I want to collaborate with you
Regarding AI, I've always been skeptical of StackOverflow.com's content due to its often contradictory solutions, which are frequently specific to particular operating systems or software versions. OpenAI trained their models using StackOverflow data, potentially incorporating these contextless guesses and version-specific solutions. In contrast, tools like Perplexity provide references to their data sources. It's crucial not to trust AI for factual information without solid reference credits.
This is another issue with AI generated content. Sometimes it's just completely wrong. Some may say that this is just another example of low effort content, but at least low effort content isn't (normally) completely wrong. And even if it is, the author is usually responsive to feedback because they wrote it which means that it's a topic that they're at least interested in.
How are you
currently, we are going to build new coin marketplace similar pump.fun or base.fun.
If you have good experience with MERN Stack, I want to collaborate with you
The focus should not be "AI content" or "listicle", because you can have good quality content that are listicle and partially generated by AI.
I will give you an example.
I'll be honest, my last week article below, is partially (the first section about Path list) generated by AI. If you check with tools like zerogpt, it probably detected as AI content.
💸 How to Make Money From Coding: A Beginner-Friendly Practical Guide
Syakir ・ Aug 8
But, I dont think that article is low quality. i still have to spend almost whole day to research, structure the content outline, curate relevant links, proofread AI generated text, and write some personal experiences there.
AI is really helpful for writers like me who is not native english, to generate some generic texts, like description of something. Really save time. I used to spend 2 or 3 days to create a content like this, because of language barrier.
I agree that a lot of content generated by AI are junks. But that's because the writer dont want to put effort to even proofread it. Without AI, they might never think to be a writer
Now, from my experience as writer, and Dev.to daily user, I found a pattern that a low quality content are usually:
There might be other criteria for low quality content. But the point is, we can decide based on these criteria rather than if its using AI or not.
This is one of the few times where generative AI is really wonderful. From the context of this post, I see nothing wrong with your writing process; you spend a ton of time researching, curating, and drafting your post, and you just use AI as a middleman to help you communicate your ideas more clearly to a larger audience. There's nothing wrong with that, and I would be sad to see one of your posts get flagged by a AI detector, even if you were able to appeal (I've read your posts before, and they are certainly high quality. 10/10).
As for detecting low quality content, I like your idea, but I'd like to present another idea. What if there still was an automatic AI detector, but some accounts were able to automatically bypass that if they were able to provide a good reason that they should be allowed to use AI (whether that be because English isn't their first language, or for another reason). I haven't thought this through though, it was just a shower thought. Let me know what you think.
How are you
currently, we are going to build new coin marketplace similar pump.fun or base.fun.
If you have good experience with MERN Stack, I want to collaborate with you
How are you
currently, we are going to build new coin marketplace similar pump.fun or base.fun.
If you have good experience with MERN Stack, I want to collaborate with you
I've seen an increase of newly created accounts posting "SEO stuffing" articles. I've seen "Best laundry services in XYZ", "Best XYZ agencies in XYZ", an empty article full of hashtags and anything in between...Even two or three accounts posting the same articles over and over. Some people are trying to use dev.to for SEO schemes not related to programming or development.
I think the new account timeout would be helpful here, but as many other people have mentioned, that could very well drive away senior writers or anyone that doesn't feel like waiting a day or two to have their account unlocked.
First detect if an article has been generated with AI, maybe you can use an algorithm or people's labeling it as AI generated. Then put a giant Label next to the title of the article stating whether it's probable to have been generated by AI or not, then monitor the interaction of people with those kind of articles, once that data has been obtained, according to that information, decide whether it should be prohibited or not.
I like the idea of having a big label :). Thank you for commenting!
I think going the 'Hackernoon' way would be best. Get editors to verify before releasing articles.
As I precedently said, many contents are nice. Even though, you must more rate about effectiveness and operationality of technical articles. Some are not easily implemented in reality.
The "listicle" tag is a sensible idea, I think. Allowing all moderators to add it to a post could help a lot.
Some of the ideas, like a period of time before being able to post, sound like the way Stack Overflow does things. In theory, it's not a bad idea, but in practice it puts people off and makes them think the platform's less friendly. I'm concerned that some approaches to "fixing" trash content might end up making DEV seem less like an inclusive community.
You phrased what I wanted to say perfectly. "Fixing trash content" is very vague, and may very well lead to people wanting to remove anything that they don't strictly love (or something along the lines of that).
As for the "listicle" tag, I think it's a great idea. I would personally be a much happier reader if I could filter out listicles.
How are you
currently, we are going to build new coin marketplace similar pump.fun or base.fun.
If you have good experience with MERN Stack, I want to collaborate with you
W take
Some comments may only be visible to logged-in visitors. Sign in to view all comments.