DEV Community

Cover image for Why OpenAI removed Web Browsing from ChatGPT

Why OpenAI removed Web Browsing from ChatGPT

Thomas Hansen on July 07, 2023

The same day we connected our ChatGPT chatbots to the internet, OpenAI disconnected ChatGPT from browsing the web. OpenAI's explanation was that, a...
Collapse
 
theaccordance profile image
Joe Mainwaring • Edited

I applaud your technical explanation regarding paywalls, but I respectfully disagree with your perspective as to the reasoning for the feature. There's a lot of bias in your explanation and simplifies the aggregate of the publisher community to a singular focus of gamifying a search engine without mentioning more practical reasons like my shareholders expect profitability or I need to increase revenue to maintain operations. It also doesn't raise the possibility that Google allows an exception to their rules because of their own motives.

Machines inherently will consume more information than what we as a human sees, the fact that publishers are exposing this information to a search engine but not the user is not about ethics or morals, rather logic. The fact that you're being denied access is because you're a new use-case, and existing solutions to problems weren't designed with your considerations in mind. Businesses and technology is still adapting.

As a final point, I want to emphasize that the internet is not free as in speech, it is free as in beer. While yes, the internet does offer avenues for unlimited expression, it requires infrastructure based in a capitalistic society. It costs money to run servers and have an internet connection. Have you any context as to the cost burden forced on publishers to support models accessing its content? A significant uptick in traffic for internet resources already at scale is not a trivial expense increase.

Collapse
 
lewiscowles1986 profile image
Lewis Cowles

You got speech and beer around the Wong way lad. It's not free beer, but it is free speech πŸ˜‰. Free does not mean freedom from consequences, but [unfair] persecution [by state entities].

Paywalling content where the scripts to paywall can be removed (hello Firefox reader mode) is foolish. Doing so to lower cost shows a fundamental set of mental gymnastics that is not conducive to cost reduction. Which by the way public unfettered access is also a barrier to.

Collapse
 
polterguy profile image
Thomas Hansen

Amen!! 😊

Collapse
 
theaccordance profile image
Joe Mainwaring

I assure you, I do not have this backwards.

Not realizing this simply tells me you don’t actually understand the economics of serving resources on the internet.

Collapse
 
schwendemannick profile image
Nick Schwendeman

This comment is the correct answer.

The article author apparently believes everything should be free. That the people who produce the content he wants to consume are in an invalid profession. They are unethical to ask to be paid for their efforts, he says! They are dinosaurs, he says! Yet he still wants to consume what they produce....just for free. Hmm. I wonder, what do you do for a living? Are you willing to do it for free?

Speaking of unethical dinosaur business models, I was forced to sign up for a Dev.to account just to leave this comment. Oh, it was "free", but I had to agree that they have access to all my tweets and other content (I used Twitter to sign up, God knows what they would have access to with the other methods!). So just to leave my opinion here, I was forced to trade away my personal data. Why?? Why put the ability to share my freedom of expression behind a personal data paywall??

I'm afraid this whole article smacks of hypocrisy.

Collapse
 
polterguy profile image
Thomas Hansen

Speaking of unethical dinosaur business models, I was forced to sign up for a Dev.to account just to leave this comment

I rest my case, but I'd like to finish with a "thank you for increasing the engagement of my article" ... ;)

Collapse
 
polterguy profile image
Thomas Hansen • Edited

Respectfully, but if ...

my shareholders expect profitability or I need to increase revenue to maintain operations

... is the only reason you're in business, you've got a holy duty to quit your job, and find another living!

Businesses and technology is still adapting

Traditional media haven't adapted an inch since 1995, at least not in a positive direction ...

internet is not free as in speech, it is free as in beer

Agree, and this is largely traditional media's and social media's fault, and it is not a good thing ...

A significant uptick in traffic for internet resources already at scale is not trivial expense increase

Because their tech is garbage, and they're downloading GBs of rubbish, where they could have optimised their content and servers, and ended up with a slick streamlined experience, not wasting hundreds of MB on "garbage" ...

You have some points, you are correct, however the moral implications of choosing your side could be used as a justification for Ted Bundy in its most extreme case ...

Ethics and technology are not two separate distinct concepts, they're intertwined, just like everything else in this world - And traditional media (and social media for that matter) are pure evil constructions, based upon tricking the masses into believing in whatever rubbish the've got some sort of stake in spreading, due to having been paid to spread it ...

Collapse
 
akashpattnaik profile image
Akash Pattnaik

Okay, first of all, I love the way you explained it! I do agree with what you said, but I think it's not right to think that they removed web browsing just because of a particular reason. There are a various factors that concluded to this decision, but yeah, one of the biggest was people were abusing this web integration in payment gateways!

Collapse
 
polterguy profile image
Thomas Hansen

Actually, this was what they gave themselves as their reasons. Then I realised most don't understand how payment walls works, and/or what it means, so I felt the urge to explain it. But their own statement is as follows;

We removed it because users could abuse it to bypass payment walls

Thx for the nice words :)

Collapse
 
simulanics profile image
Matthew A. Combatti

Sadly... All it takes to bypass WSJ Paywall is:
javascript:url=window.location.href;window.location=url.substring(0,url.indexOf('/articles'))+'/amp'+url.substring(url.indexOf('/articles'));

Just paste it in the address bar and wallah. πŸ™

Collapse
 
polterguy profile image
Thomas Hansen

Why am I not surprised ... :/

Collapse
 
mindplay profile image
Rasmus Schultz

If the above company had any other name but "Wall Street Journal", they would have been blocked out of Google's search index, since it's a black hat SEO technique often referred to as "cloaking".

While cloaking is technically implemented the same way, Google themselves provides you with directions on how to avoid getting blocked:

developers.google.com/search/docs/...

Cloaking generally refers to the idea of presenting different content to search engines - that's subtly different from merely omitting content from responses to public clients.

Yes, it's immoral, and the news internet these days is mostly paywalled.

Just add 12ft.io in front of every paywalled URL and show them who's boss. 😎

Collapse
 
polterguy profile image
Thomas Hansen

Hehe :D

Collapse
 
coolthings247 profile image
Funder Games

Love that this was writen largely by AI itself. I 100% agree. We need new frontiers, new frontiers give us new opportunities - the previous frontier will have been saturated with exploitation of its users, new frontiers give us the opportunity to break free from that and for a short time, live with a sense of freedom.

Collapse
 
polterguy profile image
Thomas Hansen

We do produce about 15+ articles each day using AI, but this was only me πŸ˜‰

oracle.ainiro.io

Collapse
 
djayed profile image
Djayed - He/Him

I thought that was a feature. It's been showing me stuff behind paywalls since before they rolled out Internet. Old articles and research papers are behind paywalls too. I've used it for school to help research and to do citations. It sucks they are, "fixing" it.

Collapse
 
polterguy profile image
Thomas Hansen

I honestly don't understand why they're fixing it. There are already high court verdicts in the US proclaiming quote "AI generated content is not possible to copyright" and it's already been established that "re-mixing content to create new content is not a copyright violating as long as no copying occurs" - And, it's the same thing we've done for centuries when citing each other. The only difference is that now there's a machine citing us ...

Copyright implies just that, "the right to copy". No copying, no copyright. The AI doesn't copy, it writes its own unique content, based upon existing content it has consumed as it was learning. If this is illegal, you cannot use any of your knowledge, since arguably every piece of knowledge you have, originates from a copyrighted book somewhere ...

Collapse
 
davidschnurr profile image
David Schnurr

This highlights the need for more transparency in search results. Users deserve to know whether an article is behind a paywall before clicking on it. OpenAI's adaptation to this situation shows their commitment to user experience when using chatgpt. Please continue to uphold!

Collapse
 
polterguy profile image
Thomas Hansen

You're correct, it is "click baiting", but OpenAI didn't remove it because of the reason you state, they removed it because of publishers complaining about "OpenAI stealing their articles".

If it's publicly available for scrapers, it should be publicly available for humans - Everything else is ipso facto click baiting ...

That Google even allows for this is quite frankly beyond my comprehension. It degrades the user experience a lot ...

Collapse
 
piamedea profile image
Pia

Hear here!!! I respect and agree with do much of what you say here. Especially in regards to pay walls they disgust me. It was cool to learn more about them here though it just exacerbated how much I hate them.

Collapse
 
polterguy profile image
Thomas Hansen

Thx Pia, I don't mind people charging, but the bait and click parts of the way these guys are implemented is not particularly ethically I think ...

Collapse
 
arianygard profile image
AriaNygard

Interesting viewpoints! Disappointing that OpenAI removed web browsing, but it's not like the only option you've got to search the web with ChatGPT was through OpenAI ;)

Collapse
 
polterguy profile image
Thomas Hansen

Hehehe, nope :D

Collapse
 
lewiscowles1986 profile image
Lewis Cowles

Not sure I like the framing of chatgpt as such an essential for-good, but aren't there plugins that bypass, which are trivial to author anyway?

Collapse
 
polterguy profile image
Thomas Hansen

No plugins. We're on our own. I'm not framing ChatGPT as essential, I'm framing the web as such πŸ˜‰

Collapse
 
vijai_bishnoi profile image
Vijay Bishnoi
Collapse
 
captaindigitals profile image
CaptainDigitals

12ft.io Is one of my favorite paywall removers, doesn't work on all but most.

Collapse
 
polterguy profile image
Thomas Hansen

The point wasn't to remove paywalls, it was to innovate and not allow ourselves to be restricted by dinosaur business models ...

Collapse
 
polterguy profile image
Thomas Hansen