DEV Community: MichaelSwerston

In depth look at social media and sneaker bot proxies

MichaelSwerston — Fri, 29 May 2020 10:03:29 +0000

In my previous articles I covered the use of residential proxy servers in sneaker bots, however the name residential proxy servers are rarely if ever mentioned when reading about this growing market. Let’s dive in and see what really happens under the hood.

What are sneaker and social media bots?

You can find a brief explanation of sneaker bots on my previous article on a similar topic. What sneaker and social media bots are, are tools that provide an easier way of managing different task by automating them with code. Usually what these bots are meant to do is impersonate a legitimate person accessing the web and completing some tasks. To make these bots even more human like companies use residential proxies which provide a genuine IP address for the bot to use.

While sneaker bots are used for direct financial benefit — by buying and selling sneakers at a higher price margin, social media bots are used by businesses to automatically update, post, and configure social media accounts. Proxies come in to play when a single entity needs to manage multiple media accounts, due to the fact that pages like Facebook, Instagram, Linkedin or Twitter only allow a certain amount of accounts to be used on a single IP address proxy servers and especially residential proxy servers provide great benefits to the cause.

If you are interested in social media managing bots one of the more popular services is Jarvee, it helps you manage and automate tasks on any of your social media accounts.

Proxy servers for automation of these and similar tasks

You can find many different proxy providers marketing their services precisely for sneaker buying, facebook / twitter / instagram / and other social media account management. These proxies are usually marketed as “the best option” for this specific use case, however what they usually fail to mention is the fact that under the hood these “special” types of proxies are nothing other than a regular residential proxy that requires no additional setup for the user or the provider.

So you might be wondering why would these companies market their products as page specific proxies. Well one of the more obvious reasons is to make their product stand out, the proxy market is big and there are many huge providers offering proxy pools that go well over 50 million proxy counts thus for the small players to stand out they need to present their products as unique benefits to specific use cases, even though the proxy servers they offer have no technical advantages over their competitors.
The only way I can think of that would differentiate these proxy servers as being good for Facebook for example is that the specific pool they offer access to is not blocked by facebook. However most proxy providers (or at least the more reputable ones) usually ask the client what purpose the proxies will be used for and provide proxies that are not blocked by that specific website.

Reputable proxy server providers

So what are the better proxy provider choices for managing social media accounts, buying sneakers or any other online automation tools? Here I will be providing a lost of the providers I had personal experience with, and who have experience in this market:

Oxylabs.io One of the biggest proxy providers with over 60 million proxies under their belt, also recently launched a self service, thus making the service more accessible to their clients.

Edit: I have found a discount code in one of the coupon pages, so if you use “COUPONIT” you can get 10% discount on residential proxy plans.

Smartproxy.com This provider recently updated their proxy pool and now has over 40 million proxies in it.
Netnut.io While the provider has only 500k residential proxies in their pool, it is one of the better ones providing choice of many city / state choices.

I’d like to point out that this is not a review of these providers, those are my personal choices providing good pricing options and having the biggest pool sizes and country selections in this market.

Final thoughts

As a proxy provider it does make sense to market your proxies being “the best” for a specific purpose. However, as a buyer you should not be fooled by these tactics, there are plenty providers to choose from and I only mentioned a couple in this article. Buyers should do their research before committing to one specific provider and look around if there are cheaper options that would satisfy their needs. I hope this article shines some light and provides you with information when choosing the service best for you.

Residential proxies: why would you need a residential proxy rotator

MichaelSwerston — Thu, 14 May 2020 14:02:28 +0000

In my previous article I dove into the specific use cases of residential proxies, which are mainly used by businesses these days. Today let’s take a look at a more specific proxy technology called rotating residential proxies — how they work and why would you need them.

As defined in my previous topic residential proxies have direct association with an ISP thus making them the least likely proxy type to be blocked by websites. Rotating your residential proxies would be a useful practice in a couple different projects, as it would make it virtually impossible to get blocked by any website.

What is a rotating residential proxy?

Unlike static proxies which provide you with a single IP address, rotating proxies allow you to access a pool of different IP addresses. This allows you to be connected through a different IP address at any given time. Usually it’s set to change in a set period of time or in some cases depending on how many requests you send out, changing when you reach a certain number of them.

What are the use cases for rotating residential proxies?

While there are probably more use cases for this type of proxy technology I will be covering the two that are most know / most used these days:

Web scraping — is the process that uses the most proxies in general. Web scraping is the automated process of collection data off of the web. Basically, it’s a bot accessing any given website and collecting the set data. This process is resource intensive in terms of proxy servers, especially due to the fact that most websites do not want their data to be collected. There are many reasons for that and we won’t get into them in this article, but basically proxy servers help a web scraping bot disguise itself as a different genuine person every time it rotates the IP address it connects under. And due to the fact that the IP is changed, the website cannot block the connections the scraper makes.

Sneaker bots — another automated process that takes the advantage of anonymity residential proxies provide. Sometimes you might see “sneaker proxies” being marketed, but under the hood they are just simple residential proxies. What sneaker bots are used for is, whenever there’s a new sought after sneaker is released and sold on online shops these, bots can access the said shop and purchase sneakers automatically. You might ask why would you want to do that automatically? Well the answer is pretty simple there are tons of people buying sneakers on their release you might be surprise how competitive this market is. Some people also use these bots to buy sneakers in bulk and resell them at higher prices afterwards. If you have a bot that can act as different people it will help you overcome limitations of the seller website (usually they allow purchase of limited count of sneakers per person).

Rotating residential proxy providers

Residential pool size is one of the more bigger factors that you should look in to when choosing a provider (as well as the quality of proxies themselves) so I will be listing some of the bigger players in this market:
Oxylabs
Smartproxy
Scraperapi

Edit: I have found a discount code in one of the coupon pages, so if you use “COUPONIT” you can get 10% discount on residential proxy plans for Oxylabs.

Final thoughts

While automation is getting bigger and bigger in online tasks such as data collection and even the sneaker buying market, residential proxies are becoming a more popular means to make these processes more reliable. With residential proxy rotators these tasks are becoming even easier to accomplish and if you’re considering to take on such a task you should invest in this technology.

This article was originally posted on medium.

Residential proxies in 2020: what they are used for and how it works

MichaelSwerston — Tue, 12 May 2020 08:45:17 +0000

Let’s start off by defining what a proxy server actually is — it’s a server that acts as an intermediary between the user and the desired web content. What this means is whenever you are connected to a proxy server all of your web traffic is first being transferred through the proxy server and then to the website itself. Due to the fact that all your requests first go through the proxy, the websites you visit will not be able to see your real IP address and thus will think that you reside in the location of the proxy server.

There are a couple of different proxy types, the main ones being: Data center proxies and Residential proxies, in this article we will be focusing on the latter of the two and see what are the most popular use cases for this type of proxy.

What are residential proxies?

The main and only difference between residential and the previously mentioned data center proxies is that this proxy type has direct association with an ISP. Meaning that the IP of the proxy server is provided by a regular ISP (internet service provider) like the one providing you internet access to read this article. Due to this residential proxies are the superior choice in proxies in terms of being undetectable by websites as a proxy server, which is usually one of the major problems in different projects that use proxy servers.

Residential proxy types

There are a couple different choices in residential proxies:

Mobile residential proxies — basically work in the same principle as regular residential proxies, but the IP address is provided by a mobile data provider;
Private proxies — this proxy type only allows a single user to connect to it, thus making it less likely to be blacklisted on the web;
Shared proxies — this proxy type allows multiple different users to connect to the residential proxy at the same time, making it more likely to be blacklisted on the web, but making these proxies much cheaper than private ones.

What are residential proxy use cases:

Travel fare aggregation
Flight and travel companies tend to block bot-like activities on their website. As they provide different pricing options for different regions in the world, you are not able to see that from your primary location. That’s where residential proxies come in handy — providing you access to different parts of the world and less chances of your bots being blocked in the process.

Ad verification
Residential proxies are great for verifying whether or not the ads you set up are working correctly on all regions throughout the world. Also, allowing a business to see whether the ads shown are real, as a lot of bad players tend to fake ads and claim the revenue.

Sneaker proxies
One of the lesser known use cases of this proxy type is using them to purchase sneakers. The basic idea is that on most premium sneaker releases a single user is limited to the amount of sneakers they can buy. Thus the residential proxies + a bot comes to the rescue allowing you to rotate the IP address making it look like many different people are purchasing the items.

Brand protection

The proxies will cover your sensitive data from unwanted eyes, while still being accessible to authorised people remotely.
Tackling counterfeiting and copyright infringement
Retail monitoring helps you pinpoint unauthorised selling of your goods.

Web scraping
Probably the most resource intensive task, when it comes to proxies. A web scraper is essentially a bot, that goes on the website of your choice and collects specific data. Many websites tend to block web scrapers and that’s where proxies come into play — rotating your IP address on every request your bot sends out and makes it virtually impossible for the bot to be blocked.

There are many proxy providers out there and as most of these use cases are business focused I would recommend choosing a proxy provider with that in mind. That being said, Oxylabs is a great residential proxy provider, fulfilling all of the above mentioned use cases and much more.
Edit: I have found a discount code in one of the coupon pages, so if you use “COUPONIT” you can get 10% discount on residential proxy plans.

Final thoughts

The ability to connect to any part of the world is useful for pretty much any business nowadays — it allows easy access to data collection, testing their brand in different regions, and so much more. That’s why residential proxy services have become increasingly popular.

The original article was posted on medium.