<?xml version="1.0" encoding="UTF-8"?>
<rss version="2.0" xmlns:atom="http://www.w3.org/2005/Atom" xmlns:dc="http://purl.org/dc/elements/1.1/">
  <channel>
    <title>DEV Community: IPOasis</title>
    <description>The latest articles on DEV Community by IPOasis (@ipoasis).</description>
    <link>https://dev.to/ipoasis</link>
    <image>
      <url>https://media2.dev.to/dynamic/image/width=90,height=90,fit=cover,gravity=auto,format=auto/https:%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Fuser%2Fprofile_image%2F4210337%2F8ea92e8a-7a14-46bd-8f69-786a03fc47b6.png</url>
      <title>DEV Community: IPOasis</title>
      <link>https://dev.to/ipoasis</link>
    </image>
    <atom:link rel="self" type="application/rss+xml" href="https://dev.to/feed/ipoasis"/>
    <language>en</language>
    <item>
      <title>HTTP Proxy vs SOCKS5: Which Gateway String to Use</title>
      <dc:creator>IPOasis</dc:creator>
      <pubDate>Fri, 20 Nov 2026 09:00:26 +0000</pubDate>
      <link>https://dev.to/ipoasis/http-proxy-vs-socks5-which-gateway-string-to-use-b45</link>
      <guid>https://dev.to/ipoasis/http-proxy-vs-socks5-which-gateway-string-to-use-b45</guid>
      <description>&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fbwm6rtkf67d822kthr3f.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2Fbwm6rtkf67d822kthr3f.png" alt=" " width="799" height="450"&gt;&lt;/a&gt;An HTTP proxy and a SOCKS5 proxy are two ways to hand a request to the same gateway. The HTTP form understands web requests, so it can open a tunnel for an HTTPS page and it can also see a plain HTTP page. SOCKS5 carries the connection without reading it as a web request, and it can ask the gateway to look up the hostname. A script that picks the wrong scheme either leaks its own address or asks the gateway a question that gateway does not answer.&lt;/p&gt;

&lt;h2&gt;
  
  
  What an HTTP proxy forwards
&lt;/h2&gt;

&lt;p&gt;An HTTP proxy is a relay that speaks HTTP. Your client sends it a web request, and the site records the relay address instead of yours. A plain &lt;code&gt;http://&lt;/code&gt; target can travel as a normal proxied request, which means the proxy can read that payload. An &lt;code&gt;https://&lt;/code&gt; target usually goes through HTTP CONNECT. The proxy opens a TCP tunnel to the host and port, then your TLS handshake runs inside that tunnel. The proxy sees the destination host, not the page body.&lt;/p&gt;

&lt;p&gt;That split is why a store on HTTPS and a health check on HTTP can behave differently on the same gateway string. CONNECT is enough for the store. The health check is a request the proxy can parse. A longer walk through the relay itself is in &lt;a href="https://www.ipoasis.com/blog/what-is-an-http-proxy-a-simple-guide-for-beginners" rel="noopener noreferrer"&gt;what an HTTP proxy actually forwards&lt;/a&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  What SOCKS5 changes
&lt;/h2&gt;

&lt;p&gt;A SOCKS5 proxy is a general relay. It accepts a connect command, opens a TCP connection, and shuttles bytes. It does not parse the request as HTTP, so the same gateway string can carry a browser, a mail client, or a script that is not speaking the web.&lt;/p&gt;

&lt;p&gt;The part that changes a scrape is where the hostname is resolved. A URL that starts with &lt;code&gt;socks5://&lt;/code&gt; often resolves the name on your machine, then sends the gateway an IP address. A URL that starts with &lt;code&gt;socks5h://&lt;/code&gt; sends the hostname and asks the gateway to resolve it. The lookup then sits next to the exit IP the site will see. A local lookup can still leave a DNS query on your office resolver, even when the page itself left through the gateway.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;socks5h://USER:PASS@HOST:PORT
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;socks5h&lt;/code&gt; is the form to copy when the dashboard offers SOCKS5 and the job is a website. The trailing &lt;code&gt;h&lt;/code&gt; is the whole difference between a name resolved at your desk and a name resolved at the exit.&lt;/p&gt;

&lt;h2&gt;
  
  
  Which string matches the client
&lt;/h2&gt;

&lt;p&gt;The client library decides the scheme, not the target site. requests, httpx, and a browser extension all want a URL. An HTTP gateway uses a URL that starts with &lt;code&gt;http://&lt;/code&gt;. A SOCKS gateway uses &lt;code&gt;socks5h://&lt;/code&gt; once PySocks, or the equivalent extra, is installed. The dashboard prints one of those two forms. That printed form is the one that belongs in the client.&lt;/p&gt;

&lt;div class="table-wrapper-paragraph"&gt;&lt;table&gt;
&lt;thead&gt;
&lt;tr&gt;
&lt;th&gt;&lt;/th&gt;
&lt;th&gt;HTTP proxy&lt;/th&gt;
&lt;th&gt;SOCKS5 (&lt;code&gt;socks5h&lt;/code&gt;)&lt;/th&gt;
&lt;/tr&gt;
&lt;/thead&gt;
&lt;tbody&gt;
&lt;tr&gt;
&lt;td&gt;What the gateway parses&lt;/td&gt;
&lt;td&gt;Web requests, plus CONNECT for HTTPS&lt;/td&gt;
&lt;td&gt;A connect command and the bytes&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;Where the name is looked up&lt;/td&gt;
&lt;td&gt;Usually at the gateway for CONNECT&lt;/td&gt;
&lt;td&gt;At the gateway, because of the &lt;code&gt;h&lt;/code&gt;
&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;A client it fits&lt;/td&gt;
&lt;td&gt;Browsers, &lt;code&gt;curl&lt;/code&gt;, HTTP libraries&lt;/td&gt;
&lt;td&gt;The same HTTP clients, plus apps that are not HTTP&lt;/td&gt;
&lt;/tr&gt;
&lt;tr&gt;
&lt;td&gt;What you copy&lt;/td&gt;
&lt;td&gt;The &lt;code&gt;http://&lt;/code&gt; string from the dashboard&lt;/td&gt;
&lt;td&gt;The &lt;code&gt;socks5h://&lt;/code&gt; string from the dashboard&lt;/td&gt;
&lt;/tr&gt;
&lt;/tbody&gt;
&lt;/table&gt;&lt;/div&gt;

&lt;p&gt;A long product crawl can use either row. The row that fails is the one you invent. &lt;code&gt;socks5://&lt;/code&gt; on a gateway that was given an HTTP port returns a handshake error. &lt;code&gt;http://&lt;/code&gt; pointed at a SOCKS port does the same. The port in the dashboard and the scheme in the string have to be the pair that page printed.&lt;/p&gt;

&lt;h2&gt;
  
  
  Check the exit before the real host
&lt;/h2&gt;

&lt;p&gt;The exit check is one request to an echo service, through the gateway, before any catalog URL. The printed address is the IP the next site will see. If that address is your office IP, the scheme or the missing dict key sent the call around the gateway, and the real host would have seen you too.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;urllib&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;parse&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;quote&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;

&lt;span class="n"&gt;user&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;quote&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"PROXY_USER"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;safe&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;""&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;password&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;quote&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"PROXY_PASS"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;safe&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;""&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;host&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"PROXY_HOST"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;  &lt;span class="c1"&gt;# host:port from the dashboard&lt;/span&gt;
&lt;span class="n"&gt;proxy&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"socks5h://{user}:{password}@{host}"&lt;/span&gt;

&lt;span class="n"&gt;session&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Session&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;session&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;trust_env&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;False&lt;/span&gt;
&lt;span class="n"&gt;session&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;proxies&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"http"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;proxy&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"https"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;proxy&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;session&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"https://api.ipify.org"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;30&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;quote&lt;/code&gt; keeps a password that contains &lt;code&gt;@&lt;/code&gt; or &lt;code&gt;:&lt;/code&gt; inside the userinfo. A raw &lt;code&gt;@&lt;/code&gt; splits the URL, and the gateway answers as if the login were wrong. &lt;code&gt;trust_env = False&lt;/code&gt; stops a leftover &lt;code&gt;HTTPS_PROXY&lt;/code&gt; in the shell from replacing the string you just set. Both dict keys carry the same string, because requests picks the key from the target URL. A missing &lt;code&gt;https&lt;/code&gt; key sends the echo service out from your own address.&lt;/p&gt;

&lt;p&gt;You point the same session at the real host only after the printed address matches the country on the proxy user. The gigabytes start on that second request.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where the country and the session live
&lt;/h2&gt;

&lt;p&gt;The country and the session mode usually live in the username, not in the scheme. A rotating user gives the next request a new exit IP from the pool. A sticky user holds one exit IP for one job, then lets it go. HTTP and SOCKS5 can both carry that username. Changing &lt;code&gt;http://&lt;/code&gt; to &lt;code&gt;socks5h://&lt;/code&gt; does not change the country, and it does not pin the IP.&lt;/p&gt;

&lt;p&gt;A price list can use the rotating user. Each page leaves from another household, and one connection does not absorb the whole list. A checkout uses the sticky user, so the item page and the payment page share a visitor. IPOasis sticky sessions on dynamic residential hold that address for up to 120 minutes. When the window ends, the next request can leave from a different household IP. The dashboard builds the username. That string goes in &lt;code&gt;PROXY_USER&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where the gateway string comes from
&lt;/h2&gt;

&lt;p&gt;The host, the port, the username, and the password come from the provider dashboard, already paired as an HTTP string or a SOCKS5 string. IPOasis lists &lt;a href="https://www.ipoasis.com/proxy/dynamic-residential" rel="noopener noreferrer"&gt;dynamic residential proxies&lt;/a&gt; from $0.78 per GB, with HTTP and SOCKS5 on the same balance. Unused gigabytes stay on the balance, so the exit check does not burn a monthly plan. You copy the pair the dashboard printed, run the echo once, and only then open the host you actually need.&lt;/p&gt;

&lt;h2&gt;
  
  
  FAQ
&lt;/h2&gt;

&lt;h3&gt;
  
  
  What is the difference between an HTTP proxy and SOCKS5?
&lt;/h3&gt;

&lt;p&gt;An HTTP proxy speaks web requests and opens a CONNECT tunnel for HTTPS. SOCKS5 carries the TCP connection without parsing it as HTTP, so the same gateway can serve clients that are not browsers. For a scrape, the practical difference is the URL scheme and where the hostname is resolved.&lt;/p&gt;

&lt;h3&gt;
  
  
  What does socks5h mean?
&lt;/h3&gt;

&lt;p&gt;&lt;code&gt;socks5h&lt;/code&gt; means the proxy resolves the hostname. The client sends the name, and the lookup happens at the gateway, next to the exit IP. Plain &lt;code&gt;socks5&lt;/code&gt; often resolves the name on your machine first, which can leave a DNS query on your own resolver.&lt;/p&gt;

&lt;h3&gt;
  
  
  Can one username work on both?
&lt;/h3&gt;

&lt;p&gt;One username can ride on either scheme when the provider serves HTTP and SOCKS5 from the same account. The host and port still have to match the scheme. The dashboard shows that pair. The country and the sticky or rotating mode stay in the username either way.&lt;/p&gt;

&lt;h3&gt;
  
  
  Why did the echo service print my office IP?
&lt;/h3&gt;

&lt;p&gt;The call missed the gateway. A missing &lt;code&gt;https&lt;/code&gt; key, an &lt;code&gt;HTTPS_PROXY&lt;/code&gt; value still set in the shell, or a scheme that does not match the port will all send that request from your own address. The printed IP is the check. The catalog request waits until that IP is the exit you picked.&lt;/p&gt;

</description>
      <category>python</category>
      <category>webdev</category>
      <category>tutorial</category>
      <category>networking</category>
    </item>
    <item>
      <title>How to Use a Proxy in Python requests</title>
      <dc:creator>IPOasis</dc:creator>
      <pubDate>Wed, 11 Nov 2026 03:17:19 +0000</pubDate>
      <link>https://dev.to/ipoasis/how-to-use-a-proxy-in-python-requests-3b2m</link>
      <guid>https://dev.to/ipoasis/how-to-use-a-proxy-in-python-requests-3b2m</guid>
      <description>&lt;p&gt;&lt;a href="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F7iwckbk3su864y1bmaw7.png" class="article-body-image-wrapper"&gt;&lt;img src="https://media2.dev.to/dynamic/image/width=800%2Cheight=%2Cfit=scale-down%2Cgravity=auto%2Cformat=auto/https%3A%2F%2Fdev-to-uploads.s3.us-east-2.amazonaws.com%2Fuploads%2Farticles%2F7iwckbk3su864y1bmaw7.png" alt=" " width="799" height="450"&gt;&lt;/a&gt;&lt;br&gt;
A proxy in Python requests is a gateway URL stored on the Session. Each call goes to that gateway first, and the site sees the exit IP instead of your machine. The dict keys name the scheme of the URL you are calling, so &lt;code&gt;http&lt;/code&gt; and &lt;code&gt;https&lt;/code&gt; both need the gateway string. An empty &lt;code&gt;https&lt;/code&gt; key sends that call out from your own address, and the echo service prints your office IP.&lt;/p&gt;
&lt;h2&gt;
  
  
  What the proxies dict controls
&lt;/h2&gt;

&lt;p&gt;The proxies dict is the map from a target scheme to a gateway URL. The key is &lt;code&gt;http&lt;/code&gt; or &lt;code&gt;https&lt;/code&gt; on the page you are opening, not the kind of proxy. A SOCKS gateway still belongs under the &lt;code&gt;https&lt;/code&gt; key when the target starts with &lt;code&gt;https://&lt;/code&gt;.&lt;/p&gt;

&lt;p&gt;That split is why a working &lt;code&gt;http&lt;/code&gt; entry can still leak the real address. requests looks up the key from the target URL. A missing &lt;code&gt;https&lt;/code&gt; entry means the library opens the store, the search page, or the echo service directly, and the gigabytes on the gateway never move.&lt;/p&gt;
&lt;h2&gt;
  
  
  Add SOCKS to requests
&lt;/h2&gt;

&lt;p&gt;SOCKS support in requests comes from PySocks, and the extra install pulls that package in.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight shell"&gt;&lt;code&gt;pip install &lt;span class="s2"&gt;"requests[socks]"&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A gateway URL that starts with &lt;code&gt;socks5h://&lt;/code&gt; asks the gateway to resolve the hostname. The lookup sits next to the exit IP the site will see, so the name and the address belong to the same hop. An HTTP gateway uses a URL that starts with &lt;code&gt;http://&lt;/code&gt; instead. The dashboard prints one of those two forms. Copy the form it prints.&lt;/p&gt;

&lt;h2&gt;
  
  
  Put the gateway on a Session
&lt;/h2&gt;

&lt;p&gt;A Session is the object that keeps the gateway, the headers, and the cookie jar together. Later calls in the same run reuse that jar, so a second page still carries the first response.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="k"&gt;from&lt;/span&gt; &lt;span class="n"&gt;urllib&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;parse&lt;/span&gt; &lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;quote&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;

&lt;span class="n"&gt;user&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;quote&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"PROXY_USER"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;safe&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;""&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;password&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="nf"&gt;quote&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"PROXY_PASS"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;safe&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="s2"&gt;""&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;host&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"PROXY_HOST"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;  &lt;span class="c1"&gt;# host:port from the dashboard&lt;/span&gt;
&lt;span class="n"&gt;proxy&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;f&lt;/span&gt;&lt;span class="s2"&gt;"socks5h://{user}:{password}@{host}"&lt;/span&gt;

&lt;span class="n"&gt;session&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Session&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="n"&gt;session&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;trust_env&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;False&lt;/span&gt;
&lt;span class="n"&gt;session&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;proxies&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"http"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;proxy&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="s2"&gt;"https"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;proxy&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;

&lt;span class="n"&gt;exit_ip&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;session&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"https://api.ipify.org"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;30&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;
&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;exit_ip&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;quote&lt;/code&gt; keeps a password that contains &lt;code&gt;@&lt;/code&gt; or &lt;code&gt;:&lt;/code&gt; inside the userinfo. A raw &lt;code&gt;@&lt;/code&gt; splits the URL early, and the gateway answers as if the login were wrong. &lt;code&gt;trust_env = False&lt;/code&gt; stops a leftover &lt;code&gt;HTTPS_PROXY&lt;/code&gt; in the shell from replacing the gateway you just set. The timeout ends the wait when the gateway does not answer, instead of leaving the process hung.&lt;/p&gt;

&lt;p&gt;You point the same Session at the real host only after the printed address matches the country you picked. The gigabytes start on that second request.&lt;/p&gt;

&lt;h2&gt;
  
  
  Match the username to the job
&lt;/h2&gt;

&lt;p&gt;The username is the field most gateways use for the country and the session mode. A rotating user gives the next request a new exit IP from the pool. A sticky user holds one exit IP for a single job, then lets it go.&lt;/p&gt;

&lt;p&gt;A long product list can use the rotating user. Each page leaves from another household, and one connection does not absorb the whole list. A checkout uses the sticky user. The item page and the payment page then share a visitor, and the cart stays attached to that address. IPOasis sticky sessions hold that address for up to 120 minutes. When the window ends, the next request can leave from a different household IP. The dashboard builds the username. That string goes in &lt;code&gt;PROXY_USER&lt;/code&gt;.&lt;/p&gt;

&lt;h2&gt;
  
  
  Read a refusal before you swap the exit
&lt;/h2&gt;

&lt;p&gt;A 200 from the echo service means the gateway accepted the login and the exit country is the one you set. A 403 from the store is a separate reply. The exit IP can be a household address and the edge can still refuse the request. &lt;a href="https://www.ipoasis.com/blog/403-forbidden-cloudflare-why-scrapers-get-it" rel="noopener noreferrer"&gt;How to read a Cloudflare 403&lt;/a&gt; keeps the status and the headers in view before the gateway string changes.&lt;/p&gt;

&lt;h2&gt;
  
  
  Where the gateway string comes from
&lt;/h2&gt;

&lt;p&gt;The host, the port, the username, and the password come from the provider dashboard. IPOasis lists &lt;a href="https://www.ipoasis.com/proxy/dynamic-residential" rel="noopener noreferrer"&gt;dynamic residential proxies&lt;/a&gt; from $0.78 per GB, with HTTP and SOCKS5 on the same balance. Unused gigabytes stay on the balance, so a short exit check does not burn a monthly plan.&lt;/p&gt;

&lt;h2&gt;
  
  
  FAQ
&lt;/h2&gt;

&lt;h3&gt;
  
  
  How do you set a proxy in Python requests?
&lt;/h3&gt;

&lt;p&gt;You set a proxy in Python requests by putting a gateway URL on &lt;code&gt;Session.proxies&lt;/code&gt; for both &lt;code&gt;http&lt;/code&gt; and &lt;code&gt;https&lt;/code&gt;. The Session then sends each call to that gateway, and the site sees the exit IP. One &lt;code&gt;requests.get&lt;/code&gt; with a &lt;code&gt;proxies&lt;/code&gt; argument works for a single call, and it drops the cookie jar when the next call starts.&lt;/p&gt;

&lt;h3&gt;
  
  
  What is socks5h?
&lt;/h3&gt;

&lt;p&gt;&lt;code&gt;socks5h&lt;/code&gt; is a SOCKS5 URL whose hostname is resolved at the gateway. The DNS lookup and the exit IP sit on the same hop, so the site and the resolver see one network. The extra package is &lt;code&gt;requests[socks]&lt;/code&gt;. An HTTP gateway uses an &lt;code&gt;http://&lt;/code&gt; URL instead, and the dashboard shows which form you have.&lt;/p&gt;

&lt;h3&gt;
  
  
  Why does the Session print my real IP?
&lt;/h3&gt;

&lt;p&gt;The Session prints your real IP when the &lt;code&gt;https&lt;/code&gt; key is missing, or when &lt;code&gt;HTTPS_PROXY&lt;/code&gt; in the shell replaced the gateway. requests picks the dict entry from the target scheme, so an &lt;code&gt;https://&lt;/code&gt; echo skips an &lt;code&gt;http&lt;/code&gt;-only map. &lt;code&gt;trust_env = False&lt;/code&gt; keeps the shell value out, and both keys need the same gateway string.&lt;/p&gt;

&lt;h3&gt;
  
  
  What is a sticky session?
&lt;/h3&gt;

&lt;p&gt;A sticky session keeps one exit IP for a single job, then releases it. A checkout stays one visitor from the item page to the payment page, so the cart does not reset. A rotating user is the other setting, and it gives the next URL a new household address. The dashboard username is where that choice is stored.&lt;/p&gt;

&lt;h3&gt;
  
  
  Which environment variables does the example read?
&lt;/h3&gt;

&lt;p&gt;The example reads &lt;code&gt;PROXY_USER&lt;/code&gt;, &lt;code&gt;PROXY_PASS&lt;/code&gt;, and &lt;code&gt;PROXY_HOST&lt;/code&gt; from the environment. The host value already includes the port the dashboard printed. Keeping the password out of the source file means a pasted snippet does not carry a live login, and &lt;code&gt;quote&lt;/code&gt; still encodes &lt;code&gt;@&lt;/code&gt; and &lt;code&gt;:&lt;/code&gt; before the URL is built.&lt;/p&gt;

</description>
      <category>python</category>
      <category>tutorial</category>
      <category>webdev</category>
      <category>beginners</category>
    </item>
    <item>
      <title>What an HTTP request looks like after it passes through an HTTP proxy</title>
      <dc:creator>IPOasis</dc:creator>
      <pubDate>Thu, 29 Oct 2026 07:15:56 +0000</pubDate>
      <link>https://dev.to/ipoasis/what-an-http-request-looks-like-after-it-passes-through-an-http-proxy-de5</link>
      <guid>https://dev.to/ipoasis/what-an-http-request-looks-like-after-it-passes-through-an-http-proxy-de5</guid>
      <description>&lt;p&gt;An HTTP proxy takes the request your script wrote and sends it out from another address. The script's socket ends at the proxy. The site's socket starts at the proxy exit.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the script sends to the proxy
&lt;/h2&gt;

&lt;p&gt;&lt;code&gt;requests&lt;/code&gt; opens the proxy when the target URL is &lt;code&gt;http://&lt;/code&gt; and &lt;code&gt;proxies&lt;/code&gt; has an &lt;code&gt;http&lt;/code&gt; entry. The request line is the full URL, which is what the proxy dials.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;GET http://checkip.amazonaws.com/ HTTP/1.1
Host: checkip.amazonaws.com
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;A direct request line is only the path, because the socket is already on the site. Through a proxy the request line is the absolute URL.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;

&lt;span class="k"&gt;def&lt;/span&gt; &lt;span class="nf"&gt;public_ip&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;proxy_url&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt; &lt;span class="o"&gt;|&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="o"&gt;-&amp;gt;&lt;/span&gt; &lt;span class="kt"&gt;str&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;proxies&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="kc"&gt;None&lt;/span&gt;
    &lt;span class="k"&gt;if&lt;/span&gt; &lt;span class="n"&gt;proxy_url&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;proxies&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"http"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;proxy_url&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;
    &lt;span class="n"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;
        &lt;span class="s2"&gt;"http://checkip.amazonaws.com"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;proxies&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="n"&gt;proxies&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
        &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt;
    &lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;raise_for_status&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;return&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;strip&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;

&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"direct "&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nf"&gt;public_ip&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="kc"&gt;None&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"proxied"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="nf"&gt;public_ip&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"PROXY_URL"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;PROXY_URL&lt;/code&gt; is that proxy URL, username included.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;http://USERNAME:PASSWORD@HOST:PORT
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;The first line is the machine running the script. The second is the exit.&lt;/p&gt;

&lt;p&gt;A proxy that asks for a login gets &lt;code&gt;Proxy-Authorization&lt;/code&gt; on this request. The site gets &lt;code&gt;Host&lt;/code&gt; and the path.&lt;/p&gt;

&lt;h2&gt;
  
  
  What the site records as the client
&lt;/h2&gt;

&lt;p&gt;The site's connection log shows the exit. That is the second printed line. The headers on that request are the ones &lt;code&gt;requests&lt;/code&gt; built, without the proxy login.&lt;/p&gt;

&lt;p&gt;Matching lines mean the call left from an address you already had. Check that &lt;code&gt;PROXY_URL&lt;/code&gt; is set, and that its host is the provider gateway, before reading anything into the echo.&lt;/p&gt;

&lt;h2&gt;
  
  
  Why a second call can leave from another exit
&lt;/h2&gt;

&lt;p&gt;A pool gateway picks the exit when it handles the request. No session id in the username, and the next request can leave from another exit. Host and port in &lt;code&gt;PROXY_URL&lt;/code&gt; stay put. The pick happens after the proxy reads the username.&lt;/p&gt;

&lt;p&gt;&lt;code&gt;requests.Session&lt;/code&gt; keeps a connection pool and a cookie store in the process. The second request still carries the same proxy URL. The exit is still the gateway's pick.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight python"&gt;&lt;code&gt;&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;
&lt;span class="k"&gt;import&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;

&lt;span class="n"&gt;proxy_url&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;os&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;environ&lt;/span&gt;&lt;span class="p"&gt;[&lt;/span&gt;&lt;span class="s2"&gt;"PROXY_URL"&lt;/span&gt;&lt;span class="p"&gt;]&lt;/span&gt;

&lt;span class="k"&gt;with&lt;/span&gt; &lt;span class="n"&gt;requests&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nc"&gt;Session&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt; &lt;span class="k"&gt;as&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
    &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;proxies&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;update&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;{&lt;/span&gt;&lt;span class="s2"&gt;"http"&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt; &lt;span class="n"&gt;proxy_url&lt;/span&gt;&lt;span class="p"&gt;}&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
    &lt;span class="k"&gt;for&lt;/span&gt; &lt;span class="n"&gt;n&lt;/span&gt; &lt;span class="k"&gt;in&lt;/span&gt; &lt;span class="nb"&gt;range&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="mi"&gt;2&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;:&lt;/span&gt;
        &lt;span class="n"&gt;response&lt;/span&gt; &lt;span class="o"&gt;=&lt;/span&gt; &lt;span class="n"&gt;client&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;get&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="s2"&gt;"http://checkip.amazonaws.com"&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;timeout&lt;/span&gt;&lt;span class="o"&gt;=&lt;/span&gt;&lt;span class="mi"&gt;20&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;raise_for_status&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
        &lt;span class="k"&gt;print&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="n"&gt;n&lt;/span&gt;&lt;span class="p"&gt;,&lt;/span&gt; &lt;span class="n"&gt;response&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="n"&gt;text&lt;/span&gt;&lt;span class="p"&gt;.&lt;/span&gt;&lt;span class="nf"&gt;strip&lt;/span&gt;&lt;span class="p"&gt;(&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;&lt;span class="p"&gt;)&lt;/span&gt;
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;Two addresses from one username means the gateway was free to move the exit. One address means it reused an exit, or the username already carries a session id. Providers put that id in the username. A new username starts a new session.&lt;/p&gt;

&lt;p&gt;A hold is a number of minutes on that username. &lt;a href="https://www.ipoasis.com/proxy/dynamic-residential" rel="noopener noreferrer"&gt;IPOasis dynamic residential&lt;/a&gt; sets it from 1 to 120 minutes. After that, the next request can move.&lt;/p&gt;

&lt;h2&gt;
  
  
  What SOCKS5 changes in this picture
&lt;/h2&gt;

&lt;p&gt;SOCKS5 uses a handshake instead of an absolute-URL request line. The script still dials the proxy, the proxy still dials the site, and the site still logs the exit.&lt;br&gt;
&lt;/p&gt;

&lt;div class="highlight js-code-highlight"&gt;
&lt;pre class="highlight plaintext"&gt;&lt;code&gt;socks5h://USERNAME:PASSWORD@HOST:PORT
&lt;/code&gt;&lt;/pre&gt;

&lt;/div&gt;



&lt;p&gt;&lt;code&gt;socks5h&lt;/code&gt; sends the hostname. The proxy resolves it. &lt;code&gt;socks5&lt;/code&gt; resolves it locally and sends an IP. Either scheme needs &lt;code&gt;PySocks&lt;/code&gt;. The echo URL stays &lt;code&gt;http://checkip.amazonaws.com&lt;/code&gt;.&lt;/p&gt;

</description>
      <category>programming</category>
      <category>python</category>
      <category>networking</category>
      <category>proxy</category>
    </item>
  </channel>
</rss>
