DEV Community: Ashish R Bhandari

SomeThing About CSS - Some CSS Things I Learnt

Ashish R Bhandari — Mon, 22 Mar 2021 14:25:43 +0000

Small-Small Things Like

Box Shadow
Good Font
Border
Border Only on One Side
Hover Effect
Transitions
Transform
Filter: Grayscale

can Actually Make your UI Look Really Good

Few Things I Learnt (I will made more)

Making Div At Center (From Left Right) Mostly Used in Container

.container {
    margin: auto;
    width: 500px;
}

Run a Program(like App Server) like a SystemD Service

Ashish R Bhandari — Mon, 22 Mar 2021 13:59:45 +0000

A Very Simple Example of a SystemD COnfigured Service.
Inorder to run a Program like a Service in Background in 2 SImple Steps

Step1: Create a File inside Dir
/lib/systemd/system/
wiith Extension as .service

Step 2: Add the Service Details and Behaviour Etc like creating a Setting File to Tell SystemD what to do

#Section 1
# Provide Details like Description and After is Requried by Your Service i.e if it requires a Interface then below After is Helpfull.
[Unit]
Description=A Simple Poster App ( A Very Cheap Posting System like Twitter) User can Create Post, Delete Post, View Other Posts etc
Documentation=https://app.poster.com
After=network.target

# Section 2 
# ENV Variables can be created here
# Very VVV IMP `WorkingDirectory`
# Most Obvious Command How to Start the Program `ExecStart`
# `Restart` if the Program crashes due to some reason u have a failover.
# User Optional and Depending on Requirement.

[Service]
Environment=NODE_PORT=81
Type=simple
User=root
ExecStart=/usr/bin/node /usr/local/src/apps/app.poster_v2/app.js
WorkingDirectory=/usr/local/src/apps/app.poster_v2
Restart=on-failure

## When and Where this Service should Start
[Install]
WantedBy=multi-user.target

Example:


#File Content
[Unit]
Description=A Simple Poster App ( A Very Cheap Posting System like Twitter) User can Create Post, Delete Post, View Other Posts etc
Documentation=https://app.poster.com
After=network.target

[Service]
Environment=NODE_PORT=81
Type=simple
User=root
ExecStart=/usr/bin/node /usr/local/src/apps/app.poster_v2/app.js
WorkingDirectory=/usr/local/src/apps/app.poster_v2
Restart=on-failure

[Install]
WantedBy=multi-user.target

(My) Simple Home Wifi Router Login Session Management(Cookie)

Ashish R Bhandari — Sat, 20 Feb 2021 07:28:24 +0000

I saw this, while i was making a change in my Wifi Router Setting,

If i think from a Security Perspective:
Basically My Wifi Router Web Interface is very unsecure, well i think it should use a better approach, this is the worst possible,
I know about JWT, this is PWT(Plain Web Tokens) 😬
Let's see why...

Snap1: Shows Cookie With Content containing Username and Password in Bas64

So Over here, we can see The Cookie Header contains a HTTP Header (Authorization) in form a Cookie name&value and the value contains the username&password that i just entered.

Base64!!! Hmmm Nice

Snap2: A Closer look at Cookie Containing Username and Password in Bas64

Snap3: Dev Tools are amazing the Network Tab has a Column Initiator Tells from where this the Request Originated, the whole Stack Trace (Beginning from the Click Event Fired)

Snap3: Finally the Code how cookie was set, Basically by Client JS

If i think from a Router's Actual Work:
Then in that case it has nothing to do with the Web Interface.
Web Interface are for us to manage few things and that too mostly used in Internal Network

Behind the Scenes (My Mind Conversation & Thoughts)

But if someone who is part of the network but does not have access can SNIFF the Traffic and capture these details
But this is totally contradictory, since he could have SNIFFED the Initial Login Traffic and received the details and Also the Web Interface is usually over Plain HTTP.

Get Product Prices via JS

Ashish R Bhandari — Sat, 23 Jan 2021 02:27:55 +0000

Get the Amazon Product Price (& Highlight The Price)
A Quick One that I Figured Out

let price_of_product = document.querySelector(`span[id*="priceblock_"][class*="a-size"][class*="priceBlock"]`);

price_of_product.style.backgroundColor = "blue";   

console.log(price_of_product.innerText);

OutPut:

A Bit More Styling

Below i am using cssText, This helps to Set the Style Attribute with Plain CSS way of Styling
Rather then using backgroundColor (JS way) as Normal CSS attr background-color (CSS way)

let price_of_product = document.querySelector(`span[id*="priceblock_"][class*="a-size"][class*="priceBlock"]`);

price_of_product.style.cssText = "background-color: orange; padding: 5px; border-radius: 10px; font-size: 24px!important;";   

console.log(price_of_product.innerText);

Output:

+++ Added For Flipkart

Worst JS Snippet But Still

let price_el = document.querySelector("#container>div>div:nth-child(3)>div:nth-child(1)>div:nth-child(2)>div:nth-child(2)>div:nth-child(1)>div:nth-child(3)>div:nth-child(1)>div:nth-child(1)>div:nth-child(1)");
if (price_el) {
    price_el.style.backgroundColor = "red";
} else {
    let price_el2 = document.querySelector("#container>div>div:nth-child(3)>div:nth-child(1)>div:nth-child(2)>div:nth-child(2)>div:nth-child(1)>div:nth-child(4)>div:nth-child(1)>div:nth-child(1)>div:nth-child(1)");
    price_el2.style.backgroundColor = "red";
}

Copy this and add it in the console of any Flipkart Product Page it will highlight the Product Price...

Later u can use it in Python Script
Mainly the CSS Selector Path

and then can automate the process...

A Small Wrong Doc Section (or Correct me if I am Wrong)

Ashish R Bhandari — Sat, 16 Jan 2021 07:56:28 +0000

Please Feel free to Comment or Point of Errors or Write your Own View Point
Also Correct me if the Article is WRONG or a OLDER one

While I was reading this Article:
(Nginx Doc) [https://docs.nginx.com/nginx/admin-guide/security-controls/terminating-ssl-http/]

I Found a Wrong Description in the Doc at Section:
(Nginx Name Based Https Servers) [https://docs.nginx.com/nginx/admin-guide/security-controls/terminating-ssl-http/#name-based-https-servers]

The Doc says that below Things
(Doc Says) []

According to me
In Simple Words, the server_name is the one which will basically checks the SNI as well as the Host Field.

Is it a Old Doc then it should changed.

I will add the Practical Setup to explain about the server_name working and basic nginx setup soon...

Error due to (.) in Domain Name End[www.ashish.com.]

Ashish R Bhandari — Thu, 14 Jan 2021 07:02:40 +0000

1) I saw a Very Interesting behavior today.
Let me explain it with an example

https://www.google.com./
https://www.google.com/

Both URL are ALMOST Same the difference is a (.)

The Above Both URL will access the same Resource, Also If we do the DNS LOOKUP, we will GET the same IP [Yes it will change depending on DNS and it's config] [To Check it Please Use a Domain that Points to only one IP]

A Quick Look [example.com]

C:\Users\@sh>nslookup example.com
Server:  dns.google
Address:  8.8.8.8

Non-authoritative answer:
Name:    example.com
Addresses:  2606:2800:220:1:248:1893:25c8:1946
          93.184.216.34


C:\Users\@ash>nslookup example.com.
Server:  dns.google
Address:  8.8.8.8

Non-authoritative answer:
Name:    example.com
Addresses:  2606:2800:220:1:248:1893:25c8:1946
          93.184.216.34

So The Main Point to Focus, is yes a "." at End is accepted in DNS Query and So in Chrome or FireFox Browser.

The Same Happens When the Browser tries to access such a URL.

The Results were Hilarious.

The Browser will Treat Both as 2 Different Domains [As per what i understood from it].
Which means that if i Go to https://www.google.com./
Then Browser will not send any Cookie that is for "Google.com" or "www.google.com"
[i.e Any Cookie Set for google.com and it's subdomain]

ALSO

Browser will not send any Cookie when the Request is made to google.com and it's subdomain via a "www.google.com." [ending with a DOT(.)]
IF the Cookie SameSite Attribute is Not None.
Because it will treat it as a CSR [Cross Site Request]
and only Cookies with SameSite Attribute as None are sent by the Browser

[Also if the Request is not a Top-Level Navigation, In Other words if the Link Accessed does not Show in Browser's URL Bar Cookies will not be Sent]

Which Means if you are logged in to Google and then on the URL Bar you access

This is the Result I Got

Let's do some Debugging

The Browser Says that it cannot set the New Cookie Received From "www.google.com." which is "www.google.com", but the Browser will match it with "www.google.com." [Since this is what was requested by the user and this is what is shown in the Browser URL Bar]

Therefore, No Cookie which was for "www.google.com" was sent and the Received Cookies were also discarded as the domain or the URL Path (Hard to tell what!!) did not qualify for "www.google.com." [With a (.)]

I am Already Logged in Therefore when I click on sign in

Note:
Please Feel Free to Comment your own View points and correct me

Notes on Proxies

Ashish R Bhandari — Sat, 09 Jan 2021 14:27:01 +0000

Reverse Proxy

** Some Good Quotes **

Not all proxies are load balancers, but the vast majority of proxies perform load balancing as a primary function.

HTTP(S) Forward Proxy (🤔 Types)

Ashish R Bhandari — Sat, 09 Jan 2021 02:42:05 +0000

I am Keeping this Post as Simple as Possible.

Feel Free to Comment and Point Out Problems, Suggestions and Your Point of View.

So There are 3 Types we can Say
1) HTTP Proxy

(Does Not Support CONNECT Method, In Simple words it cannot connect to HTTPS Sites)

Example:

2) Connect Proxy

(Supports CONNECT Method, Meaning they can Let user to connect to HTTPS Sites BUT cannot see anything as the data is Encrypted).

Example:

Tiny Proxy Have a Lot More Capability Then Just a CONNECT Proxy, it can be used as a FORWARD PROXY, REVERSE PROXY, TRANSPARENT PROXY

Mozilla Docs for Proxy & Tunneling

3) HTTPS Proxy

MitmProxy is the Perfect Example Here.
BurpSuite The All Time Favourite.

A Proxy that can Intercept & Inspect HTTPS Sites.
It can See all the HTTPS Data as Plain HTTP Request, Response
It is Also Called as
SSL/TLS Termination Proxy
HTTPS Inspection Proxy (Not Exactly, But HTTPS Inspection is a Term)
SSL Proxy and many more...

LVM (Quick Snippet)

Ashish R Bhandari — Tue, 05 Jan 2021 07:47:32 +0000

Physical Volume (PV)
Volume Group (VG)
Logical Volume (LV)

All PV COMMANDS

pvcreate /dev/sdb2

All VG COMMANDS

Syntax: vgcreate NEW_VG_NAME PV_NAME

vgcreate ubuntu-vg /dev/sdb2

Syntax: vgextend OLD_VG_NAME PV_NAME

vgextend ubuntu-vg /dev/sdb3

ALL LV COMMANDS

Syntax: lvcreate -n NEW_LV_NAME -L SIZE(G,M) EXISITING_VG_NAME

lvcreate -n app-log-lv -L 12G ubuntu-vg

Syntax: lvextend [LEFT TO ADD]

[LEFT TO ADD]

lvreduce

# Check First
lvs

LV   VG   Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
app-log-lv ubuntu-vg -wi-ao---- <12.00g


# Perform Reduce
lvreduce --resizefs -L -1G ubuntu-vg/app-log-lv


# Check Again
lvs

LV   VG   Attr       LSize   Pool Origin Data%  Meta%  Move Log Cpy%Sync Convert
app-log-lv ubuntu-vg -wi-ao---- <11.00g

A Quick Setup

Step1: Make Sure a Disk is Present or Create it using pvcreate

pvcreate /dev/sdb2

Note: Check all PVs
commands: pvs , pvscan , pvdisplay

Step2: Make Sure a Volume Group(VG) is Present or Create it using vgcreate

vgcreate ubuntu-vg /dev/sdb2

Note: Check all VGs
commands: vgs , vgscan , vgdisplay

Step3: Make Sure a Logical Volume(LV) is Present or Create it using lvcreate

lvcreate -n app-log-lv -L 12G ubuntu-vg

Note: Check all LVs
commands: lvs , lvscan , lvdisplay

Step4: Make Sure it has a File System(fs) else provide one using mkfs.ext4 LV_PATH

#Get the LV_PATH
lvdisplay
#OutPut [It will show the LV Path]

  --- Logical volume ---
  LV Path                /dev/ubuntu-vg/app-log-lv
  LV Name                app-log-lv
  VG Name                ubuntu-vg
  LV UUID                bGNxCm-Beki-UX3D-HSXm-NU3k-YWs0-1ru2Tp
  LV Write Access        read/write
  LV Creation host, time tester, 2020-11-04 10:52:02 +0530
  LV Status              available
  # open                 1
  LV Size                12 GiB
  Current LE             2092
  Segments               1
  Allocation             inherit
  Read ahead sectors     auto
  - currently set to     256
  Block device           253:8

Give File System (fs)

mkfs.ext4 /dev/ubuntu-vg/app-log-lv

Note: Check all mkfs
commands: mkfs.ext4, mkfs -t ext4

Step5: Make Sure it is MOUNTED(mount) if NOT mount it to a DIRECTORY

so that any changes made in directory is reflected in the Disk

Syntax: mount LV_Path DIR_PATH

# Create Dir If You don't have: 
mkdir /var/log/app-logs
mount /dev/ubuntu-vg/app-log-lv /var/log/app-logs

Step6: Make Sure it is PERMANENTLY MOUNTED(fstab) if NOT mount it PERMANENTLY

Quickest: Copy New Data from mtab to fstab

cat /etc/mtab

/dev/mapper/ubuntu-vg-app-log-lv /var/log/app-logs ext4 rw,relatime,data=ordered 0 0

# Add some Here
vim fstab

/dev/mapper/ubuntu-vg-app-log-lv /var/log/app-logs ext4 rw,relatime,data=ordered 0 0

Restart & Check

Simple Connect Proxy

Ashish R Bhandari — Sun, 03 Jan 2021 01:57:09 +0000

Over Here, we are Talking about Control that a Proxy Have (i.e Proxy used in a Internal Network to APPLY RESTRICTIONS)

Refer Here : What All Things are Possible in a Proxy(Forward Proxy)

So A Connect Proxy basically means that yes it supports connecting to HTTPS Website, Because there were Proxies which did not Support CONNECT Method i.e were not able to connect to HTTPS site.

So we are talking about a Proxy that handles CONNECT Method and then basically creates a TCP Socket to the Remove Server and then their are 2 TCP SOCKETS as shown below

CLIENT -----> Proxy ------> SERVER
TCP SOCKET =====> TCP SOCKET

The Proxy is going to send any data received from client as it is to server socket. because after Successful TLS Connection it is encrypted, the Proxy cannot interpret the data.

SO then What all things a CONNECT PROXY can do

Let's look at the Data that it Has

All Possible Data
1) Client IP
2) Remote Server Domain, Port and After Resolution IP Address
3) User-Agent
4) Time [When The Request Came]
5) Authentication Details [If Proxy Requires Authentication to Identify User and Then Allows]
6) Web Category/URL Category [I will later Add a Link For Reference]

A Small Note on Web Category : But in Simple Words, There is a Database of Websites which are Added to a Group of Category Example:
A Simple Table

Domain Name	Category
google.com	`SearchEngine`
facebook.com	`Social Networking`
porn.com	`Pornography`, `Adult Content`

A Quick View at Request By Client for www.example.com via a Corporate Proxy

Client[192.168.0.167] Sends Header:

CONNECT www.example.com:443 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:84.0) Gecko/20100101 Firefox/84.0
Proxy-Connection: keep-alive
Connection: keep-alive
Host: www.example.com:443

[WebCategory Check]: [Website:www.example.com] => [Category List: Test Driven Sites, Safe Sites]

Now Let's Look at the Data that the Proxy Has or May or can Have

Minimum Data

1) Client IP
2) Remote Server Domain, Port and After Resolution IP Address
3) Time [When The Request Came]

Now Coming to What Can be done

The Proxy Can Apply Rule as

Allow/Block The Access If Request is From Certain Client IP Address
Allow/Block The Access If Request is To Certain Remote Server Domain
Allow/Block The Access If Request is To Certain Remote Server Port
Allow/Block The Access If Remote Server Domain Resolved To a Certain IP or IP List or to a CNAME
Allow/Block The Access If User-Agent is of a Certain Regex String
Allow/Block The Access If User-Agent is NOT of a Certain Regex String
Allow/Block The Access If The Time When the Request Came is Between a Range
Allow/Block The Access If The Time When the Request Came is NOT in Between a Range Provided.
Allow/Block The Access If Request is Authenticated
Allow/Block The Access If Request is Authenticated and the User is Sam
Allow/Block The Access If Request is Authenticated and the User is NOT Sam

Their are Tremendous Amount Of Combination that can be done to Get what is Required

Let me Give the Condition Pattern

Fields	Values
Authentication
`AND or OR`
Client IP
`AND or OR`
Remote Server Domain
`AND or OR`
Remote Server Port
`AND or OR`
Remote Server IP
`AND or OR`
WebCategory
`AND or OR`
User-Agent
`AND or OR`
Time
`AND or OR`
Access	ALLOW/BLOCK

The Above Table Follows a AND and OR Condition
Lets Take a Example

Example 1: Allow User: Sam via IP : 192.168.0.156 to access Google.com via Modern Browsers [Chrome, Firefox, Edge] only between Office Time [9 To 5]

Well Now the Below Just Shows Allowing on Certain Condition, But it Also Depends if the Proxy has a Default Blocking Rule has another Rule to Just Cut off Access and then Create a Allow Rule, A Lot of Possibility Possible .

Fields	Values
Authentication	Sam
`AND`
Client IP	192.168.0.156
`AND`
Remote Server Domain	`Regex:google.com`
`AND`
Remote Server Port	443,80
`AND`
Remote Server IP	`ANY`
`AND`
WebCategory	`ANY`
`AND`
User-Agent	`Chrome`, `Firefox`, `Edge`
`AND`
Time	`Office Time [9 To 5]`
`AND`
Access	ALLOW

*** Regardless of Web Category i.e even if google.com falls in any Category it is not effects the Policy ***

Example 2: Block User: ANY via IP : ANY to access WebCategory SearchEngine via Modern Browsers [ANY] between Office Time [9 To 5]

Fields	Values
Authentication	`ANY`
`AND`
Client IP	`ANY`
`AND`
Remote Server Domain	`ANY`
`AND`
Remote Server Port	`ANY`
`AND`
Remote Server IP	`ANY`
`AND`
WebCategory	`SearchEngine`
`AND`
User-Agent	`ANY`
`AND`
Time	`Office Time [9 To 5]`
`AND`
Access	BLOCK

This is Where WebCategory Help you,
Now Here ANY User is NOT ALLOWED to access SearchEngine Sites like Google, Yahoo, Bing etc and many More.

Now The Allow Rule Table Says To Block All Users, Depending on the Proxy Working NEW Policy can be below to allow Access to Certain IP

Quick Example:

Fields	Values
Authentication	`Bob`
`AND`
Client IP	`ANY`
`AND`
Remote Server Domain	`ANY`
`AND`
Remote Server Port	`ANY`
`AND`
Remote Server IP	`ANY`
`AND`
WebCategory	`SearchEngine`
`AND`
User-Agent	`ANY`
`AND`
Time	`Office Time [9 To 5]`
`AND`
Access	ALLOW

Now Here User Bob is allowed to access SearchEngine Sites like Google, Yahoo, Bing etc and many More.

Last and a Quick One For OR Condition

Fields	Values
Authentication	`Annie`
`AND`
Client IP	`ANY`
`AND`
Remote Server Domain	`Regex:searchable.co.in`, `Regex:facetime.com`
`OR`
WebCategory	`SearchEngine`
`AND`
Remote Server Port	`ANY`
`AND`
Remote Server IP	`ANY`
`AND`
User-Agent	`ANY`
`AND`
Time	`Office Time [9 To 5]`
`AND`
Access	ALLOW

This one was to show that you can create a Rule to allow SearchEngine as well as two more Websites Regardless in What WebCategory they fall in.

*The Above Was a Glimpse and a Use Case and Illustration of How a Proxy Restriction Working can be. Feel Free to Interrupt and Correct me

What All Things are Possible in a Proxy(Forward Proxy)

Ashish R Bhandari — Fri, 01 Jan 2021 14:06:49 +0000

Hello All,
I am Trying to Express, Expand and Explain My Thoughts and Learning Through these Posts

> Feel Free to Point Out If I have made any mistakes

Beginning With Proxy(If Said Only Proxy then it mostly means Forward Proxy)

There are Lots of Explanation Covered Already about this and I will add links and resources that I have read and found very interesting.

Coming Back as to what am I going to add in this post.
It going to be all about
What a Forward Proxy is

Capable of doing,
What it can do,
What it cannot do (depending on Configuration, Use and Functionality)

So Long Story Short, Since You have Gone Through My Above Links (^)
Let's Start it

Now The Proxy Works and Provides Features depends on where it is Placed.

Let's Break this down
1) If the Proxy is inside the Network(Basically a Scenario of a School, Institute, Office(Could be Any Office)), Then the Reason it is used is to Let the Client Use the Proxy and do

Caching

Website Access Blocking (Well it Fall inside Filtering But There is a Reason Why I kept it over here)

Filtering

Identification (Basically Knowing Which User and Based on that Filtering)

Other Activity (Logging, Alerting)

........

2) If the Proxy is Outside the Network(Basically a Scenario of a Online Proxy, Free Proxy Server, Browser Proxy Extension(like Hola , HotSpotShield), Browser VPN Extension(* More Here) ), Then the Reason it is used is to Make the Client Anonymous Over the Internet.
Which Means that This Proxy will provide me Anonymity and Let me Access what is Blocked by My ISP.
[Now This is a Bit Contradictory of the Above that is the reason why I Said the Above Punch line.]

So we are Clear here About Proxy and How it Differs based on where it is Placed.

Let's Go To the Other

How Easy it is for a Proxy to determine the Host the user is trying to Connect to ?

Based on the Host Header
For Plain HTTP Website it is Easy to Get the Host Header because anybody can see that, same ways even Proxy can see it Very Easily

For HTTPS Website, The Browser will First Send a CONNECT Request to indicate that it is going to initiate a SSL Connection with the Remote Website.
It Looks Like This

CONNECT b.config.skype.com:443 HTTP/1.1
Host: b.config.skype.com:443
Proxy-Connection: keep-alive
User-Agent: curl/7.58.0

The Important Headers are Only Below one, Rest Proxy-Connection, User-Agent are Important But not Compulsory/Mandatory for CONNECT Request [Well For User-Agent it can create problem accessing websites]

CONNECT b.config.skype.com:443 HTTP/1.1
Host: b.config.skype.com:443

The Connect Also specifies Port, indicating whether to connect on Standard or any other Port.

So, In Simple It works This Way

In Below Scenario, Imagine it This Way, We are Using Firefox and There is a Setting in browser: The Proxy Setting

I am Running a Proxy on My Machine Locally [I will Later Update which proxy I am running]

Plain HTTP Site

We will connect to: http://example.com/

The Browser Will Send a HTTP Request to The Proxy as Follows:

hmmm@demo:~# curl -vk "http://example.com/" -x 127.0.0.1:8080
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> GET http://example.com/ HTTP/1.1
> Host: example.com
> User-Agent: curl/7.58.0
> Accept: */*
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 OK
< Age: 341302
< Cache-Control: max-age=604800
< Content-Type: text/html; charset=UTF-8
< Date: Fri, 01 Jan 2021 13:50:29 GMT
< Etag: "3147526947+ident"
< Expires: Fri, 08 Jan 2021 13:50:29 GMT
< Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT
< Server: ECS (nyb/1D2C)
< Vary: Accept-Encoding
< X-Cache: HIT
< Content-Length: 1256

and Then the Proxy will make a Request to Remote Server (Gathered From Host Header, well it can be extracted from Request Line But Host Header is the Standard)

Let's Look at
A Simple DIRECT Request Looks as Follows

hmmm@demo:~# curl -vk "http://example.com/"
*   Trying 2606:2800:220:1:248:1893:25c8:1946...
* TCP_NODELAY set
*   Trying 93.184.216.34...
> GET / HTTP/1.1
> Host: example.com
> User-Agent: curl/7.58.0
> Accept: */*
>
< HTTP/1.1 200 OK
< Age: 341302
< Cache-Control: max-age=604800
< Content-Type: text/html; charset=UTF-8
< Date: Fri, 01 Jan 2021 13:50:31 GMT
< Etag: "3147526947+ident"
< Expires: Fri, 08 Jan 2021 13:50:31 GMT
< Last-Modified: Thu, 17 Oct 2019 07:18:26 GMT
< Server: ECS (nyb/1D2C)
< Vary: Accept-Encoding
< X-Cache: HIT
< Content-Length: 1256

HTTPS Request

We will connect to: https://example.com/


hmmm@demo:~# curl -vk "https://example.com/" -x 127.0.0.1:8080
*   Trying 127.0.0.1...
* TCP_NODELAY set
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to example.com:443
> CONNECT example.com:443 HTTP/1.1
> Host: example.com:443
> User-Agent: curl/7.58.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection Established
< Connection: Keep-Alive
< Proxy-Connection: Keep-Alive
<
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* CONNECT phase completed!
* CONNECT phase completed!
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (OUT), TLS change cipher, Client hello (1):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Unknown (8):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS Unknown, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=Los Angeles; O=Internet Corporation for Assigned Names and Numbers; CN=www.example.org
*  start date: Nov 24 00:00:00 2020 GMT
*  expire date: Dec 25 23:59:59 2021 GMT
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert TLS RSA SHA256 2020 CA1
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* Using Stream ID: 1 (easy handle 0x56435e5025c0)
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
> GET / HTTP/2
> Host: example.com
> User-Agent: curl/7.58.0
> Accept: */*
>
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
* Connection state changed (MAX_CONCURRENT_STREAMS updated)!
* TLSv1.3 (OUT), TLS Unknown, Unknown (23):
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
< HTTP/2 200
< age: 87233
< cache-control: max-age=604800
< content-type: text/html; charset=UTF-8
< date: Fri, 01 Jan 2021 14:04:25 GMT
< etag: "3147526947+ident"
< expires: Fri, 08 Jan 2021 14:04:25 GMT
< last-modified: Thu, 17 Oct 2019 07:18:26 GMT
< server: ECS (nyb/1D0D)
< vary: Accept-Encoding
< x-cache: HIT
< content-length: 1256
<
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
* TLSv1.3 (IN), TLS Unknown, Unknown (23):
<!doctype html>
<html>
<head>
    <title>Example Domain</title>

    <meta charset="utf-8" />
    <meta http-equiv="Content-type" content="text/html; charset=utf-8" />
    <meta name="viewport" content="width=device-width, initial-scale=1" />
    <style type="text/css">
    body {
        background-color: #f0f0f2;
        margin: 0;
        padding: 0;
        font-family: -apple-system, system-ui, BlinkMacSystemFont, "Segoe UI", "Open Sans", "Helvetica Neue", Helvetica, Arial, sans-serif;

    }
    div {
        width: 600px;
        margin: 5em auto;
        padding: 2em;
        background-color: #fdfdff;
        border-radius: 0.5em;
        box-shadow: 2px 3px 7px 2px rgba(0,0,0,0.02);
    }
    a:link, a:visited {
        color: #38488f;
        text-decoration: none;
    }
    @media (max-width: 700px) {
        div {
            margin: 0 auto;
            width: auto;
        }
    }
    </style>
</head>

<body>
<div>
    <h1>Example Domain</h1>
    <p>This domain is for use in illustrative examples in documents. You may use this
    domain in literature without prior coordination or asking for permission.</p>
    <p><a href="https://www.iana.org/domains/example">More information...</a></p>
</div>
</body>
</html>
* Connection #0 to host 127.0.0.1 left intact

As you can see above Curl Sends a Connect Request Prior.

So, we found that it is Very Easy for a Proxy to identify which remote server the user wants to connect to.

and based on that, The Proxy can have add a Condition whether to let users to access Remote Server or do not allow it.
How Efficiently, Easy, Useful depends on the Proxy

Quick Example: Inorder to Block a Domain in Squid, we need to restart Squid Service.
Same if we do that in Any Other Proxy it will Provide you a Dashboard to add domains and BLOCK it in Realtime

........... Post Will be Updated Soon.........

DOT(.TCP) DOH(oooHTTP) via curl & kdig

Ashish R Bhandari — Wed, 30 Dec 2020 02:24:20 +0000

> Feel Free to Point Out If I have made any mistakes

Intro To DNS

DNS (Plain Text Over UDP)
Well Also Over TCP But Still Plain Text

I Love Cloudflare Docs and It Pretty Much Covers Most of the part with Ease.
https://www.cloudflare.com/learning/dns/what-is-dns/

DNSSEC (Security Extn But Still Plain Text)

Well still Plain Text Over UDP, But validates the DNS Records
https://www.cloudflare.com/learning/dns/dns-security/

And Then Comes Security(Encryption) DOT & DOH

Here is Where Plain Text is Encrypted Text
[DOT] https://www.cloudflare.com/learning/dns/dns-over-tls/

Cloudflare Detailed Explanation and Practical View
[DOT] https://developers.cloudflare.com/1.1.1.1/dns-over-tls
[DOH] https://developers.cloudflare.com/1.1.1.1/dns-over-https

I am Capturing the Practical View , How it looks like and how you can try one via CLI using CURL and Kdig.
Cloudflare Provides details on how you can achieve it via CURL
https://developers.cloudflare.com/1.1.1.1/dns-over-tls

So Let's Begin With The Practical View

DOH (DNS Over HTTP)

A DOH Query using Curl Pre-requisites: Curl v7.4+

So, Below it basically does is, it gets the DNS Record (IP) from the DOH Server (cloudflare-dns.com) Since it is Over HTTP and you need a Resource so it is Therefore (https://cloudflare-dns.com/dns-query).
And The Makes a Connection After Fetching the IP

A Simple Curl
(-v => Verbose)
(-I => Head Request)
(--doh-url => Ask DNS Over HTTP to which DOH Server >https://cloudflare-dns.com/dns-query)



$ curl -v -I --doh-url https://cloudflare-dns.com/dns-query https://www.google.com

Details:



$ curl -v -I --doh-url https://cloudflare-dns.com/dns-query https://www.google.com

* Found bundle for host cloudflare-dns.com: 0x7fffe56ebe80 [serially]
* Server doesn't support multiplex (yet)
*   Trying 104.16.249.249:443...
* TCP_NODELAY set
* Hostname 'cloudflare-dns.com' was found in DNS cache
*   Trying 104.16.249.249:443...
* TCP_NODELAY set
* Connected to cloudflare-dns.com (104.16.249.249) port 443 (#1)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* Connected to cloudflare-dns.com (104.16.249.249) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* Server certificate:
*  subject: C=US; ST=California; L=San Francisco; O=Cloudflare, Inc.; CN=cloudflare-dns.com
*  start date: Jan 28 00:00:00 2019 GMT
*  expire date: Feb  1 12:00:00 2021 GMT
*  subjectAltName: host "cloudflare-dns.com" matched cert's "cloudflare-dns.com"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert ECC Secure Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fffe56e5290)
> POST /dns-query HTTP/2
Host: cloudflare-dns.com
accept: */*
content-type: application/dns-message
content-length: 32

* We are completely uploaded and fine
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=San Francisco; O=Cloudflare, Inc.; CN=cloudflare-dns.com
*  start date: Jan 28 00:00:00 2019 GMT
*  expire date: Feb  1 12:00:00 2021 GMT
*  subjectAltName: host "cloudflare-dns.com" matched cert's "cloudflare-dns.com"
*  issuer: C=US; O=DigiCert Inc; CN=DigiCert ECC Secure Server CA
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fffe56df9e0)
> POST /dns-query HTTP/2
Host: cloudflare-dns.com
accept: */*
content-type: application/dns-message
content-length: 32

* We are completely uploaded and fine
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 256)!
< HTTP/2 200
< date: Sun, 30 Aug 2020 14:22:56 GMT
< content-type: application/dns-message
< content-length: 74
< access-control-allow-origin: *
< cf-request-id: 04e15902c60000de8edb16a200000001
< expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
< server: cloudflare
< cf-ray: 5caf2ab13b45de8e-BLR
<
* Connection #1 to host cloudflare-dns.com left intact
* a DOH request is completed, 1 to go
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 256)!
< HTTP/2 200
< date: Sun, 30 Aug 2020 14:22:56 GMT
< content-type: application/dns-message
< content-length: 62
< access-control-allow-origin: *
< cf-request-id: 04e15902d60000deb22981d200000001
< expect-ct: max-age=604800, report-uri="https://report-uri.cloudflare.com/cdn-cgi/beacon/expect-ct"
< server: cloudflare
< cf-ray: 5caf2ab15b01deb2-BLR
<
* Connection #0 to host cloudflare-dns.com left intact
* a DOH request is completed, 0 to go

----- DOH Response [Starts] -----
* DOH Host name: www.google.com
* TTL: 43 seconds
* DOH A: 172.217.160.164
* DOH AAAA: 2404:6800:4009:080a:0000:0000:0000:2004
----- DOH Response [Ends] -----

*   Trying 172.217.160.164:443...
* TCP_NODELAY set
* Connected to www.google.com (172.217.160.164) port 443 (#0)
* ALPN, offering h2
* ALPN, offering http/1.1
* successfully set certificate verify locations:
*   CAfile: /etc/ssl/certs/ca-certificates.crt
  CApath: /etc/ssl/certs
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.3 (IN), TLS handshake, Encrypted Extensions (8):
* TLSv1.3 (IN), TLS handshake, Certificate (11):
* TLSv1.3 (IN), TLS handshake, CERT verify (15):
* TLSv1.3 (IN), TLS handshake, Finished (20):
* TLSv1.3 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.3 (OUT), TLS handshake, Finished (20):
* SSL connection using TLSv1.3 / TLS_AES_256_GCM_SHA384
* ALPN, server accepted to use h2
* Server certificate:
*  subject: C=US; ST=California; L=Mountain View; O=Google LLC; CN=www.google.com
*  start date: Aug 11 08:59:33 2020 GMT
*  expire date: Nov  3 08:59:33 2020 GMT
*  subjectAltName: host "www.google.com" matched cert's "www.google.com"
*  issuer: C=US; O=Google Trust Services; CN=GTS CA 1O1
*  SSL certificate verify ok.
* Using HTTP2, server supports multi-use
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* Using Stream ID: 1 (easy handle 0x7fffe56bfaa0)
> HEAD / HTTP/2
> Host: www.google.com
> user-agent: curl/7.68.0
> accept: */*
>
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* TLSv1.3 (IN), TLS handshake, Newsession Ticket (4):
* old SSL session ID is stale, removing
* Connection state changed (MAX_CONCURRENT_STREAMS == 100)!
< HTTP/2 200
HTTP/2 200
< content-type: text/html; charset=ISO-8859-1
content-type: text/html; charset=ISO-8859-1
< p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
p3p: CP="This is not a P3P policy! See g.co/p3phelp for more info."
< date: Sun, 30 Aug 2020 14:22:56 GMT
date: Sun, 30 Aug 2020 14:22:56 GMT
< server: gws
server: gws
< x-xss-protection: 0
x-xss-protection: 0
< x-frame-options: SAMEORIGIN
x-frame-options: SAMEORIGIN
< expires: Sun, 30 Aug 2020 14:22:56 GMT
expires: Sun, 30 Aug 2020 14:22:56 GMT
< cache-control: private
cache-control: private
< set-cookie: 1P_JAR=2020-08-30-14; expires=Tue, 29-Sep-2020 14:22:56 GMT; path=/; domain=.google.com; Secure
set-cookie: 1P_JAR=2020-08-30-14; expires=Tue, 29-Sep-2020 14:22:56 GMT; path=/; domain=.google.com; Secure
< set-cookie: NID=204=faTLwUwByLcvvqmTO0G45YKfiKg9_eBHAJG51-GL6xLFiSZSGxNPB4_AEi1NbR_3MkBFwBu1Km2PLw0h6Xh2ZjGO6RHbTR7AEnPHcHiqkC90Zc9XJqsQugw4zOzThkXwufU_YM2x1o4N40JrWvnKKhxG8v5ntJYdlZbWIF13EXk; expires=Mon, 01-Mar-2021 14:22:56 GMT; path=/; domain=.google.com; HttpOnly
set-cookie: NID=204=faTLwUwByLcvvqmTO0G45YKfiKg9_eBHAJG51-GL6xLFiSZSGxNPB4_AEi1NbR_3MkBFwBu1Km2PLw0h6Xh2ZjGO6RHbTR7AEnPHcHiqkC90Zc9XJqsQugw4zOzThkXwufU_YM2x1o4N40JrWvnKKhxG8v5ntJYdlZbWIF13EXk; expires=Mon, 01-Mar-2021 14:22:56 GMT; path=/; domain=.google.com; HttpOnly
< alt-svc: h3-29=":443"; ma=2592000,h3-27=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-T050=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"
alt-svc: h3-29=":443"; ma=2592000,h3-27=":443"; ma=2592000,h3-T051=":443"; ma=2592000,h3-T050=":443"; ma=2592000,h3-Q050=":443"; ma=2592000,h3-Q046=":443"; ma=2592000,h3-Q043=":443"; ma=2592000,quic=":443"; ma=2592000; v="46,43"

<
* Connection #0 to host www.google.com left intact

DOT (DNS Over TCP)

Pre-requisites: kdig
A DOH Query using kdig



$ apt install knot-dnsutils
$ kdig -d @1.1.1.1 +tls-host=cloudflare-dns.com  google.com

Details:



# kdig -d @1.1.1.1 +tls-host=cloudflare-dns.com  google.com
;; DEBUG: Querying for owner(google.com.), class(1), type(1), server(1.1.1.1), port(853), protocol(TCP)
;; DEBUG: TLS, imported 138 system certificates
;; DEBUG: TLS, received certificate hierarchy:
;; DEBUG:  #1, C=US,ST=California,L=San Francisco,O=Cloudflare\, Inc.,CN=cloudflare-dns.com
;; DEBUG:      SHA-256 PIN: V6zes8hHBVwUECsHf7uV5xGM7dj3uMXIS9//7qC8+jU=
;; DEBUG:  #2, C=US,O=DigiCert Inc,CN=DigiCert ECC Secure Server CA
;; DEBUG:      SHA-256 PIN: PZXN3lRAy+8tBKk2Ox6F7jIlnzr2Yzmwqc3JnyfXoCw=
;; DEBUG: TLS, skipping certificate PIN check
;; DEBUG: TLS, The certificate is trusted.
;; TLS session (TLS1.2)-(ECDHE-ECDSA-SECP256R1)-(CHACHA20-POLY1305)
;; ->>HEADER<<- opcode: QUERY; status: NOERROR; id: 33976
;; Flags: qr rd ra; QUERY: 1; ANSWER: 1; AUTHORITY: 0; ADDITIONAL: 1

;; EDNS PSEUDOSECTION:
;; Version: 0; flags: ; UDP size: 1232 B; ext-rcode: NOERROR
;; PADDING: 409 B

;; QUESTION SECTION:
;; google.com.                  IN      A

;; ANSWER SECTION:
google.com.             101     IN      A       216.58.193.78

;; Received 468 B
;; Time 2020-12-30 02:36:00 UTC
;; From 1.1.1.1@853(TCP) in 9.6 ms