DEV Community: KetanIP

Analytics Box - A simple privacy focused analytics in GO.

KetanIP — Thu, 02 Sep 2021 05:36:19 +0000

Repo: https://github.com/KetanIP/analytics.

Hello guys, today I am here to share with you a project that I've made recently, it is Analytics Box - A simple privacy focused analytics in Golang.

What is Analytics Box ?

Analytics box is analytics tool like google analytics but instead of ripping user's privacy off them like google it respects their privacy and doesn't collect any unnecessary information.

Features

No Cookies.
Forgets user in 24 hours so even if wish to know user's private info you can't.
Can track unique page views effectively without hurting user's privacy.
Custom events available.
Super simple to setup. ( 1 go binary + 1 super small script on frontend ).
Super low resource consumption.
Dashboard is decoupled from the tool so you can use any tool ( like Metabase for visualization of data ).
Supports multiple SQL database.

You may wish to ask some question here are some FAQ.

How do we uniquely identify users?

We identify users uniquely by creating a hash and storing it in a Redis database along with a UUID which automatically deletes itself in 24 hours.
hash = md5( raw_public_ip + browser_name + operating_system + date + secret)

It generates a 32 character long string with is stored in database along with a UUID and it is only valid for a day.

Parameters used in hash have the following reasons,

IP - to identify user ( It may be a public IP like café or a user's own IP )
Browser - So if user changes his/her browser we forget it. Operating System - So if user changes its device we forget him.
Date - So we forget user daily.
Secret - Here comes the most important part we while production recommend to use a UUID that isn't logged anywhere so even if want to view user's private info you can't, but while development use a hardcoded string as you restart server many times and it will generate false unique visitors.

What makes it different ?

There are many privacy friendly analytics tool but here are some points that makes it special.

It is written in go so you can but it on the same server as your app as it uses nearly to negligible amount of resource ( in large schema of things ).
It is written in go fiber which can handle a by lot I mean around 35,000 requests per second ( as per their website).
It currently supports PostgreSQL and in future I plan to add support for Clickhouse because of its amazing performance, and I will continue to keep postgres supported for as long as possible because of its low resource consumption it would be a great tool for small websites like blogs, like one that, I have though it hasn't been updated for a while now.
I don't plan to convert it into a SaaS application ( at least in near future ) as I don't want to make another SaaS privacy focused analytics app as there are already many good in market. I'm making this in a way that you can get most insights from your data so it doesn't have a dashboard ( use something like Metabase for it here are some screenshots of dashboard that I build within 3-4 minutes while writing this readme and I'm no profession and it on par with what SaaS provides apps provides to you) with complete control over your data and it can run on simple commodity hardware for a long period of time without any manual Interventions.

Why don't it has a dashboard build in ?

There are many privacy friendly analytics tools and they have a dashboard built in as most of them are SaaS application but this isn't gonna be a SaaS app so in order to make most sense out of the data we recommend users to use something like Metabase as even a non-techie can make custom charts and if you are a techie then you can make even more sense of data by writing custom SQL Queries to obtain and visualize data.

The Images you saw above were made within 3-4 minutes while I was writing README for GitHub repo, and it was auto generated I just changed its sizes and names that's it. So we have made a conscious decision to decouple analytics tool and dashboard for it.

API that converts Article to JSON

KetanIP — Thu, 29 Jul 2021 10:28:21 +0000

Edit: Its free ( no limits) ( no Card needed ), feel free to play and experiment with it. You can find it at Rapid API Marketplace

Hello guys.

Today I'm happy to announce that I have launched a simple micro-saas application. My API helps you to convert any article be it a blog, news article or any thing to a well structured json document.

It will parse it and return its title, top image, authors, publish date, text, summary, videos, images, tags, keywords, and url.

Not only this but it also returns sentimental analysis of the text so you can use it forward in your application. So you can use the sentiments in your application.

You can find it at Rapid API Marketplace


var axios = require("axios").default;

var options = {
  method: 'GET',
  url: 'https://article-parser.p.rapidapi.com/',
  params: {
    website_url: 'https://www.washingtonpost.com/politics/2021/06/18/gops-gaslighting-biden-putin-trump/'
  },
  headers: {
    'x-rapidapi-key': 'YOUR_RAPID_API_KEY',
    'x-rapidapi-host': 'article-parser.p.rapidapi.com'
  }
};

axios.request(options).then(function (response) {
    console.log(response.data);
}).catch(function (error) {
    console.error(error);
});

The parsed article returned is,

{
   "title":"The GOP’s gaslighting on Biden, Putin and Trump",
   "top_image":"https://www.washingtonpost.com/wp-apps/imrs.php?src=https://d1i4t8bqe7zgj6.cloudfront.net/06-16-2021/t_31136d8876b14fb0a02e726c1921b27e_name_biden_presser_0616_scaled.jpg&w=1440",
   "authors":[
      "Senior Reporter"
   ],
   "publish_date":"2021-06-18T00:00:00",
   "summary":"In their quest to pitch Biden as weak on Russia, Republicans are attacking pretty much everything they ignored about Trump.",
   "text":"Again: election interference. Again: human rights. Again: cyberattacks. Trump declined to focus on these things when he appeared with Putin. There was essentially nothing in his news conference geared toward holding “Russia accountable.” Biden might not have been as specific as some would have liked on things like human rights, but he at least brought them up and criticized Russia on them. Trump, by contrast, declined to and regularly suggested that the United States shouldn’t judge Russia’s human rights record.",
   "videos":[

   ],
   "images":[
      "https://www.washingtonpost.com/wp-apps/imrs.php?src=https://d1i4t8bqe7zgj6.cloudfront.net/06-16-2021/t_31136d8876b14fb0a02e726c1921b27e_name_biden_presser_0616_scaled.jpg&w=1440"
   ],
   "tags":[

   ],
   "keywords":[
      "gaslighting",
      "biden",
      "suggested",
      "united",
      "specific",
      "putin",
      "trump",
      "human",
      "declined",
      "russia",
      "rights",
      "states",
      "things",
      "gops"
   ],
   "url":"https://www.washingtonpost.com/politics/2021/06/18/gops-gaslighting-biden-putin-trump/",
   "sentimental_analysis":{
      "positive":0.085,
      "negative":0.038,
      "neutral":0.878,
      "overall":0.4767
   }
}

Guys please let me know your experience with it. Hope you like it.

Introducing DBJson Document Database.

KetanIP — Tue, 09 Feb 2021 15:30:57 +0000

Hello guys today I am here to introduce my 🐶pet project DBJson. It is a Document Oriented Database.

This is a simple data base which stores its data in the form of collections and records format in JSON files!

Structure of data:

DON'T WORRY IT IS AUTOGENERATED.

This was made by me as sometimes I don't like to setup ORM or Databases for small projects that are never going to production ever.

So I though why not make a make a database for such thing only. I decided to make a NoSQL database as it can store any data without setting up tables as we need to do it with RDBMS.

At its current you can download/view it on Github also check out DBJson's docs here.

At its current stage it can do the following things:

Creating records
Reading records
Updating records
Deleting records
Filter records

What are its inspiration ?

It has got its inspirations from tinyDB ( for its simplicity ) and Firestore( NOSQL database offered by firebase for its document style datamodelling ).

Getting Stated

Getting stated with it is damn easy,

from dbjson.main import DB

# Instatilizing DB class
db = DB()

# Test Data
data = {
  "id": 1,
  "first_name": "Vivyan",
  "last_name": "Treherne",
  "email": "vtreherne0@jigsy.com",
  "ip_address": "94.254.247.240"
}
collection = "users"

# Adding Record
data = db.createRecord(collection, data)
print(data)
# Response -> {'__id__': 'f00ae4e3ca8c3e318a68acc460e5f401', '__data__': {'id': 1, 'first_name': 'Vivyan', 'last_name': 'Treherne', 'email': 'vtreherne0@jigsy.com', 'ip_address': '94.254.247.240'}}

# Updating Record
record_key = "f00ae4e3ca8c3e318a68acc460e5f401"
to_update = [
    {"email": "jhon@email.com"},
    {"ip_address": "google.com"}
]
data = db.updateRecord(collection, "f00ae4e3ca8c3e318a68acc460e5f401", to_update)
print(data)
# Response -> {'id': 1, 'first_name': 'Vivyan', 'last_name': 'Treherne', 'email': 'jhon@email.com', 'ip_address': 'google.com'}

# Deleting Record
db.removeRecord(collection, record_key)

That's all you need to generate use your db. This is still in its very early stage of development so it may be unstable and may be subjected to change, so if you want to follow it updates you can follow it on GitHub or you can follow me here on dev.to.

You can explore all its features at https://ketanip.github.io/dbjson/main.html.

Hope you guys like it. You can contribute it too at https://github.com/ketanip/dbjson/.

👨‍💻 Routing in Django ( Part 9 ) - Django For Beginners

KetanIP — Sun, 06 Dec 2020 15:47:27 +0000

This post was originally posted here 👨‍💻 Routing in Django ( Part 9 ) - Django For Beginners.

This is part of the series Django For Beginners we are going to see about apps in app and we will see about urls in python.

In this post we are going to learn all about urls in django,, let's get started.

Understanding URLS in Django

Let us first understand where we write our urls, we write our urls in our project folder in urls.py file, as shown below.

 📂 myDjangoSite
    - 📂 myDjangoSite
        - 📄 __init__.py
        - 📄 asgi.py
        - 📄 settings.py
        - 📄 urls.py
        - 📄 wsgi.py
    - 📄 manage.py

In urls.py specifically we write path for our django website in urlpatterns list, in a patterns as shown below,

from django.urls import path
from . import views

urlpatterns = [
    path('', views.Home, name="home"),
    path('about/', views.About, name="about"),
]

The syntax for path is as follows,

path( route, view, name )

Where route is a string argument and denotes the relative path we want for that view. View is a view it may be a class-base or a function-based view...

Sorry to interrupt you, you can read this complete post here 👨‍💻 Routing in Django ( Part 9 ) - Django For Beginners.

Handling Files in Python 🐍

KetanIP — Fri, 04 Dec 2020 16:50:24 +0000

This post was originally posted here, you can eat the complete post here.

File handling is one of the most important part of a programmer's life let him/her be a data scientist, web developer or even a cyber security expert. So I have decided to write this post all about handling files in python.

To open files and bring them in python we have the following syntax,

file = `open(file_name, mode)`

To open a file in read mode we use the following syntax, . The first argument is obviously the file_name and the second one is a bit more interesting in it is the mode in which you want to open the file and it determines whether we can read,append, write , open in binary and so on we will see it down.

For the entirety of this blog post we are going to assume that our python file and our text file which we will be using to perform operations upon are located in the same folder, and the content of the text file is as follows, I have not written it just copied it from a random text generator online as I don't like lorem ipsum that much.

readme.txt

He found himself holding onto the worst type. 
It was the type of secret that could gnaw away at your insides if you didn't tell someone about it, but it could end up getting you killed if you did.

Reading Files

Reading files in python is damn simple, let's understand it by an example, in this example we will be reading our text file readme.txt` and is located in the same folder as our python files is located.

To open file in read mode we pass 'r' place of mode in the open file syntax, to read the the file don't worry about these modes I will provide a list at the end of the post with all of the them along with their functions.

main.py

file = open('readme.txt', 'r')

if file.mode == 'r':
    fileContent = file.read()
    print(fileContent)

In above code we are opening a file in read mode and then checking whether it is open in read mode and if it is then we are reading the content of file by using .read() method in it and then printing it.

Sorry to interrupt you but I want to tell you one thing that I have started a small blog you may continue reading it here it will help me a lot.

👨‍💻 Basics of Views and Routing in Django ( Part 8 ) - Django For Beginners

KetanIP — Sat, 28 Nov 2020 08:43:43 +0000

This is part of the series Django For Beginners we are going to see about apps in app and we will see about models in django in this part.

You can this read complete post here.

In this post we are going to understand the idea of views and routing from scratch and see some types of responses and then we will learn about basic routing and to separate urls.py in separate files files.

what are views ?

Views are like middle man between request and response on your server they contain all you logic for a route, we will talk about route later in this post. They essentially process get the data needed for the request and process it and the return it as a response. This response can be of a variety of types, text, html , json , xml to name a few.

Now let's see a example of a view,

from django.http import HttpResponse

def home_response(request):
    return HttpResponse("Hello World 😊.")

I tried to make is as basic as possible. What is happening above is when we receive a request and we call this view function it will return an simple text response, that is Hello World 😊. .

You may have noticed about request argument in the home_response view above it is compulsory argument it is a dictionary object and contains lot, by lot, I mean really a lot of information about the request of which most of the times you are not going to use directly. With request object we can access headers, user agent, ip, cookies to name a few.

In above example we returned text response but it is not what happens in real world, right 🤔 ? In real world scenarios we need to either prove a html response or a json or xml as a response. So let us see how to pass html as response for now.

from django.http import HttpResponse

def home_response(request):
    return HttpResponse("<h1>Hello World 😊.</h1>")

Returning HTML Pages in Response

I am lazy so I copied above code and just added <h1></h1> tags around our plane text response and wallah, it is now a html response but don't you think it would be bad for long html responses with which we have to deal most of the times, so for that we keep html away from views all together so to keep code clean and follow the DRY principle .

Sorry to interrupt you but I want to tell you one thing that I have started a small blog you may continue reading it here it will help me a lot.

👨‍💻 Queries in Django ( Part 7 ) - Django For Beginners

KetanIP — Sat, 21 Nov 2020 18:51:19 +0000

This is part of the series Django For Beginners we are going to see about apps in app and we will see about models in django in this part.

In this post we are going to learn a lot queries in django. I recommend you to read last post so as it will be easy to understand context of this post , now let's get started.

These are the models which we are going to use for most part in this post,

from django.db import models
from django.utils import timezone
from djnago.contrib.auth.models import User

class Post(models.Model)
    title = models.CharField(max_length=250)
    created_at = models.DateTimeField(default=timezone.now)
    author = models.ForeignKey('Author', on_delete=models.CASCADE, related_name="author")
    likes =  models.ManyToManyField(User, blank=True, related_name="likes")
    dislikes =  models.ManyToManyField(User, blank=True, related_name="dislikes")
    likes_count = models.PositiveIntegerField(default=0)
    dislikes_count = models.PositiveIntegerField(default=0)
    rating = models.DecimalField(max_digits=4, decimal_places=2)


class Author(models.Model):
    user = models.OneToOneField(User, on_delete=models.CASCADE)
    liked_posts = models.ManyToManyField(Post, blank=True, related_name="liked_posts")
    disliked_posts = models.ManyToManyField(Post, blank=True, related_name="disliked_posts")

Get All

To get all data of the model we use this query, it is not used most often as it costs a lot of resources if data is large and it is always a good practice to limit the query results which we will see how to do after this,

all_posts = Post.objects.all()

syntax = ModelName.objects.all()

Limit Query Results

To limit results of any query we add [ start_count : end_count ] to get results from start_count and end_count you can also use [ : end_count ] to get results from 0 to end_count. Let's see an example,

only_five_posts = Post.objects.all()[:4]

The above query will give 5 results only as index start from 0 and till 4 it would be five results.

Get Query

There is situation many times when we want to retrieve only one result from the query with exact match of fields for such situations we use get query, let me show you an example,

one_post = Post.objects.get(id=1)

syntax = ModelName.objects.get( pass_keyword_arguments )

Sorry to interrupt you but I want to tell you one thing that I have started a small blog you may continue reading it here it will help me a lot.

👨‍💻 Basic operations on Models In Django ( Part 6 ) - Django For Beginners

KetanIP — Tue, 17 Nov 2020 04:31:04 +0000

This is part of the series Django For Beginners we are going to see about apps in app and we will see about models in django in this part.

In this post we are going to learn about queries in django. When we use django we do not need to write SQL Queries it is done by django itself, we write super user friendly syntax for the queries ,

Let's see we want to get all the objects in a model ( note each model is a table, it may be more depending on the type of relation used while creating it ), let the schema of this model be simple it will be as follows,

from django.db import models

class Book(models.Model):
    title       = models.CharField(max_length=250)
    createdAt   = models.DateTimeField(default=timezone.now)
    description = models.TextField()
    orders      = models.PositiveItegerField(default=0)

Now to get it all we use the following syntax,

allBooks = Book.objects.all()

and that's it will return all the books of QueryList form.

How to run Queries in Django ?

I should have told it first I know but I told it now because you should understand how easy it is to write queries in django. Now let's see how to run them,

There are two ways,

Interactive Way ( in command line ).
In any python file.

Interactive Way

To use it interactively just go to your root directory of your django project and type the following command to start the interactive console,

python manage.py shell

It will start the interactive console from which you can import any model in the project, you can access all of the project files. Also you can also import normal python libraries.

Regular Way

You can import models to any file in the project and can use it as per your will and in later part of this series we are going to use in this way in views.py to write views for our website.

Now let's see the basic operations on models,,

To Create New Model Object

To create new model object we first import it and then we use create method which is build in model on it and create it. Let me show a example to you,

from myapp.models import Book

newBook = Book.objects.create(title="Harry Potter - Some Part", description="Don't you know it ?")
newBook.save()

As you may have seen that creating models is simple damn simple. We just need to pass values in objects.create method of that model and store it in variable and than use save method on it. It is as simple as it could get.

You may have seen that I have not passed certain parameters, such as createdAt and orders, because we have provided default values to it. But we can also pass it. Let me show you how,

from myapp.models import Book
from datetime import datetime
from random import randint

currentDateTime = datetime.now()
orders = randint(0,10)

newBook = Book.objects.create(title="Harry Potter - Some Part", description="Don't you know it ?", createdOn=currentDateTime, orders=orders)
newBook.save()

This is how it is done. You may have seen that I have passed createdOn in third place but while creating model it was in 2^nd^ place I did it to show that order in which you pass these keyword arguments doesn't matter.

Query The Object

Now comes the part of querying the model, here I will cover basics as querying is a very big topic, with the part I cover here would e more than enough for many project , at least in the beginning. Now let's start,

Sorry to interrupt you but I want to tell you one thing that I have started a small blog you may continue reading it here it will help me a lot.

💻 Exploring Django Project Structure ( Part 3 ) - Django For Beginners

KetanIP — Sat, 14 Nov 2020 16:37:30 +0000

In this part of the series Django For Beginners we are going to explore project struture of a new Django project which we create in last part check it out here after installing Django.

The poject structure should look something like this,


 📂 myDjangoSite
    - 📂 myDjangoSite
        - 📄 __init__.py
        - 📄 asgi.py
        - 📄 settings.py
        - 📄 urls.py
        - 📄 wsgi.py
    - 📄 manage.py

Sorry to interrupt you but I want to tell you one thing that I have started a small blog you may continue reading it here it will help me a lot.

Models In Django

KetanIP — Tue, 10 Nov 2020 15:51:06 +0000

This is part of the series Django For Beginners we are going to see about apps in app and we will see about models in django in this part.

What are models ?

Models play an important role in django, it makes it what it is. Models is the way to represent our database in a pretty form. We can also change databases without any changes to models in django, and we use these models to query our database. Models make life easy.

We define models for our application in apps in models.py file. Now let's see the example, in this example we are going to design a model for a blog post and tags associated with blog post. In real-life scenario you can also you libraries such as django-taggit for tagging the blog post and to do the heavy lifting for us. Now let's see the example,

from django.db import models
from django.utils import timezone
from django.contrib.auth.models import User

class Blog(models.Model):
    title = models.CharField(max_length=250)
    post  = models.TextField()
    slug  = models.SlugField()
    tags  = models.ManyToManyField('Tag', blank=True)
    createdAt = models.DateTimeField(default=timezone.now)
    createdBy = models.ForeignKey(User, on_delete=models.CASCADE)

class Tag(models.Model):
    tag = models.CharField(max_length=250)
    createdAt = models.DateTimeField(auto_now=True)
    blogs = models.ManyToManyField(Blog, blank=True)

Sorry to interrupt you but I want to tell you one thing that I have started a small blog you may continue reading it here it will help me a lot.

👨‍💻 Getting Started With Django ( Part 1 ) - Django For Beginners

KetanIP — Tue, 20 Oct 2020 16:08:38 +0000

This post was originally posted on 👨‍💻 Getting Started With Django ( Part 1 ) - Django For Beginners.

Django

This is first part of the series Django For Beginners we are going to see an introduction to Django, we will then see about why to use django, and then its design philosophies then we will see about advantages of django and at last we will which comanies uses django.

What is Django ?

Django is a high-level web framework written in Python. It encourages clean and rapid development. It takes all the burden of handeling complicated tasks on itself and provides developer with a rich hassel free delevlopment experince .

Why Django ?

Well Django is a complete framework, here are some resons why you should use django,

It is fast.
It is secure.
Encourages rapid development.
It is scalable.
Complete Framework.

Design Philosophies

Django follows certain design philosophies, some of them are,

Being Decoupled : Django aims and encourages each element of the project independent of each other as it becomes easy to maintain the project and an single element can be exported and used in other project easily.
Quick Development : Django was made in mind to provide a fast and rich development experice.
Don't Repeat Yourself (DRY) : Django follows DRY, which literally means not repeating code, one of the ways of doing this is by de-coupling elements in an application and using them in a efficient way.
Clean Design : Django strictly maintains the project design in such a way that it makes it easy to follow best practices.

Advantages of Django.

There are many advantages of django for the sake of this blog here are some the big once,

Django ORM : Django ORM is Django's biggest power, well all can agree to it. It suppport most of the popular SQL databases. It creates and manages the database without any manual intervensions in a very optimized way. It can also be used with NO-SQL database with the help of third-party libraries.
Administration GUI : Django comes with a powerful admin interface right out of the box. It supports all kind of CRUD operations on the database right from admin panel. It has a beautiful interface. It can even be customized according to once needs.
Build-in Framework Support : Django supports many frameworks like RSS Feeds, AJAX and many more out of the box.
Powerful Third Party Package Ecosystem : For django you can find third party package for literally any thing, wanna handle payments use dj-stripe, wanna use social auth use django-allauth, want REST API use django-rest-framework and many more. As you may have seen there are lierally frameworks build on django itself so what do you want more. On Django Packages.org you can find packages for your django porject.
Development Enviorment : Django comes with a development server right out of the box that does't require any kind of configuration.
Changing Database is Super Easy : In django changing databases is super easy you can use sqlite3 for development and change production database to postgres within seconds. Before some of you may say what about data, then listen if you are doing this in production than data is your duty.

Who Uses Django ?

Now the famous question, who uses django ?. Django is used by a variety of people for a variety of purposes. It is used by management systems, social networks, scientific computing platforms. Let me name a few,

Instagram
Disqus
Pinterest
National Geographic
Spotify
YouTube
The Washington Post
BitBucket
DropBox
Quora

and many more...

You can support me by checking out Django For Beginners Series.

🍪 Cookies in Flask 🍾

KetanIP — Tue, 13 Oct 2020 06:16:43 +0000

What are cookies ?

Cookies are a kind of storage, but a very small storage. It is mainly used to store small information such as csrf_token, login credentials ( encrypeted ), preferences, site statics and/or user activities. By using cookies one can provide an immersive experice to the users of the websites by making the website more personalized for them.

How to use "Cookies" in Flask ?

In flask using cookies is super simple. In flask cookies are Immutable Dictionary, we can also call as a dictionary. To view the value of the request.cookies attribute it will return an dictionary object. Let's see an understand it with an example,

app.py

from flask import Flask, request, make_response, redirect, url_for, render_template

app = Flask(__name__)

@app.route("/")
def home():
    userName = request.cookies.get("name")
    if userName:
        return f"Hello {userName}." 
    else:
        return redirect(url_for("setName"))

Continue reading here 🍪 Cookies in Flask 🍾.