DEV Community: Gautam Krishna R

How to prevent GitHub from suspending your cronjob based triggers

Gautam Krishna R — Sun, 26 Sep 2021 12:54:50 +0000

I created a GitHub action to prevent GitHub from suspending your cronjob based action triggers due to repository inactivity.

gautamkrishnar / keepalive-workflow

GitHub action to prevent GitHub from suspending your cronjob based triggers due to repository inactivity

Keepalive Workflow

GitHub action to prevent GitHub from suspending your cronjob based triggers due to repository inactivity

Why

GitHub will suspend the scheduled trigger for GitHub action workflows if there is no commit in the repository for the past 60 days. The cron based triggers won't run unless a new commit is made. It shows the message "This scheduled workflow is disabled because there hasn't been activity in this repository for at least 60 days" under the cronjob triggered action.

What

This workflow will automatically use the GitHub API (or create a dummy commit) in your repo if the last commit in your repo is 45 days (default) ago. This will keep the cronjob trigger active so that it will run indefinitely without getting suspended by GitHub for inactivity.

How to use

There are three ways you can consume this library in your GitHub actions

GitHub API Keepalive Workflow -

…

View on GitHub

Why

What

This workflow will automatically create a dummy commit in your repo if the last commit in your repo is 50 days (default) ago.
This will keep the cronjob trigger active so that it will run indefinitely without getting suspended by GitHub for inactivity.

How to use

There are two ways you can consume this library in your GitHub actions

Via GitHub Actions (For GitHub Actions users)

You can just include the library as a step after one of your favorite GitHub actions. Your workflow file should have the checkout action defined in one of your steps since this library needs git CLI to work.

name: Github Action with a cronjob trigger
on:
  schedule:
    - cron: "0 0 * * *"

jobs:
  cronjob-based-github-action:
    name: Cronjob based github action
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - # step1
      - # step 2
      - # step n, use it as the last step
      - uses: gautamkrishnar/keepalive-workflow@master # using the workflow with default settings

Let's take an example of Waka Readme

name: My awesome readme
on:
  workflow_dispatch:
  schedule:
    # Runs at 12 am UTC
    - cron: "0 0 * * *"

jobs:
  update-readme:
    name: Update this repo's README
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: athul/waka-readme@master
        with:
          WAKATIME_API_KEY: ${{ secrets.WAKATIME_API_KEY }}
      - uses: gautamkrishnar/keepalive-workflow@master # using the workflow with default settings

Via JavaScript library (For GitHub Actions developers)

For developers making awesome GitHub actions, you can consume the library in your javascript-based GitHub action by installing it from NPM. Make sure that your GitHub action uses checkout action since this library needs it as a dependency.
You can also ask your users to include it as an additional step as mentioned in the first part.

Install the package

Install via NPM:

npm i keepalive-workflow

Install via Yarn:

yarn add keepalive-workflow

Use it in your own GitHub action source code

const core = require('@actions/core');
const { KeepAliveWorkflow } = require('keepalive-workflow');

// Using the lib
KeepAliveWorkflow(githubToken, committerUsername, committerEmail, commitMessage, timeElapsed)
  .then((message) => {
    core.info(message);
    process.exit(0);
  })
  .catch((error) => {
    core.error(error);
    process.exit(1);
  });

Options

For GitHub Action

If you use the workflow as mentioned via GitHub actions following are the options available to you to customize its behavior.

Option	Default Value	Description	Required
`gh_token`	your default GitHub token with repo scope	GitHub access token with Repo scope	No
`commit_message`	`Automated commit by Keepalive Workflow to keep the repository active`	Commit message used while committing to the repo	No
`committer_username`	`gkr-bot`	Username used while committing to the repo	No
`committer_username`	`gkr@tuta.io`	Email id used while committing to the repo	No
`time_elapsed`	`50`	Time elapsed from the previous commit to trigger a new automated commit (in days)	No

For Javascript Library

If you are using the JS Library version of the project, please consult the function's DocString in library.js to see the list of available parameters.

Liked it?

Hope you liked this project, don't forget to give it a star ⭐.

Show your latest dev.to posts automatically on your GitHub profile readme

Gautam Krishna R — Tue, 21 Jul 2020 15:29:08 +0000

I just created a GitHub Action that lets you add links to your latest blog posts from Dev.to and other sources to your Github Profile readme automatically.

Check it out if you want a readme like this:

Project on GitHub: https://github.com/gautamkrishnar/blog-post-workflow
Example readme: https://github.com/gautamkrishnar

Demo video

How to use

Star this repo 😉
Go to your repository
Add the following section to your README.md file, you can give whatever title you want. Just make sure that you use  in your readme. The workflow will replace this comment with the actual blog post list:

# Blog posts
<!-- BLOG-POST-LIST:START -->
<!-- BLOG-POST-LIST:END -->

Create a folder named .github and create a workflows folder inside it if it doesn't exist.
Create a new file named blog-post-workflow.yml with the following contents inside the workflows folder:

name: Latest blog post workflow
on:
  schedule: # Run workflow automatically
    - cron: '0 * * * *' # Runs every hour, on the hour
  workflow_dispatch: # Run workflow manually (without waiting for the cron to be called), through the Github Actions Workflow page directly
permissions:
  contents: write # To write the generated contents to the readme

jobs:
  update-readme-with-blog:
    name: Update this repo's README with latest blog posts
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: gautamkrishnar/blog-post-workflow@master
        with:
          feed_list: "https://dev.to/feed/gautamkrishnar,https://www.gautamkrishnar.com/feed/"

Replace the above url list with your own rss feed urls. See popular-sources for a list of common RSS feed urls.
Commit and wait for it to run automatically or you can also trigger it manually to see the result instantly. To trigger the workflow manually, please follow the steps in the video.

Options

This workflow has additional options that you can use to customize it for your use case. The following are the list of options available:

Option	Default Value	Description	Required
`feed_list`	`""`	Comma-separated list of RSS feed urls, eg: `https://example1.com,https://example2.com`	Yes
`max_post_count`	`5`	Maximum number of posts you want to show on your readme, all feeds combined	No
`readme_path`	`./README.md`	Path of the readme file you want to update	No
`gh_token`	your GitHub token with repo scope	Use this to configure the token of the user that commits the workflow result to GitHub	No
`comment_tag_name`	`BLOG-POST-LIST`	Allows you to override the default comment tag name (`<!-- BLOG-POST-LIST:START --><!-- BLOG-POST-LIST:END -->`), if you want to show multiple instances of the action on the same repo, see advanced usage for more info	No
`disable_sort`	`false`	Disables the sorting of the list based on publish date	No
`template`	`default`	Allows you to change the structure of the posts list by using different variables. By default this workflow uses markdown list format to render the posts, you can override this behavior using this option. Eg: `[$title]($url)` will give you a space-separated list of posts. Supported variables `$title`: Title of the post `$url`: URL of the post `$description`: Description of the post `$newline`: Inserts a newline `$date`: Inserts the post date based on the `date_format` specified `$randomEmoji`: Allow you to use random emojis in the post, pass emojis as the parameter to chose one of it randomly in each post item. Eg: `$randomEmoji(💯,🔥,💫,🚀,🌮)`. See the issue comment for more details `$emojiKey`: You can use this argument to show emojis on each of your post item sequentially in the order you specify. Example: `$emojiKey(💯,🔥,💫)`. See the issue comment for more details	No
`date_format`	`UTC:ddd mmm dd yyyy h:MM TT`	Allows you to change the format of the date or time displayed when using the $date in the template option. This uses NPM dateformat library, please read the library documentation for the supported formats	No
`user_agent`	`rss-parser`	Allows you to customize the user agent used by the RSS feed crawler	No
`accept_header`	`application/rss+xml`	Allows you to customize the accept header of the http requests	No
`tag_post_pre_newline`	`true` if you are not using template option	Allows you to insert a newline before the closing tag and after the opening tag when using the template option if needed, for better formatting	No
`filter_comments`	`medium,stackoverflow/Comment by $author/,stackexchange/Comment by $author/`	Comma separated list of platforms you want to enable the comment filter. Available filters `medium`: Allows you to filter out the Medium comments. Known issue: #37 `stackoverflow/Comment by $author/`: Allows you to filter out the StackOverflow comments. Argument to this filter is optional, it defaults to 'Comment by $author'. If you use any language other than English on StackOverflow, you can use this argument to customize it. See #16 for more info. `stackexchange/Comment by $author/`: Allows you to filter out the StackExchange comments. Argument to this filter follows the same format as `stackoverflow` filter's argument.	No
`custom_tags`	`""`	Allows you to use the custom tags from your feed items in your template. Format: `variableName/tagName/,variableName/tagName/`. Please see the issue comment for more details	No
`title_max_length`	`""`	Allows you to trim the title in the posts list, excess text will be appended with an ellipsis `...`	No
`description_max_length`	`""`	Allows you to trim the description in the posts list, excess text will be appended with an ellipsis `...`	No
`item_exec`	`""`	Allows you to execute custom JavaScript code on each post item fetched from the xml to do advanced text manipulation. Please see the issue comment as an example	No
`commit_message`	`Updated with the latest blog posts`	Allows you to customize the commit message	No
`committer_username`	`blog-post-bot`	Allows you to customize the committer username	No
`committer_email`	`blog-post-bot@example.com`	Allows you to customize the committer email	No
`output_only`	`false`	Sets the generated array as `results` output variable so that it can be consumed in other actions and parsed via utilities like jq. This will also prevent committing to readme. See #51 for more details about the output format and how to use it.	No
`enable_keepalive`	`true`	Workflow will automatically do a dummy commit to keep the repository active if there is no commit activity for the last 50 days. GitHub will stop running all cron based triggers if the repository is not active for more than 60 days. This flag allows you to disable this feature. See #53 for more details.	No
`retry_count`	`0`	Maximum number of times to retry the fetch operation if it fails, See #66 for more details.	No
`retry_wait_time`	`1`	Time to wait before each retry operation in seconds.	No

Advanced usage examples

StackOverflow example

The following configuration allows you to show your latest StackOverflow activity along with your latest blog posts in the Github profile or project readme:

Follow the steps mentioned in the how to use section
Add the following section to your README.md file, you can give whatever title you want. Just make sure that you use  in your readme. The workflow will replace this comment with the actual StackOverflow activity:

# StackOverflow Activity
<!-- STACKOVERFLOW:START -->
<!-- STACKOVERFLOW:END -->

Create stack-overflow-workflow.yml in your workflows folder with the following contents, replace 4214976 with your StackOverflow user id:

name: Latest stack overflow activity
on:
  schedule:
    # Runs every 5 minutes
    - cron: '*/5 * * * *'
  workflow_dispatch:
jobs:
  update-readme-with-stack-overflow:
    name: Update this repo's README with latest activity from StackOverflow
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v2
      - uses: gautamkrishnar/blog-post-workflow@master
        with:
          comment_tag_name: "STACKOVERFLOW"
          commit_message: "Updated readme with the latest stackOverflow data"
          feed_list: "https://stackoverflow.com/feeds/user/4214976"

See the result!

Popular Sources

Following are the list of some popular blogging platforms and their RSS feed urls:

Name	Feed URL	Comments	Example
Dev.to	`https://dev.to/feed/username`	Replace username with your own username	https://dev.to/feed/gautamkrishnar
Wordpress	`https://www.example.com/feed/`	Replace with your own blog url	https://www.gautamkrishnar.com/feed/
Medium	`https://medium.com/feed/@username`	Replace @username with your Medium username	https://medium.com/feed/@khaosdoctor
Medium (Sub Domain)	`https://username.medium.com/feed`	Replace username with your Medium username	https://timsneath.medium.com/feed
Stackoverflow	`https://stackoverflow.com/feeds/user/userid`	Replace with your StackOverflow UserId	https://stackoverflow.com/feeds/user/5283532
StackExchange	`https://subdomain.stackexchange.com/feeds/user/userid`	Replace with your StackExchange UserId and sub-domain	https://devops.stackexchange.com/feeds/user/15
Ghost	`https://www.example.com/rss/`	Replace with your own blog url	https://blog.codinghorror.com/rss/
Drupal	`https://www.example.com/rss.xml`	Replace with your own blog url	https://www.arsenal.com/rss.xml
Youtube Playlists	`https://www.youtube.com/feeds/videos.xml?playlist_id=playlistId`	Replace `playlistId` with your own Youtube playlist id	https://www.youtube.com/feeds/videos.xml?playlist_id=PLJNqgDLpd5E69Kc664st4j7727sbzyx0X
Youtube Channel Video list	`https://www.youtube.com/feeds/videos.xml?channel_id=channelId`	Replace `channelId` with your own Youtube channel id	https://www.youtube.com/feeds/videos.xml?channel_id=UCDCHcqyeQgJ-jVSd6VJkbCw
Anchor.fm Podcasts	`https://anchor.fm/s/podcastId/podcast/rss`	You can get the rss feed url of a podcast by following these instructions	https://anchor.fm/s/1e784a38/podcast/rss
Hashnode	`https://@username.hashnode.dev/rss.xml`	Replace @username with your Hashnode username	https://polilluminato.hashnode.dev/rss.xml
Google Podcasts	`https://podcasts.google.com/feed/channelId`	Replace `channelId` with your Google podcast channel Id	https://podcasts.google.com/feed/aHR0cHM6Ly9mZWVkcy5zb3VuZGNsb3VkLmNvbS91c2Vycy9zb3VuZGNsb3VkOnVzZXJzOjYyOTIxMTkwL3NvdW5kcy5yc3M=
Reddit	`http://www.reddit.com/r/topic/.rss`	You can create an RSS feed by adding '.rss' to the end of an existing Reddit URL. Replace `topic` with SubReddit topic that interest you or localized to you.	http://www.reddit.com/r/news/.rss
Analytics India Magazine	`https://analyticsindiamag.com/author/author_name/feed/`	Replace `author_name` with your name	https://analyticsindiamag.com/author/kaustubhgupta1828gmail-com/feed/
Feedburner	`https://feeds.feedburner.com/feed_address`	Replace `feed_address` with your Feedburner feed address	https://feeds.feedburner.com/darkwood-fr/blog
Tumblr	`https://blog_name.tumblr.com/rss` or `https://example.com/rss`	You can create an RSS feed by adding '/rss' to your main blog page or to your own domain if it is configured. Replace `blog_name` with your blog name	https://goggledoddle.tumblr.com/rss

Examples

Update 22-July-2020

Thanks for the overwhelming response everyone, Just added StackOverflow support to the project. Now you can have your dev.to posts along with your StackOverflow activity in your GitHub readme. The action now supports having multiple instances on the same repo. See an example readme:

God Mode in browsers: document.designMode = "on"

Gautam Krishna R — Thu, 16 Jul 2020 10:40:11 +0000

Just type document.designMode = "on" on you favourite browser's devtools and see the magic.

It will make any website editable:

Skipping the Chrome "Your connection is not private" warning

Gautam Krishna R — Sun, 03 May 2020 15:14:39 +0000

You guys might have encountered a warning saying that "Your connection is not private" while doing some web development. This tutorial is about how you can skip the same. (Feel free to skip the tutorial, if you are not using chrome or chromium-based browser)

You will get this kind of error usually when you are not using localhost as the domain for development. This is implemented to make your browsing experience more secure. If you are visiting a website via HTTPS and the server doesn't support it, you are vulnerable to the normal attacks that can happen when accessing a website via HTTP. You are vulnerable to the MIM (Man in the middle) attacks, these attacks a quite serious since it can gather private information like usernames and passwords from the victim.

But having this feature enabled while developing a web application is a headache. Organizations can set browser policies to prevent accessing such pages especially if you are using a machine provided by the company. The screen will look below:

As you can see that there will be no skip button on this. The only way you can access the page is by setting up a correct certificate or launching the Chrome browser in insecure mode via flags.

But there is a cheat code you can use to skip this warning 😄 Just type "thisisunsafe" in the chrome warning message page. (Not in the address bar, just type it in the page) Boom! 💣 You can see that the warning message disappeared.

To test this yourself you can visit: https://self-signed.badssl.com/

Congrats, You've just added a permanent exception for the corresponding domain for the SSL certificate check. 😄

Technical details

Chromium source code: https://chromium.googlesource.com/chromium/src/+/master/components/security_interstitials/core/browser/resources/interstitial_large.js
Stackoverflow thread: https://stackoverflow.com/questions/35274659/does-using-badidea-or-thisisunsafe-to-bypass-a-chrome-certificate-hsts-error

Stay safe, Happy coding.... 😷

Microsoft Student Partners – Geek is the new rockstar

Gautam Krishna R — Sun, 25 Jun 2017 16:35:48 +0000

The Microsoft Student Partner (MSPs) Recruitment for the year 2017 will be happening from 13th to 10th July. This is a shoutout to all the students out here… Don’t miss the chance for being a part of this amazing community… ðŸ™‚

Who’s a Microsoft Student Partner

From wikipedia:
“The Microsoft Student Partners is a worldwide recognizable program to sponsor students majoring in disciplines related to technology. The MSP program enhances students’ employability by offering training in skills not usually taught in academia, including knowledge of Microsoft technologies.”

Simply the Microsoft Student Program is a great way for students to get involved in the world of technology. It’s a great program by Microsoft to empower students who are passionate about Microsoft technologies.

Microsoft Student Partners do incredible things:

Run workshops and hackathons
Host tech events and give demos on campus
Work alongside Microsoft professionals, assisting them during hackathons and events
Join exclusive Microsoft training events to learn the latest technologies
Grow and nurture a community of students and faculty on-campus and online
Enhance their skills and career opportunities
Get amazing jobs And much more…

Perks of being an MSP

This is a question that is asked by most students about the program. Is it worth something being an MSP?. Why MSP?, Let’s find out….

Exclusive event invitations

Being an MSP will increase your chances for being selected in Microsoft invite only events, conferences and meetups. You can meet amazing people in these events.

Free MSDN subscription

This is one of my favorite perks being an MSP. Being an MSP, Microsoft will give you a free MSDN (Visual studio enterprise) subscription (Which costs around $5,999). It will give you access to all Microsoft products for free. You can easily download and use genuine Microsoft products from your MSDN subscription portal. You can get up to 5 genuine serial keys for most products completely for free. What I liked most about the MSDN subscription is that it gave me Visual studio Enterprise and Windows 10 Enterprise for free. It also includes free tons of Azure credits to run up to 3 high end VPS VMs simultaneously. It also offers one to one support on all Microsoft products.

Learn new technologies and programming languages

Being an MSP, you will receive exclusive access to Many online learning platforms like Microsoft E-Learning, Xamarin university etc. It will help you to add up to your skillset and sharpen your current skills.

Networking opportunities

Being an MSP you will be added to an exclusive closed group of MSPs and microsoft employees. You will recieve a @studentpartner.com email ID, it will give you access to the Yammer network and the office 365 portal. Using this email ID you will be eligible for 1TB of microsoft one drive cloud storage space. You will also get a free office 365 with all of its 3rd party extensions. Participating in MS events will allow you to grow your network with MVPs & Microsoft Employees. Whether you have any doubts related to anything under the globe, there will be at least a single person to help you out in our community.

Internship & Recruitments

Microsoft may offer you internships if you perform well as an MSP. You may also land into a job offer if you are the best.

Boost your resume

Experience being an MSP will be surely the best thing to put in your resume. It will make you stand out from the crowd. It will surely catch the eyes of the recruiters.

Beta testing

Microsoft announces internal beta test for most of its products. As a student partner. if you apply for any of the Microsoft’s beta test programs, you have a high chance of being selected. I used my student partner ID to apply for Microsoft’s Zo project and I was selected as a beta tester. Being an MSP will keep you updated about Microsoft’s new products and its releases.

Swags, swags and swags…

Whether you are attending an MS event or hosting an event, Microsoft will shower you with lots and lots of goodies like tee shirts, headphones, or even a brand new windows phone… ðŸ˜€

Eligibility criteria for MSP 2017

All applying students must be enrolled full time in an accredited academic institution from July 1, 2017 through June 1, 2018 and all students must have valid identification.

Anyone who satisfies the above requirements can apply to the program. The person need not have to be a computer science student to apply for MSP program. I know many MSPs who are B-School students, One of my friend MSP is even studying B.S in Botany.

Prerequisites

You must make sure that you have the following things in your hand (Taken from official page):

Student Verification: Scan and upload your college ID Card in an Online Drive. The Uploaded ID Card link should be directly viewable and the image should be properly visible.
Video Challenge: Please create a 1-minute or less video explaining why you should be a Microsoft Student Partner on your campus. Elaborate on technical skills, events you’ve been involved with, and describe how you would motivate students on your campus to learn about technology. The video should be uploaded on Youtube and made publicity visible. Only applications with completed videos will be reviewed. You can take reference as to how to upload a video on Youtube from here

If you have any questions about the MSP program or the application process, please contact: newmsp@microsoft.com

So what are you waiting for? Apply now! ðŸ˜€

Know more: https://msdn.microsoft.com/en-in/microsoftstudentpartners.aspx
Apply now: https://aka.ms/JoinMSP2017
Apply now (India): https://www.microsoft.com/en-in/campaign/msp2017/default.aspx

Originally published on http://www.gautamkrishnar.com/microsoft-student-partners/

An Introduction to NumPy

Gautam Krishna R — Wed, 19 Apr 2017 15:34:15 +0000

Numerical python or simply NumPy is one of the best modules to perform scientific computing in python. It is extensively used for data science as well as image manipulation using python. I recently learned how to use this module effectively in my projects, when started learning Machine learning and Data Science using python. I was amazed with the features of NumPy and I found it quite interesting to work with NumPy. Within some hours of usage i fell in love with it.

What is NumPy

Numpy’s GitHub readme defines it as:
NumPy is the fundamental package needed for scientific computing with Python. This package contains:

a powerful N-dimensional array object
sophisticated (broadcasting) functions
tools for integrating C/C++ and Fortran code
useful linear algebra, Fourier transform, and random number capabilities.

It derives from the old Numeric code base and can be used as a replacement for Numeric. It also adds the features introduced by numarray and can be used to replace numarray.

Simply NumPy is an open source python library that allows us to do scientific calculations in python. It has superpowers to magically support daunting vector and matrix computations. The core of the NumPy package, is the ndarray object. This encapsulates n-dimensional arrays of homogeneous data types, with many optimizations for performance. Narrays are created in contrast to Python’s built-in list data structure.

Why NumPy

As i said earlier, NumPy is created to overcome the limitations of python’s list data structure. We can use python lists instead of NumPy to perform various calculations like matrix multiplication, vector products etc. Using NumPy instead of lists will not only improve the performance of the code, but also will reduce the number of lines of the code. In this blog i will compare list datatype with numpy.

Install NumPy

You can use pip to install NumPy. If you don’t have python and pip installed, you can download it from here. After installation, use the command below to install NumPy:

pip install numpy

Now the NumPy will be installed on your machine.

Numpy v/s Lists

Python has a powerful built in data type known as lists. It has everything in its superpower to make it useful for almost any advanced scientific applications, but it is still limited while comparing to NumPy’s array data type. Lets compare python lists with NumPy arrays. I will be using python shell in this examples. Just type the python command to launch the shell.

You can create a NumPy array using the code below:

>>> import numpy as np
>>> a = np.array([1,2,3,4])
>>> a
array([1, 2, 3, 4])

You can create a new python list containing same elements using the code below:

>> b = [1, 2, 3, 4]
>>> b
[1, 2, 3, 4]

lets print both of them to the console:

>>> for element in a: # Numpy Array
 ...     print(element)
 1
 2
 3
 4
 >>> for element in b: # Python List
 ...     print(element)
 1
 2
 3
 4

As you can see that NumPy array works exactly the same way as list. We can simply use a loop to print its elements. Even though both looks the same, there’s some difference with the python lists and NumPy arrays. We can simply use the code below to add a new element to the list:

>>> b.append(5)
>>> b
[1, 2, 3, 4, 5]
>>> b += [6,7]
>>> b
[1, 2, 3, 4, 5, 6, 7]

We can’t do the same in NumPy arrays, It will throw an error:

>>> a.append(2)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'numpy.ndarray' object has no attribute 'append'
>>> a += [5,6]
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
ValueError: operands could not be broadcast together with shapes (4,) (2,) (4,)

As you can see that both doesn’t works in the case of NumPy array. Lists use plus operator for concatenation, but NumPy arrays use the plus operator differently. So lets check it out. Lets find the element wise sum of an array using NumPy and python lists:

>>> b2 = [] # Temporary List
>>> for element in b: # Using List
...     b2.append(element + element)
>>> b2
[2, 4, 6, 8, 10, 12, 14]


>> a + a # Using NumPy array
array([2, 4, 6, 8])

Pretty easy right!😀

NumPy in action

NumPy arrays treat plus operator(+) as the element wise addition operator. We can also use it to add two different arrays, or even we can use it to perform scalar addition to an array. NumPy array treats multiplication operator(*) as matrix multiplication operator. Most operators act element wise in NumPy arrays. Lets see the superpowers of NumPy arrays 😀:

>>> a # NumPy Array
array([1, 2, 3, 4])
>>> a2 = np.array([4,5,6,7]) #New NumPy array

>>> a + a2 # Matrix addition
array([ 5,  7,  9, 11])
>>> a + 3 # Addition with a scalar
array([4, 5, 6, 7])

>>> a * a2 # element wise multiplication
array([ 4, 10, 18, 28])

>>> a * 3 # Multiplication with a scalar
array([ 3,  6,  9, 12])
>>> a ** 3 # Power operator
array([ 1,  8, 27, 64], dtype=int32)
>>> a.sum() # Sum of elements in a
45

As you can see that the matrix arithmetic works like a breeze in NumPy arrays. We don’t need to use the annoying loops anymore to perform those 😉 . If you have an N dimensional matrix you can use NumPy to perform all these operations on it. NumPy has everything built in to perform these operations effectively by providing an abstract layer to you. For a 2D matrix lets check these operations:

>>> a = np.array([[1,2,3],[4,5,6],[7,8,9]])
>>> a
array([[1, 2, 3],
       [4, 5, 6],
       [7, 8, 9]])
>>> a2 = np.array([[10,11,12],[13,14,15],[16,17,18]])
>>> a2
array([[10, 11, 12],
       [13, 14, 15],
       [16, 17, 18]])

>>> a[0] # access each row
array([1, 2, 3])

>>> a[0][1] # access element i=0 j=1
2
>>> a[0,1] # access element i=0 j=1
2


>>> a * 2 # Scalar Multiplication
array([[ 2,  4,  6],
       [ 8, 10, 12],
       [14, 16, 18]])

>>> a + a2 # Addition
array([[11, 13, 15],
       [17, 19, 21],
       [23, 25, 27]])

>>> a * a2 # Multiplication
array([[ 10,  22,  36],
       [ 52,  70,  90],
       [112, 136, 162]])

>>> a - a2 # Subraction
array([[-9, -9, -9],
       [-9, -9, -9],
       [-9, -9, -9]])

>>> inva = np.linalg.inv(a) # a inverse
>>> inva
array([[ -4.50359963e+15,   9.00719925e+15,  -4.50359963e+15],
       [  9.00719925e+15,  -1.80143985e+16,   9.00719925e+15],
       [ -4.50359963e+15,   9.00719925e+15,  -4.50359963e+15]])

>>> np.linalg.det(a) # Determinant of a
6.6613381477509402e-16

>>> np.diag(a) # Diagonals of a
array([1, 5, 9])

>>> np.trace(a) # Sum of diagonals
15

>>> x = np.linalg.eig(a) # Eigen values and eigen vectors of a
>>> x
(array([  1.61168440e+01,  -1.11684397e+00,  -1.30367773e-15]), array([[-0.23197069, -0.78583024,  0.40824829],
       [-0.52532209, -0.08675134, -0.81649658],
       [-0.8186735 ,  0.61232756,  0.40824829]]))
>>> x[0] # eigen value
array([  1.61168440e+01,  -1.11684397e+00,  -1.30367773e-15])
>>> x[1] # eigen vectors
array([[-0.23197069, -0.78583024,  0.40824829],
       [-0.52532209, -0.08675134, -0.81649658],
       [-0.8186735 ,  0.61232756,  0.40824829]])

>>> a ** 2 # Power
array([[ 1,  4,  9],
       [16, 25, 36],
       [49, 64, 81]])

>>> a2.T # Transpose of a2
array([[10, 13, 16],
       [11, 14, 17],
       [12, 15, 18]])

>>> a.mean() # mean
5.0

>>> a.var() # variance
6.666666666666667


>>> a = np.matrix([[1,2,3],[4,5,6],[7,8,9]])
>>> a # NumPy Matrix type
matrix([[1, 2, 3],
        [4, 5, 6],
        [7, 8, 9]])

NumPy also have a matrix type in addition to NumPy arrays. The official documentation recommends using matrix type for matrix operations. NumPy performs well for multidimensional matrices as well. In addition with these standard operations NumPy has several other functions available to make your programs a lot more simpler. Lets see some of the examples:

>>> a = np.array([1,2,3])
>>> a
array([1, 2, 3])
>>> np.sqrt(a)
array([ 1.        ,  1.41421356,  1.73205081])
>>> np.sin(a)
array([ 0.84147098,  0.90929743,  0.14112001])
>>> np.cos(a)
array([ 0.54030231, -0.41614684, -0.9899925 ])
>>> np.tan(a)
array([ 1.55740772, -2.18503986, -0.14254654])
>>> np.log(a)
array([ 0.        ,  0.69314718,  1.09861229])
>>> np.exp(a)
array([  2.71828183,   7.3890561 ,  20.08553692])

So simply NumPy treats an array like a vector or a mathematical object. To do operations on list, you need to use a for loop. Since for loops are slow it may take more time to perform various operations while compared to NumPy arrays. Lets do some operations on vector and matrices:

>>> a
array([[1, 2, 3],
       [4, 5, 6],
       [7, 8, 9]])
>>> a2
array([[10, 11, 12],
       [13, 14, 15],
       [16, 17, 18]])
>>> a.dot(a2) # Dot product a.a2
array([[ 84,  90,  96],
       [201, 216, 231],
       [318, 342, 366]])
>>> a2.dot(a)# Dot product a2.a
array([[138, 171, 204],
       [174, 216, 258],
       [210, 261, 312]])
>>> np.dot(a,a2) # Dot product a.a2
array([[ 84,  90,  96],
       [201, 216, 231],
       [318, 342, 366]])
>>> a = np.array([2,3])
>>> a2 = np.array([4,5])
>>> maga = np.linalg.norm(a) # magnitude of a
>>> maga2 = np.linalg.norm(a2) @ magnitude of a2
>>> angle = np.arccos(a.dot(a2) / (maga * maga2)) # angle between a and a2
>>> angle # in radian
1.1352271440633694

To generate random matrices for testing just use:

>>> z = np.zeros(10) # Generates zero array
>>> z 
array([ 0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.,  0.])

>>> np.zeros((5,5)) # 5*5 Zero matrix
array([[ 0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.],
       [ 0.,  0.,  0.,  0.,  0.]])

>>> np.ones((5,5)) # 5*5 Unit matrix
array([[ 1.,  1.,  1.,  1.,  1.],
       [ 1.,  1.,  1.,  1.,  1.],
       [ 1.,  1.,  1.,  1.,  1.],
       [ 1.,  1.,  1.,  1.,  1.],
       [ 1.,  1.,  1.,  1.,  1.]])

>>> np.random.random((5,5)) # random 5*5 array with elements < 0
array([[ 0.95798455,  0.24020745,  0.62194033,  0.93840616,  0.40785382],
       [ 0.01294948,  0.7228686 ,  0.67448551,  0.20403856,  0.046528  ],
       [ 0.63331545,  0.89097084,  0.01754348,  0.17084474,  0.32112247],
       [ 0.97881143,  0.83247286,  0.65629919,  0.21386575,  0.72251318],
       [ 0.20167738,  0.24018638,  0.85572554,  0.7706282 ,  0.80284553]])

>>> np.random.randn(5,5) # Random 5*5 matrix
array([[ 0.16603893,  1.14554164,  0.40170708,  0.52864275,  1.50740231],
       [ 0.13218522, -0.20418907, -1.09940842, -1.25180194,  0.6859655 ],
       [ 0.09053258,  1.11002797,  0.1455936 , -0.33915414,  0.25604553],
       [ 0.96807902, -0.03155716, -0.79001785,  0.4567955 , -1.93929055],
       [-1.38540075, -1.82320053,  0.02358358, -1.13975953, -1.23515682]])

Summing up all these, even though the NumPy arrays contain several advanced functions when compared with lists datatype, it can’t be considered as a replacement for python lists. NumPy is really useful if you want do do some mathematical operations on an array, If we are using a list instead of NumPy arrays we need to traverse each and every elements using a loop, it will significantly reduce the overall performance of the program. Using NumPy will not only improves the performance of the program, but also adds advanced functionalities to the code.

Visit the official docs to learn more: https://docs.scipy.org/doc/numpy/

Any suggestions to this article is always welcome. Please don’t forget to comment on this article if you found any mistakes 😀

Originally published on: gautamkrishnar.com

DuckDuckGo – The search engine redefined

Gautam Krishna R — Thu, 05 Nov 2015 19:49:25 +0000

DuckDuckGo is a search engine just like Google and Bing but the only difference is that it is open source. It is powered by an awesome open source community. Also, it won’t track your searches to show you personalized ads as google does. I hope you have heard the recent news that Google is tracking users even in incognito or private browsing mode. You can search and surf the web anonymously using DuckDuckGo.

This blog will help you to understand more about DuckDuckGo and some of its amazing features:

Social media bio

DuckDuckGo lets you see the social media profiles of people without leaving the search engine. So, search for @gautamkrishnar and you can see my twitter profile information on the same page. If you want to see a person’s Google+ profile you can simply search g+ followed by a person’s name. Another cool feature is that you can perform a search on other website’s search box using DuckDuckGo, we call it bangs. If you want to search someone in LinkedIn just type !linkedin followed by the person’s name. You can use !a to search for some products in amazon.com. At the time of writing this post, there are 13,505 bangs available. You can see more bangs here.

App store search

Search for any popular apps in DuckDuckGo and it will show a carousel full of apps with similar names. Another useful feature shows alternatives to apps in a neat carousel. Tired of MS Office? Just search for “Alternative to Word” and see the results for yourself. It also works for Web services, so you can even search for “Alternative to DuckDuckGo“ if you like.

Stopwatch

DuckDuckGo has a stopwatch, simply search for the stopwatch.

Shorten and expand links

Want to find out which websites are at the end of shortened links such as http://bit.ly/blahbla? Open DuckDuckGo and search “expand http://bit.ly/blahbla”. Also, if you want to shorten long URLs, enter “shorten www.gkrish.xyz/isabella” to get a URL shortened using is.gd.

Generate passwords

DuckDuckGo also helps you to generate strong passwords. Just head to DuckDuckGo and search for “Password 20”. You’ll see a strong 20-character password. You can replace the number to change the number of characters in the password. If you find those random passwords hard to remember, you can make DuckDuckGo generate XKCD-style passwords. These passwords comprise four common words put together, which are easy to remember and hard to crack, and were first suggested in the popular Web comic XKCD. For these passwords, search “Random passphrase”.

Change case and check the number of characters

Want to change a sentence to Title Case, lowercase or UPPERCASE? Just add one of those three terms before your sentence and key it in on DuckDuckGo.

Another interesting feature is the number of characters. You can quickly check how many characters are in your search query by adding “chars” before or after the query. Try searching for “chars I really like this article” without the quotes and see the results.

rhyme with

You read that right. DuckDuckGo also has an instant answer that finds rhyming words. Search “rhyme with heart” or any other word you can think of. You’ll see an instant answer with rhyming words, from RhymeBrain.

Calendar

Google and DuckDuckGo can show you the current date and time. But DuckDuckGo goes one step further by supporting the calendar as an instant answer. Search “calendar” to see one, with the current date highlighted. It can also show you the calendar for any month and year. Try searching for “calendar january 1899“.

Loan calculator

Another useful DuckDuckGo instant answer is a loan calculator. It’s useful if you want to see how much your monthly installments will be and how much interest you’ll be paying in total. The search query for this is a little hard to remember, like the terms and conditions for most loan agreements. It is “loan AMOUNT at INTEREST with PERCENT down for DURATION”. All the uppercase words are what you need to key in – total amount, interest rate, down payment percentage and the duration of the loan. We tried “loan Rs 50,00,000 at 4.5% with 25% down for 15 years” to get the answer in the picture below.

Cool features for developers

There are several instant answers that will be useful to software developers but might be less important for the general public. Here’s a quick list of what DuckDuckGo can do:

Miscellaneous features

Run a search for ‘xkcd’ and DuckDuckGo will automatically fetch a most recent comic right there on the search result page.
You can get DuckDuckGo to explain you an emoticon straight away. Just type its code 😀
DDG lets you use “” to go to directly to the first search result after inputting a query. We call this “i am feeling ducky” 😀

There are several other instant answers on DuckDuckGo that are quite cool, but not very useful. Here’s a quick list of some of the best.

As DuckDuckGo is an open-source search engine, its possibilities are limitless. There are several other things DuckDuckGo can do. Check out this page.

DuckDuckGo is powered by an open-source community, it has a team of dedicated open source developers who contribute part-time to it. We can even say that DuckDuckGo has more features than google because the developer community is adding more and more features day by day. I am really happy to say that I am also a part of that awesome community (DuckDuckGo Community Leader 😀).

Update: At the time of writing this post, DDG had stopped accepting external contributions. Now they are working on developing Privacy focused browsers and extension: https://github.com/duckduckgo