DEV Community: vaidhyanathan

[help needed] Rails application upgrade from 3.2 to 5.2

vaidhyanathan — Wed, 23 Oct 2019 19:07:35 +0000

We were planning to upgrade my rails application which is a monolith application and having around a million LOC. What would be the best way to upgrade?

The following are the options
1) 3.2 -> 5.2 (Separate branch running for a long time, Total mess up)
2) 3.2->4.2 ->5.2 ((Separate branch running for a short time)
3) 3.2->4.0->4.1->4.2->5.0->5.1->5.2(Running in master periodically, But we may need to write code in particular version which may become obsolete in the next coming version)

I am planning to follow approach 3, but I am not sure which part is added in version (4.0 to 5.1) and removed in(4.2 to 5.2).

Did anyone did rails upgrade for their application, what was the approach.

PS: I saw many posts around this, but nothing solid.

Deepest Truth in Huawei problem(Monopoly) and how vulnerable we all are!!!

vaidhyanathan — Thu, 30 May 2019 16:19:02 +0000

DISCLAIMER: This topic is created only for technology discussion and possible solutions, not about any political related discussion.

I and my colleague started discussing Google's ban on Huawei phones. Which may be a death blow to Huawei. Jokily we mentioned that what will happen of Google ban's service in our country or Oracle immediately imposes the ban on usage of MYSQL and JAVA. Basically, the monopoly in most of the internet segment is the key problem.

UI frameworks such as Angular, React, Polymer etc were controlled by Google, facebook/ twitter. Almost all the popular frameworks and technologies are being controlled by them. Which make us vulnerable and need to bend for all those regulation imposed by them.

Practically it takes years for Frameworks like Vue to take off. While shitty frameworks like Angular1.0 which takes off much easier and almost everybody started using it because of Google.

I personally feel that the internet and important technologies should be free and open and always available to all. And open source projects should be truly open and should continue to deliver good things at all cost.

We need an alternate way to solve those problems. Internet and key technologies should not be restricted by ethnicity, Location, political factors etc.

The alternate should be truly distributed, easily available to all the people.

Please suggest your thoughts here. Do we need an alternate solution? If so what will it be?

Help needed on Deep learning design

vaidhyanathan — Thu, 09 May 2019 18:41:34 +0000

I am learning pytorch. The use case is I wanted to give a personalised recommendation to users based on region. The problem is I needed to create multiple Graphs which are more or less the same(But not exactly equivalent) based on each region.

The solution some people suggested is to create separate graphs for each region and use them. But I am not convinced with this approach, because region may quickly grow.

Can somebody help?

Share your thoughts on Google IO

vaidhyanathan — Wed, 08 May 2019 15:32:32 +0000

I used to watch Google IO. I used to like them, But this time it was too boring. I feel everything announced in the IO is just an update.

I hate 3a and 3a XL. The spec is low and we can easily buy that kind of phones from any Chinese manufacturers.

Share your thoughts here

How to debug in production like a pro?

vaidhyanathan — Tue, 07 May 2019 12:35:50 +0000

If you are working on the customer-facing products, then you might have experienced the production issues. For others, Production issues are the high-level bugs that are in production and affecting the customers to a great extent.

Then and their production issue happens because of the ignored test case, No test cases or environmental issues etc.

I follow the below things when there is a production issue. The issue may be notified to you by either your team fellow/support people/ Twitter/other sources.

Step 1: Classify the Issue(0-10 seconds)

Check and classify the issue of what type it is. The issue usually belongs to one of the below categories.

1) DOS - People might complain that there is connection time out issue
2) Slow response - Almost all/ certain type of requests are slow, which might lead to the first case
3) improper behaviour - The product is not working properly
4) Login issues - Session based issues.
5) High error rate - These are the silent issue that only developers know about.

Step 2: Check your monitoring tool(1 - 5 mins)

Almost all organisation will be having some tool for monitoring the servers. If you are a new startup/product who didn't use any tool, It is the right time to check it out.

Check the stats in the order below

Average Transaction time

DOS and slow request. Check if all request is a slow one or a particular type.
If all requests are slow, then it is a problem with either current app server/ Dependency services like MySQL, Redis, Kafka etc. So check the below for respective machine

Machine status

Including checks for CPU(> 80%), Free memory(<10%), Active threads/ process(>90% of allocated), request wait time before processing(< 2s), GC Time(If you use GC based language > 10s),Free Disk size(< 5% ), heap memory(< few MBs).

If any of the above occurs, Take the necessary steps below.
High CPU - wait for some time/ check threads/process which takes a long time. It will always be code issue or increase machine size
Free memory - Always code issue. Check the last deployment or increase machine size
Active threads - Too many active threads is because of too many requests got affected because of downstream / code issue. Check accordingly 5.GC Time - Your code is the problem. Check logs to see where GC occurs. Get a thread dump and check it out. 6.Free Disk size - Your code wrote something in the machine and it is not cleaned up. Check your app server for folder wise disk usage.
Head memory - Your code used many objects and didn't clear it out.
Dependency services - Check the above for all the services that your app depend upon like MYSQL, Redis, etc

Most of the above cases can be fixed by reverting your code.

Logs(5 - 10 mins)

If the above turns out empty handed, monitor your logs for recent errors. It will give you an overview or idea of what is happening. If you are using some log tool use dashboard for error module wise or region wise.

All the above check should reveal all the necessary things needed to fix. If not check with experts in your team immediately.

Step 3 - Check affected areas(1 - 10 mins)

Check what are the areas that were affected by the issue. If it is customer facing then immediately inform affected customers or severely affected customer. There are chances that downstream services might be affected. Check the below list and tick the areas where it was affected

DB/Cache/Downstream services data corruption
Data not sent to downstream
Replication Lag

Step 4: Fix the issue & deployment (3 - 30 mins)

If it is severe fix the issue by making the quick patch and update it all server. It should be solid and might require approval.
If it not server try to run all the test coverage and fix the main issue.
For downstream related problems write a script to correct the data. If data is completely corrupted use backup for recovery but customer concern is required.

Step 5: RCA and test case

Put down a neat RCA and inform the team. write down test cases for all those scenarios.

This is what I follow which enables me to fix the issue in production between 10 - 20 mins.

** If you follow some good practice. Let me know. **

My ultimate VSCode configuration

vaidhyanathan — Sun, 05 May 2019 14:19:22 +0000

I used to love vim a lot but now a days I switched to VSCode because of the delicate balance between the UX and plugins.

The following is my settings.json file. You can probably use the same too.

Please do add your configuration too

Disabled Minimap - since we are writing modular code there is no need for this.

"editor.minimap.enabled": false

Autosave Enabled - AutoSave enabled so that there is no need for cmd+s always

    "files.autoSave": "afterDelay",
    "files.autoSaveDelay": 5000,

Exclusion - These are the large files and folders which will make VSCode cry . Please do add the same to your configuration also.

    "files.watcherExclude": {
        "**/.git/objects/**": true,
        "**/.git/subtree-cache/**": true,
        "**/node_modules/**": true,
        "**/tmp/**": true,
        "**/.git": true,
        "**/.svn": true,
        "**/.hg": true,
        "**/CVS": true,
        "**/.DS_Store": true,
        "**/node_modules": true,
        "**/bower_components": true,
        "**/dist/**": true,
        "**/log/**": true,
        "**/logs/**": true,
        "**/.fdk/**": true
    },
    "files.exclude": {
        "**/.git/objects/**": true,
        "**/.git/subtree-cache/**": true,
        "**/node_modules/**": true,
        "**/tmp/**": true,
        "**/.git": true,
        "**/.svn": true,
        "**/.hg": true,
        "**/CVS": true,
        "**/.DS_Store": true,
        "**/node_modules": true,
        "**/bower_components": true,
        "**/dist/**": true,
        "**/log/**": true,
        "**/.fdk/**": true
    },
    "search.exclude": {
        "**/.git/objects/**": true,
        "**/.git/subtree-cache/**": true,
        "**/node_modules/**": true,
        "**/tmp/**": true,
        "**/.git": true,
        "**/.svn": true,
        "**/.hg": true,
        "**/CVS": true,
        "**/.DS_Store": true,
        "**/node_modules": true,
        "**/bower_components": true,
        "**/dist/**": true,
        "**/log/**": true
    },

Tab Size - We can still be friends with 2 spaces for tab.Change this based on your source code.

"editor.tabSize": 2,

Final Json for VSCode


{
    "ruby.intellisense": "rubyLocate",
    "gitlens.advanced.messages": {
        "suppressShowKeyBindingsNotice": true
    },
    "workbench.statusBar.feedback.visible": false,
    "search.location": "sidebar",
    "window.zoomLevel": 0,
    "workbench.iconTheme": "vscode-icons",
    "files.watcherExclude": {
        "**/.git/objects/**": true,
        "**/.git/subtree-cache/**": true,
        "**/node_modules/**": true,
        "**/tmp/**": true,
        "**/.git": true,
        "**/.svn": true,
        "**/.hg": true,
        "**/CVS": true,
        "**/.DS_Store": true,
        "**/node_modules": true,
        "**/bower_components": true,
        "**/dist/**": true,
        "**/log/**": true,
        "**/logs/**": true,
        "**/.fdk/**": true
    },
    "files.exclude": {
        "**/.git/objects/**": true,
        "**/.git/subtree-cache/**": true,
        "**/node_modules/**": true,
        "**/tmp/**": true,
        "**/.git": true,
        "**/.svn": true,
        "**/.hg": true,
        "**/CVS": true,
        "**/.DS_Store": true,
        "**/node_modules": true,
        "**/bower_components": true,
        "**/dist/**": true,
        "**/log/**": true,
        "**/.fdk/**": true
    },
    "search.exclude": {
        "**/.git/objects/**": true,
        "**/.git/subtree-cache/**": true,
        "**/node_modules/**": true,
        "**/tmp/**": true,
        "**/.git": true,
        "**/.svn": true,
        "**/.hg": true,
        "**/CVS": true,
        "**/.DS_Store": true,
        "**/node_modules": true,
        "**/bower_components": true,
        "**/dist/**": true,
        "**/log/**": true
    },
    "editor.fontSize": 16,
    "editor.tabSize": 2,
    "emmet.includeLanguages": {
        "html": "html",
        "javascript": "javascriptreact"
    },
    "files.autoSave": "afterDelay",
    "files.autoSaveDelay": 5000,
    "terminal.external.osxExec": "iTerm.app",
    "terminal.integrated.shell.osx": "zsh",
    "editor.wordWrap": "on",
    "editor.formatOnSave": true,
    "editor.formatOnPaste": true,
    "editor.minimap.enabled": false,
    "editor.fontLigatures": true,
    "workbench.editor.highlightModifiedTabs": true,
    "explorer.sortOrder": "modified",
    "files.trimFinalNewlines": true
}

How not to architect your project?

vaidhyanathan — Thu, 02 May 2019 17:06:09 +0000

I have been working with some of the most successful products in the Saas Industry. Here are some of the things I used to say a big NOOO ! Probably you must also do the same.

Wrong framework

Know your product and usecase and choose the correct framework. Choosing GoLang for web app development and ruby for high performance code is wrong . Have a good knowledge on all the frameworks and languages, choose wisely. There is no going back on this.
I see people choose Java for webapps because thier employess only knows Java, but inreality Java is no good for webapps.

Using V1.0

In the year 2013–2014, When asked people not to use angular they used to mock at me. After a couple of years Google ended it support. Simple Thumb rule is dont use any new frameworks/tools/standards till thier version 2.0/ atleast stable for 3 years. First few stable versions also will be having architect failures most of the times. So use some standard approach.

Hyper Scalabilty

Yes your read it right. People tend to perfect thier product to scale millions of request per second, but in reality it wont happen soon! Customer aquestation is a gradual process and it takes time to reach potential users. Even a killer product will take couple of years to pick up first few thousand active customers.So all the scalability efforts put during the initial products are worthless. It's like you made the yummest recipe but no one is there is take it.

Nanoservice(Heavy microservices)

Writing eveything as Microservice is the one of the things that usually kills the product soon. Microservices is only recommended if your product has a mamoth code base, which basically needs to be splitted to microservice ,and your are having resources to maintain. Starting the product with everything as microservice service which is a overkill. My recommendation is to start as modular monolithic and move to microservice if necessary.

Refactoring

Refactoring the whole code base is wrong. If there was some wrong decision taken in initial days, do refactor as much early as possible. Refactoring too much is a bug prone , vague and non productive job. Customers dont care what is behind they only see if things works. These kinds of problems can be solved by having a good Architect.

New Inhouse solutions

We cannot solve all the problems , even we have resources and talent. If there are some ready made solution, Please use it. It is not because of saving money. It is because existing solution usually takes all the learning from the community or customer. I saw many projects used thier inhouse search service instead of elasticsearch or Solr, With in couple of years they failed and return back to the open source solution.

The above are the major things I used to say a big no while designing new product.

Please add your suggestions also below.