DEV Community: Mohan Ganesan

Developers, Start Paying For Stuff, Will You?

Mohan Ganesan — Tue, 17 Jan 2023 11:03:45 +0000

I can sense your contempt. But hear me out here.

There are some counter-intuitive advantages to taming that inner instinct and natural pride a developer has, which makes him/her want to always develop stuff instead of paying for it. Here is a couple…

You will be able to relate: Till I started paying for stuff, I had a hard time relating to people who pay for stuff. That’s not a helpful mindset to have when you are developing products yourself that you expect people to pay. If you have to create high quality, you will need to experience high quality. Most of them cost some money.
You will see that it’s not just about the code: A right product means care, attention to detail, handling weird edge cases, proper documentation, UX, excellent onboarding, follow-up, support, and sound design. Signing up and paying for products like Intercom or Basecamp exposed me to the other aspects of product making and showed me first-hand what it ‘feels’ like to be a part of a pleasant experience.
Your app/project inherits the qualities of high quality paid products: Contrary to popular belief, open-source products aren’t that efficient at getting jobs done. It is because a random set of developers builds it from different parts of the world with different agendas. There is almost no research, no prioritization, and because developers use it, there is not much need for great onboarding, great UI, and support. Using Slack or Base camp makes your projects better. They allow you to consider more things, think broader and more organized get better consensus, and overall make for a better whatever-you-are-making. It is a little known secret that a product maker cannot make a great product without having experienced several great products themselves. Conversely, an excellent maker builds his creation by the combined influence of the ethos of all the products he/she has experienced until then. If you want to make great products, experience great products

I am the maker of Proxies API, a rotating proxy service that you will need to pay for if you want a reliable way to bypass IP blocks when web scraping.

6 Ways I Get Things Done Fast As A Bootstrapped Founder

Mohan Ganesan — Fri, 13 Jan 2023 05:16:41 +0000

Here is the productivity stack I use to get many things done first in my startup Proxies API

One of the ways to lose momentum is to forget your original strategy. So you need to load this up into your brain the moment you wake up, so you are not swayed by every blog post, podcast episode, and new growth hack and lose the moment that you have accumulated so painfully.

Remind yourself of the original strategy every day

I need to make myself smile every day. And I count them. I need to make sure I have these moments where I am happy with my work in total to about 300 till I can say that I have done an excellent job.

What’s your north star metric?

If something takes less than 2 minutes, do it.

*The two productivity tip that changed my life *

If you find yourself exhausted and find yourself cribbing, sulking, you need a break. You are taking access to work for granted. Make it scarce. Just stop working cold turkey. Go away from your computer and enjoy yourself till you feel the guilt. I love guilt. I need the pull of sin to come back to work. Sin is productivity if you do it right.
If you are overwhelmed, it probably because you are keeping too many things in your mind. Your brain cannot slow down because it is always going round and round all the things you need to do. It’s time for the brain dump.

The magic of the brain dump

If you have lost momentum, let us say after a long break, illness, or a holiday, it is normal for us to fail to pick up the energy we used to have. The more we push ourselves, the more we feel we are rooted to the floor with concrete. So we freeze and do nothing at all. In that case, stop and reduce the expectation to more or less zero. Now tell yourself that you will do the smallest action you can take to get going. This action almost as a rule has to be dumb and make you feel silly. But try it, it works for me. If you have to write a blog post like this one, the action could be to write the headline. Wait even that is too intelligent. The move could be, open the editor and create a new page. That’s it. Now build from there.

The author is the founder of Proxies API, a proxy rotation API service.

How To Write Blog Posts From The Subconscious?

Mohan Ganesan — Wed, 04 Jan 2023 12:50:12 +0000

Apart from working on my startup Proxies API, I also dabble in standup comedy from time to time.

One of the fascinating things about stand up material generation is that it is not that difficult to come up with jokes after you have been in it for a few years. Any comedian who has been around will tell you that. The most challenging part about stand up is coming up with an exciting premise.

You don’t come up with a premise; you identify one. From your real-life, ideally. That way, people can relate to it.

There is a parallel to writing blog posts here. The most challenging part of writing for me is not the words that go into the article, but coming up with exciting ideas that have the potential to be an article.

So coming back to stand up, I realized a couple of years into comedy that I need not worry about jokes and only worry about having a list of exciting premises.

At this point, I open up a channel for conversation with myself about the areas of my life that has always fascinated me.

a) Public behavior of people frustrates me.

b) My daughter/wife and all the frustrating/cute things they do.

c) I travel between the US and India a lot, the differences in their cultures fascinate me.

d) Life in Delhi, the way they name their streets, the heat, the cold, etc.

So what I do is I have several unfinished/running jokes in all these areas at any time. My brain can hold about 4–5 at a time. I try NOT to write the punch lines that turn these premises into jokes. Instead, the idea is to try to build the observation muscle inside me trained in these areas.

I typically have four open documents on my laptop ready and waiting for further ideas in these areas. And I expect the list to be long before I start writing punch lines.

I tell myself, ‘I will not write punch lines till I get to about 50 premises in each.’

All this sets up my subconscious to hunt for things that are interesting to talk about on stage. In the beginning, I ask myself every hour to see if there was something in these areas that came up. But eventually, even this kind of prompting is not needed, and the brain automatically starts noticing interesting premises until you have an abundance of them.

I applied the same to article writing. I know the few areas that I want to write.

I start asking myself a few times every day if I have any interesting thoughts about them.

After a week of doing this, I will have a bunch of headlines of articles enough to last a few months.

Then I start writing 5–6 articles simultaneously. I don’t write much in each, just the key ideas. I DONT expand on them. It is essential not to allow that part of the brain to kick in just yet.

The next day I try to expand on them across all six articles. Still more ideas. No growing on them.

At this point, the subconscious should take over, giving me further ideas and angles and even specific phrases that I can go into these articles. I allow this to brew for a week. I am just collecting them into the documents. A week later, I have enough to make an article that I could have never written consciously.

I sat to work on them one by one and put the whole thing together into a cohesive draft. I wait another couple of days to see if the subconscious has something more to say about them. After that, I schedule them for publishing.

The author is the founder of Proxies API, a proxy rotation api service.

Do 1 Scary Thing Every Day

Mohan Ganesan — Sat, 31 Dec 2022 06:09:09 +0000

Entrepreneurship is a ride of discomfort. You are almost as a rule doing things that are outside your comfort zone. I have in so deep I can’t find my way back to my comfort zone. While I was gone, someone came and walked away with it, I think. Or at least, that’s what it feels like for the time being.

a) I am not a writer, but I write.

b) I am a programmer at heart, but I market what I have built.

c) I reach out to people who might be interested in my product. That must be amazing to my mother if she could understand what I do.

d) I chat with people on my website and turn it into sales conversations where possible.

e) I follow up on sales conversations. That’s like the toughest thing in sales, they say, and I do it.

f) Right now, I feel like I am bragging, and I shouldn’t be writing this blog post. But what I have written is true. Unbelievable to me even, but true.

So I follow this rule of doing one scary thing for just 10 minutes every day, not the whole creepy thing. That’s too much. I might die. So let me enter the lion’s den, stay there for just 10 mins and get out. And the next day I do it again.

It’s gotten me this far. In the last year, I created a SAAS product, developed a website(Proxies API) to sell it, and started marketing and selling it every day. All unlike me, accept the programming part. It enabled me to build a business that, from the looks of it, will sustain me for a long time and has helped me to quit my job.

The author is the founder of Proxies API, a proxy rotation api service.

How I batch My Blogging Process for Greater Productivity

Mohan Ganesan — Sat, 24 Dec 2022 06:46:11 +0000

Since I launched my startup Proxies API, to date, I have created nearly 100 blog posts here on Medium and our blog combined. I have also posted about 30 articles designed explicitly for communities like Reddit, Quora, and Facebook.

That’s 130 blog posts in a year, bringing it to about 2.5 articles per week.

The consistency is staggering, considering my weekly average in my entire life before this, which was a big fat: ZERO.

One of the things that suddenly helped me get to this sudden level of productivity is merely batching the blogging process.

I divided it into four steps because I believe they are four different brain functions.

a) Hunting for and coming up with a list of ideas to write about

b) Writing the draft with gay abandon

c) Editing and making the selection publish worthy

d) Scheduling and pushing the publish button

So when I started this process, I spent time hunting down, getting inspiration from other bloggers I admire and listing out actual titles of articles I could write about and made a list of 100 strong. Yes, before I ever wrote a word of actual content.

It served as super motivation because I knew that I would love writing at least some of the articles I had listed. I could see the whole thing finished in front of me on WordPress and Medium. That was motivation enough. Then I decided that I would open ten drafts at a time. I think of this as opening ten channels in the brain which the brain can then go to work out simultaneously. So day 1, write just the main ideas of all ten articles quickly. Day 2 rolls in, I have some quotes rolling in my head, some old memories refreshed. Day 3, the thoughts are turning into insights and summaries of what exactly is the “core” of what I want to say. Day 4, I find my brain is coming up with a better turn of the phrase to describe the insights I tried to communicate the day before. It goes on sometimes for a couple of weeks for individual articles. In the last few days, I try to let it run on empty just to makes sure it is has exhausted all the ideas for now.

The editing process is more of a checklist. Reading and re-reading out loud, just to see if the language flows well and if there are too many jumps in the flow, enhancing the article with images, research, links where necessary, proofreading with Grammarly, etc.

The last step is scheduling the posts that are ready onto a calendar. What should go out when? How I can repurpose and cross-link content. Which channel to publish on etc. all this is written out. At this point, because of repurposing, there is something to be released every day or at least promoted various channels appropriate to the content.

The author is the founder of Proxies API, a proxy rotation api service.

Is This Your First Time Doing Something? It Is No Big Deal

Mohan Ganesan — Wed, 14 Dec 2022 06:57:28 +0000

One of the things I noticed that I tend to do is build up a milestone I want to achieve, especially “the first x” and make a big deal about it in my head. Like when I was building Proxies API, I would build this narrative that I couldn’t wait to finish developing. It creates weird anxiety that helps nobody. Then when I finished it, I would make a big deal about “the first customer.”

“If only I can get my first customer…. then it would be xxx”, I would say to myself. Then if it didn’t happen, I would feel desperate for nothing. It is funny, but I wouldn’t make such a big deal about my second customer. Why? There is something weirdly superstitious about the 1st time of anything.

This very act of giving it space in my mind makes it somehow more and more challenging to achieve. Now I have changed my attitude towards the 1st anything.

My first customer happened soon after, and so did the next hundred.

I want to notice milestones in my rearview mirror, never as I reach them, and not anticipate and look for them. I wish to acknowledge signs but never focus on them. I don’t even want to celebrate them too because that tells my brain that the next one is even more difficult and yet significant a deal, and my mind then makes it real. Now when I achieve them, I stand my own as if to say, of course, I realized them.

If you can’t stop making a big deal about the 1st of something, counter it with making a big deal about the 2nd one to mess with your head.

Let’s celebrate the 88th customer rather than the 100th one. The brain will then realize the ridiculousness of its belief and will let go.

The author is the founder of Proxies API, a proxy rotation API service.

It Is Ok To Build Your Startup Where There Is Competition

Mohan Ganesan — Thu, 08 Dec 2022 05:48:46 +0000

It sounds counterintuitive and unnecessarily aggressive, but it is ok. It sort of is frowned upon but not illegal.

Look at these guys:

That guy on the left came back one afternoon after a fight with his missus, and this guy had popped up. Then he had another fight with his missus that night. Plus, it probably will turn out that that new guy was his spiteful brother, making you feel worse for this unknown (Chinese?) guy. Poor guy.

So what happens now are they both gonna have to get good at their business and can’t just be fighting their misuses no more.

And they all get a piece of the pie, which they can hide somewhere and eat.

I built a Proxies API like that. Right next to my competition. Right after a fight.

And I don’t need much from it. Some crumbs here and there, and I will do just fine.

Plus, I get to learn from the bunch of the guys before me. That’s where their 80/20 rule comes in. It should be renamed the ‘here’s how the pioneers wasted most of their effort finding the way (the 80%) and here is what finally worked (the 20%), so just copy this’ rule.

Using the 80/20 rule and some real personal smarts, I was able to work out.

The kind of places they got traffic from, it is hilarious to see the stunts they tried before they realized the now obvious. One guy even started a Facebook group! What DUDE! Developers HATE talking work on Facebook. I could have told you that then.
I know what kind of content works, so I just write those but way better. Use their complete exhaustion to my advantage 😈
The type of ads they have put up. You can see the evolution of their ads using SpyFu.
The kind of website copy they used. You can use the Way back Machines to see their journey from humble beginnings to sophisticated war mongering beasts that they have become, and you know, learn from that.
Learn the amount of work they put in. I just need to put in a couple of hours more than them, to be honest, and I will be well & truly ahead. But seriously, how the hell do you know what it takes to do anything? It turns out it is not much in my case, but how DO you know that?
The kind of money they make. Let us face it, and everyone loves to be on podcasts where they like to vomit out every personal struggle and detail about themselves, their pets, and their businesses. Good for them.

I am glad I came in late. In fact, I would have been well advised to probably wait another year.
They don’t write half as funny as me. Which means. I have a better Rotating Proxies Service, AND I can make you laugh. That’s something you always look for in a Rotating Proxies API Service.
And lastly, never, ever get on a podcast. No, thank you.

The author is the founder of Proxies API, a proxy rotation API service.

No Thanks, I Don't Want to Build A Billion-Dollar Company

Mohan Ganesan — Thu, 01 Dec 2022 06:37:54 +0000

When I started to build out my SAAS app Proxies API, I didn’t want a billion-dollar company or even a million-dollar company. Those are weird goals and don’t have any power behind them. The why of doing a startup should be so powerful it drives you every morning and every hour on its own. For that, it has to be natural. Lying in reality. A lot of gurus advocate having a strong why to their students. To find you are why they say. Find it SO that you can build a company. The irony and dishonesty of this approach are lost on them. It’s like saying, go find a disease you have so I can cure it. It’s never worked that way. Because you can’t trick yourself. Then why should it be obvious? If it is not, then you can’t wear a psychological version of VR goggles to psyche yourself that there is a why. Here is a strange thought — if you can’t find a way to start a company, then don’t. If you don’t have the motivation to do a startup, then why do you want to do it.

I think of it this way- If I know there is a tsunami coming my way in 6 months, and I am on an island, and I have no way to get out, it’s natural for me to plan and do everything in my power to deal with the upcoming threat. I don’t need to find me why. It is natural, and the goal of building a strong tower tall enough to survive the tsunami is not arbitrary. The fear fuels the work that’s needed and the research that’s required.

For me, I had to quit my day job because of the way my personal life was structured. Terrible things would happen, or at least had a high chance of happening if I didn’t become location independent. I can’t share more than this in a public forum, but all you need to know is why I was looking large in front of me, and I had anywhere between a year to 18 months to get independent.

After this threat, my calculations are modest. All I needed was to be able to pay rent and sustain living expenses. Period No other jazz was necessary. Figures like a million dollars or think big was laughable guru speak who didn’t have the sense of reality or urgency about my situation. You see, beyond the exact number needed to sustain me a month on month and free from having to work a job, the extra money meant nothing to me.

A focus like this is based on fear, a fear that is natural and real. It’s impersonal, and it has great power. It doesn’t stress you out like a personal fear like the fear of humiliation, which is your very own. This is my power. And this made sure I left no stone unturned to get to my goal. Everyday scenarios would play out in my head as to how else I could fail. Failing was not an option, so it fuelled me. I wanted it.

A potential billion-dollar company someday when my personal life is in tatters is of no use to me. Just earning 2.5 k per month meant everything to me. It’s my dream come true, and possibly a nightmare averted.

The author is the founder of Proxies API, a proxy rotation API service.

The Stack Behind A Production Level Rotating Proxy Service

Mohan Ganesan — Wed, 23 Nov 2022 07:03:53 +0000

Background

Proxies API is a rotating proxies API that developers use to fetch hard to scrape data at scale and consistently. We auto-rotate millions of proxy servers, and also handle auto retries, rotate user agent strings, handle cookies, CAPTCHAs behind the scenes.

The Constraints

The service has to scale to millions of URL fetches a day without lagging on speed and should work out of a single API like this.

curl “http://api.proxiesapi.com/?key=API_KEY&url=https://example.com"

The Architecture

The architecture looks somewhat like this.

A more in-depth look at this architecture is provided in this article.

Load balancing & Serving

The whole infrastructure is on Amazon EC2. The load balancer plays a significant role in calling the right service based on just the variables passed and the API KEY that holds some secret information like what pool of servers are assigned for them etc.

Our plans have higher concurrency as customers pay more, so we need to pass them to more reliable, dedicated servers as they have higher concurrency needs.

Luckily amazon elastic load balancer can make this using rules you can set, which look at the URL patterns and direct them to different pools of servers intelligently.

Without this exact combination of facilities, we would not have been able to create it all using a single API call without compromising speed and performance somewhere.

We use Apache if there is a need for HTTP servers. Many of these are super optimized to be able to crawl data concurrently using multiple cores. We will soon write an article on how we can get the most amount of crawling ability out of Apache instances.

Proxy Components

For CGI, we use a combination of Python and even PHP where required.

Memcache is used to monitor and throttle concurrencies gracefully, so some rouge code by one of our clients doesn’t bring down the entire setup.

As I described earlier, most of this throttling happens in the way we route out requests from the load balancer.

We use the Node Js/Puppeteer library extensively as an interface to connect to Browserless docker instances located on multiple servers. Browserless is optimized to run as many ready-to-go instances of Chromium as possible, so there is no lag in loading any of them for our clients.

Database

We use good old MySQL in a Master/Slave configuration to hold user info, cache info, millions of proxy info, quality metrics for each, etc.

Maintenance tools

We have a pool pruner that’s written in Python running as a Cronjob. Uses Scrapy and Scrapyd extensively in making calls, monitoring, benchmarking, and pruning proxies from our database.

We use the Beanstalk library for Queues. We use Datadog for monitoring and PagerDuty for alerts.

The author is the founder of Proxies API, a proxy rotation API service.

Personal vs. Natural Goals

Mohan Ganesan — Wed, 16 Nov 2022 07:31:19 +0000

I have wanted to write about this for a long time. I have seen that there are two types of Goals if I can even give it that honor. In actual terms, one is a Goal, and the other is a delusion. One is natural and the other is an un-natural personal one.

When we make a goal, the reason or the intent behind it is the power behind that goal. Have the wrong intention, and the target goes nowhere, and the real purpose will perpetually power that goal making you almost unstoppable.

That’s what happened to me while building my startup Proxies API.

Let me define this as well as I can.

A Personal goal has a very psychological origin to it. There is probably nothing in real life to support it. It’s almost unnecessary. It might be motivated by fear but an absolute psychological horror like shame or resentment or jealousy. These are optional emotions powered by people simply identifying with them. When powered by them, they have a Gollum like quality to them. There is something inherently ugly, pathetic, and selfish about them.

A Natural goal might also be fear-based in its origin. For example, you have received an advisory from the government that there is a massive hurricane headed your way, and you need to take precautions when it hits 48 hours later. It should create fear, which is natural and necessary. There is nothing wrong with feeling the fear and taking measures to counter the situation. Your “Goal” then becomes the survival of you and your loved ones, and you will have to do everything in your power to achieve it. The fear will power it. You don’t need Tony Robbins pumping you up, and you don’t need affirmations. You are powered by instinct.

Building Proxies API was like that for me. I HAD to escape the hurricane coming into my life. In this case, I had around a year, and I would be homeless with my loved ones (I won’t go into details here). It’s a real fear. I embraced it and set my goals.

Doing what was necessary came easy—no need for motivation. If I NEED motivation, then what’s the motivation behind lacking that motivation? I believe that if you need to generate it or fake it, then that’s not it. You haven’t got it; you are missing the point. It is best not to do it.

The single-minded dedication came easy too. If you have a hurricane hitting, trust me, you don’t need a ‘social media blocker’ to block off distractions. There is no process to it. You do it.

Once I achieve the goal, it can hardly be called achievement. There is probably a sense of relief. But there is a matter of fact about it. It merely needed to be done, and it is done.

The author is the founder of Proxies API the rotating proxies service.

Seven Ways To Trick Your Brain Into Working

Mohan Ganesan — Fri, 11 Nov 2022 06:29:14 +0000

While building a bootstrapped startup, I realized early on that the whole heuristic of measuring a startup comes down to the habits that I have constructed that can repeat on a day to day basis consistently for months and years.

I have already written that in the case of Proxies API, my SAAS startup, the two main habits that have made it successful were.

a) Make small, consistent improvements to the product every single day.

b) Write (blog, make videos) every single day.

Now, this is a big ask. I have been able to maintain a streak for months now. So in this marathon, you don’t always feel it. Here are some hacks I have used to get my brain back into perspective that lead to the success of habit-building, which in turn leads to the success of the startup.

Have an impersonal, clear goal. I/we need to get x to y in x days/months under these constraints/conditions. The only rule here is that the goal should not be personal. Personal goals are ones that have a feeling of vendetta like a Gollum would create. They are created in mind, and you don’t get a sense that they are natural.

Personal vs. Natural Goals

Have a work backward excel sheet: Life is long. We eat three times, need to go to the toilet ten times, clean up, meet friends and family, power off for 6 hours, and then there are social media, there is Trump, and North Korea is attacking us. In all this, it is difficult to remember what it takes to get to that long term goal. What do we have to do this again? What should be the minimum calls I need to do to get my target number of clients? How many words should I write to keep on track for my goal of having a decent content base to attract SEO and other inbound traffic again? We can’t recalculate it every day. Best to have it in a ‘working backward’ sheet that can be tweaked based on what we learn from reality.
The brain needs reminding: Again, in a marathon with a hundred distractions, it is easy to forget the daily targets, the exact strategies we had researched and decided on early in the planning phase. I find that reading my goals and my strategies every day reminds my brain before it becomes vague.
Remind yourself first thing in the morning. Ideally, before checking the phone, definitely before any social media or emails. Your goals don’t get overshadowed by concern for how the world is about to end. Someone tweeted something because your neighbor posted a picture from their vacation.
Use the 2-minute rule to get a ton of small tasks done very quickly and promptly.
Do a brain dump. My brain will get overwhelmed because of all the choices of actions, tactics, growth hacks, and strategies out there. I use the brain dump every two weeks at least to overcome this and get moving again.
Make it inevitable. I approach everything with a sense of wanting to make the result inevitable, so instead of doing the bare minimum, I tend to do the overwhelming maximum, and also I make it almost a part of destiny by setting up my environment. More about this technique that I learned from Eben Pagan.

Remember To Forget About Your Long Term Goals Every Day

Mohan Ganesan — Wed, 02 Nov 2022 09:40:44 +0000

One of the problems of having a long term goal and strategy is that it becomes overwhelming when you think and plan too much.

For example, for my startup Proxies API one of the long-term goals is to write 100 articles in the year.

All this overwhelm means that when I wake up, I find it difficult to “get down to” the daily tasks that lead me to my goal in the first place. It is like my brain. Once it starts to think about long term goals and strategies, it can’t switch to doing simple tasks and tactics.

So every day, I wake up and revise my goals and strategies and get them out of the way. Once my brain is reassured that I am on the right track, I try to forget them altogether. Just try shaking it off with the promise of returning to it in the evening. Till then, no re-strategizing, no making sure we are on the right track. I am not even going to remember them.

It might be pronounced by some of you and is probably weird that I need to do it to myself, but hey, that’s how my brain is designed, and it’s a big deal for me now that I have discovered this works for me.

The author is the founder of Proxies API, a proxy rotation api service.