I worked on ~16 different teams within Search. I built high-volume, low-latency, highly-available distributed systems that served ~12,000 requests/...
For further actions, you may consider blocking this person and/or reporting abuse
Where do you think Google, Bing, and the rest of the search landscape might look like in ten years?
Google probably isn't losing its position tomorrow, but what about a decade from now? Will web search still be the leader, will it be more audio assistants? Will Bing/Microsoft make ground? Will someone else emerge?
Search is a very dynamic landscape and there are some of the smartest people working on it. In the tech industry, a lot of the engineers are poached between the same set of large corporations (Microsoft -> Amazon -> Google -> Facebook -> Twitter -> eBay -> Netflix, etc.) and they tend to bring a lot of tribal knowledge with them even if they don't directly disclose things under NDA. This means that no specific feature is out of reach for any given competitor. That said, they are always trying to innovate, but it can be short-lived before a competitor pulls one on them.
Most of these search engines are constrained with resources and time. Google has quite a headstart compared to Microsoft and other search engines. Yahoo has discontinued it's Search Engine and has been using Bing under the hood (at least they were a few years ago when I left). People are attracted to DuckDuckGo because of their enhanced privacy, but DDG still has a long way to go. They have managed to barely capture .25% of the market share. There are at least a hundred other search engines which are struggling to gain traction.
Now, it is hard to ask someone to switch from Google to another search engine like Bing or DDG. If Google is working for you, you are unlikely to try Bing unless Google was down or a family/friend encourages you to try it out. Even then, you have high expectations and will most likely go back after trying a couple of things depending on your experience.
Traditionally, this approach hasn't worked well for Bing to acquire users organically. Instead, they have tried to piggyback on other products like the default search bar in IE / Edge, Windows, Cortana, Surface, XBox, Windows Phone, etc. to drive traffic to the site. Most people don't know how to change the default search engine and stick to whatever loads and sometimes they don't have an option. Often you will subscribe to an ecosystem and not worry about switching.
Search is always going to need on-going improvements. What worked 20 years ago, did not work 10 years ago and doesn't work as efficiently today. Think about the amount of data generated every day. It is around several hundred million documents. In addition, the types of documents vary and how search engines can make sense out of them. Throughout the process, they have mostly focused on showing 10 blue links out of billions of documents. That is tricky because how do you accurately pick 10 links out of billions? In the long run, this will be considered the antiquated search experience and those who innovate via AR/VR will be regarded as trailblazers.
The latest trend is ML combined with audio interfaces. These have come a very long way and lend themselves to searching/browsing content. My opinion doesn't shed new light here as we all know that things are headed towards voice interfaces. This has been true since the day of IVR systems. Nowadays, kids talk to their Phones, TV's, XBox's, Echo's, Computers, etc. That is not going to change. While there will be more audio assistants, web search is always going to be there.
In the next ten years, these companies will continue to invest in AI and Audio Interfaces. However, a search index will always be under the hood. They will need to improve on context-sensitive searches to be able to effectively integrate with Audio interfaces. Most search engines will think about modular design like DDG and allow developer integrations into search. They will integrate search with deeper into cars, refrigerators and other mainstream appliances.
One of the emerging trends in the past few years has been vertical search. A lot of sites other than search engines are catering to specific segments and that tends to do a lot better. Google will maintain its position in generic search, but other players may start to take away their market-share for specific verticals as they might not provide the same user experiences.
I've been moonlighting on my own search engine for a while now and I'm hoping I can put it out there for everyone to use. Perhaps, I need to team up with other developers or make it opensource to speed things up. :)
What advice would you have for those of us wanting to begin consulting and start a side-business, go freelance, etc. ? Given your breadth of experience I'm sure you have some great advice ;)
If you're referring to starting a consulting business on the side, there are a few things to consider.
You need to build expertise in a specific area. It could be in a tech-stack or a specific niche like building websites for dentists or mobile apps for schools, or being a Shopify expert, etc.
There is a lot of work out there and you need to work on strategizing how to find it. Before you start side-gigs, take some time to focus on building your technical and soft-skills. Running a consulting business is quite different than a full-time job. You will also need to focus on building a portfolio and drive traffic to it organically.
You will make mistakes in your first few gigs. That's okay. You want to make sure that you learn from those mistakes and that you do the right thing for your client and your business. However, keep in mind that you also need to strike a balance there. My first client didn't pay me and they still owe me $3,000 (and now they're a pretty big well-funded startup)! Then, I was quite lenient with my second client and ended up doing a lot of extra work due to the changing requirements and their constant request for a low-budget implementation which meant I had to cut corners. Initially, I didn't think it was a big deal as they were still paying me 28k for the project in three phases. However, I ended up doing almost three times the amount of work than what I anticipated and they kept taking advantage of that.
That brings me to billing hourly vs. project based. There's no clear formula here, but you need to evaluate on a project basis. Usually, I charge hourly for up to a few hours a day. Then, I have a daily rate that's a bit lower and I scale that to a weekly and monthly rate without additional discounts. In many cases I make exceptions. For instance, when I'm teaching someone I will reduce my rate so I can help them.
There's a lot to manage when you are running a consulting business. You have to constantly look for work. You have to keep up-to-date with technology. You have to do the work and provide value to your clients. You need to follow up with them as needed. And most importantly, you need to bill them and often chase them down when they don't pay. The best way to deal with this is to sign some agreements ahead of time and get 20-25% upfront payment and use a CRM-like software to manage this.
Remember that your clients are paying you to deliver immediate value. Depending on your area of expertise, you could bill accordingly. I started billing clients at $60/hr years ago. Often clients are not respectful of your time when you don't bill them enough. It is sort of a reverse psychology. The most problematic clients have been those who have requested $60/hr or lower rates. Over the years, I have continued to increase my rate and I try to stick to a specific range. However, I've also charged upwards of $500/hr in exceptional circumstances.
Overall, you need to focus on building a process for yourself. You need to rely on tools/services to run your business. Services like a scheduling platform, invoicing software, time-tracking, email, etc. These things save a lot of time. I spend ~$250 a month on tools.
At the end of the day, you have to start somewhere. You can start out by building your resume/portfolio by doing ad-hoc work. Think of everything as a learning experience and a stepping stone towards something bigger. Think about the long-term and always keep looking for something better, otherwise, you get complacent. There's nothing wrong with being satisfied with a specific point in life, but it can take a while to get there.
The best thing about this is that you are working for yourself and you are responsible for all your decisions. Another thing to keep in mind is that with independent consulting, you may NOT have consistent work so you need to account for all that when you bill clients. You also have to purchase your own insurance if you're in the US and all of that adds up quickly. Don't forget about taxes and keeping money aside for paying that.
In the short-term, you can start building your personal brand (or business if you prefer that). Regardless, you should form an LLC. Then, start writing articles, teach people, create a youtube channel, go to local meetups, give local tech-talks, etc. Do whatever you can to get noticed. Get involved in open-source or building projects for local communities/groups. This will lead to more work. It takes time and you need to plan for the long haul. It is a competitive space, but if you're really good at what you do, you should be able to transition into doing this full-time.
Thanks so much for the in-depth response! I'll have to read over that a few times and take notes ;) I appreciate it!
What was your initial path to reach to this point in your career? Like what you did in college (if any) and right after that? And what would be the best advice for someone who is just starting professionally?
I never studied in college. I didn't buy books most of the time and I was happy with a C or D if that's what my class required. However, I was really passionate about computers and spent every minute of my life on one. I fixed computers for family and friends because I loved doing that. I was constantly broke so I looked for work in tech since I was 15. One of the first things I did was help students in nearby colleges with their exams and projects. Sometimes I got multiple students from the same class and I had to implement the same project in 3-4 unique ways so that neither of them got caught. This gave me a different outlook on programming. I was forced to think about solving the same problem in 3-4 different ways. At the same time, I took up a job in the community college I attended to teach computers to the professors.
The next best thing I did were a couple of internships. I went overboard and I worked two jobs, a co-op outside the college as a Web Application Developer and an internship at a lab in college as a Software Engineer (which didn't last too long because of time constraints).
This gave me an edge compared to my peers.
I graduated with the lowest GPA, so I had a hard time getting interviews at large companies mainly because they were so fixated on that metric. It is a very skewed metric and does not reflect a person's abilities to be successful at programming. All my friends with 3.8+ GPA had interviewed at large companies including Microsoft and none of them got in (One of them did about 4 years later)! Eventually, five years later, I got an interview at Microsoft. It was out of the blue when I wasn't even looking. The rest is history!
I also spent a lot of time before and during college working on ad-hoc stuff. I installed Windows 3.1/95/98 over a couple of hundred times for everyone I knew. I dabble with Linux. I ran a BBS. I taught myself game-programming in C++. Later, a little bit of Java when it was released. I helped people with their projects in Basic, Pascal, C, C++, Visual Basic, Java, Classic ASP, Foxpro, DBase, SQL, and Access. I started teaching myself HTML and Javascript and took up a short-consulting gig building a website for a close friend's cousin.
Although it appeared like I was all over the place, everything I did, enhanced my resume and helped hone my skills around problem-solving. This is the basis for being successful in the long-run. You have to keep learning all the time and get to a point where you can do just-in-time learning. Once you know a programming language like C++ very well, you can pretty much learn anything. Other than that you want to focus on Data-Structures, Algorithms, and Design Patterns.
The best advice I can give is to worry about yourself and forget about what everyone else is doing. Forget about the frameworks of the day and libraries of the year. Come up with a plan and stick to it. Learn programming like it's the 90's and focus on the basics and practice on real-world projects.
I ended up doing a whole post about the advice you gave, it is very enlighten and made me remember many things that happened lately and some questions I had. Check it out if you have the time. :D
One thing that I didn't say there and I plan to talk about later is about the multitude of frameworks and the difficulty of choosing a path. This advice was big, I really need to focus and I'm trying to.
Thanks, Leonardo, for writing that post. I enjoyed reading it.
What's the most difficult technical issue/solved problem you have ever faced during your working time?
Throughout my career, I've faced challenging problems in many aspects of my work. I tend to seek out that kind of work primarily because a good challenge pushes me to think, learn and improve myself.
About 15 years ago, I designed access control security systems. We needed to test hardware with physical cards and sensors and do a stress test with over 100,000 transactions/sec. Now, we could have simulated it, but that wasn't enough. We needed to test the real hardware. So, we came up with a plan to implement it in our office. We mounted several hardware controllers on a wall and connected multiple card readers, biometric readers/scanners, and sensors to it. Next, we had to swipe cards on those card readers which in turn would trigger a bunch of sensors which would cause a door-lock to unlock for instance. Our hardware was all configured and ready to test.
We went to the junk-yard and got an old windshield wiper motor. Then, we headed to Home Depot to purchase a few pegboards, a regulator for the motor, and tracks.
Next, we attached a variety of card readers to the hardware and mounted them to the wall below the rest of the hardware controllers. Then, we placed the tracks on the walls and attached the pegboard to it with hundreds of cards stuck to the board facing the wall (towards the card readers). We connected the windshield wiper motor & regulator to the pegboard in a way that when the wiper was turned on the pegboard would slide back and forth sort of like how the wiper goes back and forth.
What's interesting is how windshield wiper motors work. The motor rotates, but the way the arm is attached causes it to go back and forth. We used this motion to slide the pegboard back and forth on the tracks against the wall. This resulted in cards being swiped on the card readers and various sensors getting triggered. We managed to get about a 150,000 real transactions/sec and I was able to test my enterprise-grade server with the actual pipeline.
I definitely have more stories, but I'll leave those for another time. :)
WoW! Very impressive, thanks!
How much did it take to go on production for each product ?
And what Agile framework do the fellows there use ?
For such distributed systems, how does team management work? small teams or how ?
What maintenance model does your team follow ? Sprints & new releases ? or when the managers put the requirements for the dev team(s) ?
The Search pipeline consisted of at least a thousand services. In order to deploy something of that scale, you have to employ robust tools & solid processes. Especially, when the team consists of several thousand engineers. Normally, each service is managed by a team. It resides in its own source control repository. Sometimes services depend on base-services, so you would have to do the forward/reverse integrations before deployment.
Prior to deployment, you would need a deployment plan (generally created at the beginning of the feature sprint). The service would be developed locally but deployed to a dev-environment (manually back in the days). This is where the dev team would get to test it. There would be a test cycle and bug-fixing phase that you would go through, followed by green-lighting meetings with the service team, partners & stakeholders. In this meeting, they discuss potential risks and dependencies and delays on other teams, fallback, etc.
When everything is good, all the dependent services get pushed their respective test environments and they undergo some sort of test-automation run which automatically creates bugs. This will lead to a bug-triaging process, followed by bug-fixes and deployment. Everything has to pass here before it can be upgraded to the integration environment. This is where they will test all the dependencies as a whole, etc.
Once everything passes, the services are moved to staging (aka pre-production), where the service/feature is tested by the entire team, generally through a scheduled bug-bash.
Once deployment is done, you have to ensure that logs, monitors, alerts are all functional up. We had to also eyeball the service/UI to ensure everything looks good.
Also, sometimes new features are flighted (exposed to ~3% of the userbase). This allows the team to ensure that there are no catastrophic failures. If something were to go wrong, it doesn't affect the entire userbase.
After deployment, the PM/Testers monitor that service for a few days. They will look at usage analytics, precision-recall measurements in some cases, and other statistics. They might decide that the feature is not used by end-users and make a decision to pull it back.
This process was mainstream before I left Bing in 2012. Now, things are different from what I heard.
Team sizes vary. Usually, a lead may have up to 5-6 developers under them. A development manager might have 2-5 leads and a couple of Senior IC's. Principal Development Managers may have up to ~32 people on their team. People tend to wear the # of subordinates as a badge of honor!
There are a few important roles in the organization. You have Software Development Engineers (SDE) and Software Development Engineers in Test (SDET or SDE/T) (90% of tester roles were eliminated from Bing in 2012 I think as it was a joke!). Then there are Software Test Engineers (STE) which are basically manual testers for certain roles. You might get someone watching images all day to identify porn for instance or redlines matching or simply executing scenarios manually before things are automated.
Then, there are Program Managers who manage a feature on a team and there are Release Program Managers who are responsible for the coordination of certain pipelines (a bunch of feature teams pretty much). They will communicate delays or changes, etc. across the larger feature teams and ensure everyone is on the same page.
There's also a Product Manager who is the liaison between the Business side (Biz dev manager) and Engineering side (Program Manager). It's all a well-oiled machine and a marvel to observe. Everyone has their own little thing to do. However, these processes and redtape kill agility and innovation at times.
Every team is different. Most teams in Bing followed a Scrum / Sprint style. Some teams used a KanBan board of post-its, whereas others used Microsoft Project.
There's a lot more to it with source code branching that coincides with the teams, stages, features, versions, etc. There's a live branch for quick bug fixes. There's a hotfix branch. There's an out-of-band deployment for breaking changes, etc.
In the past, most teams released every 3-6 months and that was considered fast. I was on a team that changed things and started releasing every 2-4 weeks and that stirred the pot. I've heard that they might be doing daily releases now. Not sure entirely. I also forgot to mention things like, our build took about 18 hours initially. When I joined, it was down to ~10 hours. Over time, things got better because we had teams dedicated to optimizing the build process. I think we got it down to as low as 30 minutes for some teams. Also, deployments are isolated as much as possible. Sometimes you deploy one service, whereas other times you may have to deploy 5 services or wait for other teams to deploy. That schedule is also managed by release managers.
Overall, it is a nightmare when you think about deploying across data-centers, maintaining version compatibility, ability to revert, monitoring all that stuff, etc. However, tooling has been the key. That along with the effort of some brilliant people, they have constantly improved the process.
Woohoo... a heck of comment.
Thank you for the well-detailed comment, I really appreciate it :)
I can see now why such products barely have bugs with the end users !
What are Microsoft's reasons for developing edge and Bing? Why do they want to compete in those markets?
I'm not sure about Edge entirely, but Microsoft often tries to rebrand its products when those products have a bad reputation in the industry. While I'm not sure if that was the reason, I can also speculate that the code-base for IE was antiquated and convoluted to work with and it made sense to start from scratch.
Initially, Microsoft got into the browser market as there were no web-standards and it was easier for them to push their own products into their own browser via ActiveX controls. In addition, their approach is to be #2 so they can easily replicate what competitors have done and try to make it slightly better in some cases.
As for Bing, Search has been a huge industry worth several hundred billion dollars. There were so many upsides to offering your own search engine. For instance, you could control the content that is displayed. Microsoft could promote its own sites like MSDN and MSN ahead of other online resources. They could accumulate so much data for training machine learning algorithms. If I recall, a 1% market share was worth something like 400 million dollars about ten years ago. It makes for a very good foundation to build other technologies on top.
Besides, the company didn't believe in the future of the internet in the 90's but later started to rush into it once they saw Google and Yahoo's successes.
How is Bing better than Duck duck go? Or Google if you believe it is?
All search engines are very similar at the end of the day. Bing has been trying to get better than Google. It focused more on Instant Answers early on. Duck Duck Go was inspired by that and created a nice ecosystem around building modular Instant Answers through third-party integrations.
The thing that sets one apart from another is really context. You could search for the term 'Cancer' and get a star-sign on Google, but the disease on Bing. Which one is accurate? Does it make that search engine better? Most likely not. What matters is relevance when you search. This is a hard problem to solve, even after it has been solved. So it is a moving target. That is because the amount of data and number of users searching are growing every day.
All the engines have been trying hard to get better at improving Search (Results) Relevance. There is a lot of work that goes on behind the scenes. You have to understand the context of the query. You may need to auto-correct the query. You need to understand the domain the query is referring to. The type of results that need to be returned for that query. How do you rank those results? Previous searches and actions. Your preferences. Etc. How do you pick 10 blue links in a split-second out of billions of links? These are some of the challenges that all of them face.
Where each engine stands varies significantly with each algorithmic update. Google has been better at relevance for a long time. Bing was better in certain domains like Shopping and Image Search. Duck Duck Go has done a good job of innovating and positioned themselves around security. Whether their results are accurate or not, I'm not sure as I have tried it only a couple of times. They seemed similar overall. In order to compare search engines, you have to look at a lot of types of queries (including tail-end queries), domains and context.
At the moment, Google is still better at relevance. Its image search has gotten better.
As much as I loved to use Bing, I switched back to Google a few years after I left MSFT because I need to use it as a tool to get work done.
This says it all "As much as I loved to use Bing, I switched back to Google a few years after I left MSFT..."
What is the best way to make scalable services without over engineering too much? ( I know this question is open and big, but I just want to hear some of your thoughts on the matter.. take it in any direction you like.. thanks)
Think about what you are building and start illustrating it as arrows and boxes on paper. Then, walk through a few simple use-cases, amplify them and try to identify points of failures and bottlenecks.
For instance, if you built a web service which exposes an endpoint to upload an image and then processes that image to generate thumbnails for hundreds of social media formats, what could go wrong?
It is possible that the server blocks all incoming requests while it is processing an image that was previously uploaded. What can you do about that? Do you add a second service? Or, do you extract the functionality to upload images and generate thumbnails into a separate service? Furthermore, could a third-party service be utilized instead of implementing this feature yourself. What are the associated risks and limitations with all the above approaches? Are there any security or resource constraints? Are there budget limitations? A lot of factors are involved while deciding when to scale services.
There are so many questions that you need to ask yourself. So, what do you do? You read/learn what is involved in making highly-scalable services and you apply the best possible design patterns to architect your single service. Make it modular so that you can always move things around with the least amount of disruption. Rely heavily on test automation.
What happens if 100 people use this service vs. 1,000 vs. 1,000,000? When you have an ideal solution, these growing numbers won't bother you. Would a reverse-proxy load balancer be the answer here? Perhaps. Often you can take multiple approaches to scale.
As a general rule, write the most efficient code you can in the shortest amount of time. Don't spend too much time optimizing it until you need to. Focus on high-level architecture as splitting things is always difficult to do later on. However, you want to start with a monolith for a v1 product and then think about distributed-systems or microservices. You will need to run tests to measure service performance. Your decisions should be data-driven.
What would you consider the most important things to mention and agree on if your going to partner with another developer or marketing person whom will bring in jobs for you, then you develop the products as to the clients specifications and how best should you share the payment, lastly should you share the payment right away or after a months work?
Sorry I'm learning as I go, and from my experience with a marketer, I realized I worked more burned myself out during the process.
The most important thing is to draft an agreement and sign it. Your payment will depend on the project. For gigs lasting less than a day, I typically bill the client the same day and expect the payment by the next day. For gigs that are a few days, I apply the same strategy. However, as the gigs get longer and longer, the risk is higher. I ask for 20-25% upfront payment. This might be possible most likely after you have established a reputation.
You should form a company and have everything drafted formally if you are serious about this. As for partnering with someone else, you need to know them well and they need to bring something to the table as well or it is a recipe for disaster. One more thing, have your project requirements clarified before accepting the project.
Thanks Nick, so would you mind if I try out your search engine for a little bit, test it out and give you some feedback later or your still working on it?
Hi Leslie! Thanks for offering to help. I had exposed a work-in-progress version of the search engine a few months ago to get feedback, but I got busy and couldn't keep up with updates as I originally intended to. Since it was costing quite a bit, I took it down until I could work on those updates. It's on my list of things to do and I'm hoping to revisit some of that in the next month. Ideally, I would like to just release it and keep updating it incrementally. However, I need to rethink some of the decisions I had made (a bit hastily to just push it out). I'll try to post something here once I push it out. Thanks!
What advice can you give those of us who have been building UIs and CRUD backends for years but want to break out into different domains? For example, how do we follow in your footsteps and build "high-volume, low-latency, highly-available distributed systems" or "Big-Data Validation Framework?"
When I started out, I didn't plan on building these kinds of systems. I just wanted to learn everything I could and land a job. For several years, I had learned Data Structures & Algorithms but didn't get to apply them. However, I kept practicing implementing that stuff frequently. Once you get good at those, you can build standalone applications that are robust and efficient. You learn little things that go a long way. You become a performance junkie and try to get things done in fewer and fewer ticks and memory as possible. This is really your foundation for everything else. Later on, I was in a situation where I had to build a system that supported 100k operations per minute without crashing.
When I joined MSFT, I got to experience first-hand what engineering excellence really meant. However, the high-volume, low-latency, highly-available systems that I built were not my idea as such. It was a result of specific requirements at work. Our services had to support a certain number of queries per second. Over the course of time, I learned via observation and from code-reviews and picked up techniques for building better systems. I read a lot of architecture documents to understand how various search pipelines worked.
As for the Big-Data Validation Framework, I had cycled through a lot of teams in Bing. In the process, I was really tired of the redundant work I had to do across many of those teams. I noticed that most of my peers were in the same boat. I ended up building a tool to solve my problem and built it in a way that everyone else could contribute to it and all of us would benefit. This was partly a result of my laziness and partly boredom with the mundane tasks.
Most people don't decide to build stuff like this out of the blue. Perhaps you can think about joining one of the big companies where you can work on large-scale problems. It is something you should experience once in your engineering life as you get a chance to work with and learn some incredible stuff.
What kind advice would you give to a young developer like myself who is entering to a career of software engineering?
You might find one of the comments above useful.
If you are starting out, you need to take every opportunity to learn. Don't pass up on that. Do free work for people if you need to. If they pay you, even better. Make friends and go to parties. Focus on building your soft-skills. Run for student senate in college. Join some clubs. Try to lead one. This will teach you how to interact with different people in various situations and you will learn valuable leadership skills.
If you are still in college before you graduate you should do an internship or two. Do everything you can to set yourself apart from others. Every bit you do will add up in the long-run.
If your interest is in programming, you need to be one with the computer. If you want to excel in this field and be one of the best, you need to live and breathe computers. Before you know it, you will have learned a lot of ad-hoc things.
Find a hobby that you absolutely love and combine that with programming. If you love playing games, try to learn to build a game. If you love physics, build a physics engine for a game. If you love designing, work on redesigning your favorite website. The best way to learn is when you pick these skills while working on a real-world project. Try to teach someone about a specific technology or language.
I'm going to reiterate what I said in the comment earlier. You need to focus on yourself and don't worry about the rest of the world. You need to focus on the basics and don't be overwhelmed with the plethora of frameworks and libraries. They will come and go. Your core skills will stick with you. A lot of this might not sound exciting now, but it will play a role in shaping you years from now.
Finally, pick a stack and learn it well and apply it to real-world projects. If you don't have an idea or a project to work on, look towards OpenSource repos on GitHub.
Thanks for not one but the many advices you have given. This just made me look into my career from another perspective. I hope we can connect on LinkedIn incase I need some guidance on things.
When you want to look for online resources do you bing it?
We did use Bing exclusively when working on Bing (because you have to dogfood your own stuff). I liked it quite a bit, but a couple of years after I left MSFT I was not so keen on dealing with delays in my workflow because of irrelevant results and I wasn't in touch with my team to report issues. So, I made a conscious decision to switch to Google. Occasionally, I might use Bing for Image Search. However, Google's image search is very good these days.
What was the most influential book/presentation or alike that changed your way of thinking about software/development?
I think this has been an ongoing thing. You never really stop learning. There are various books/talks that influence you over time. Usually, I don't get swayed easily by them because I take the time to research things on my own and my knowledge applies collaborative filtering to grok things that I find useful.
One book that I loved about 15 years ago was Head First Design Patterns.
I used to love watching talks by Martin Fowler.
If you love gaming, I would highly recommend reading Masters of Doom: How Two Guys Created an Empire and Transformed Pop Culture!
How often are Hackatons organized at Microsoft? Is a cross-company thing?
Is only me or most of the development teams are US based, at Microsoft?
Microsoft is such a large organization (~150k employees + contractors during its peak time). Each team is run independently, so some of these things are team-specific.
Bing hosted HackDays on a Thursday/Friday once a year (if I recall correctly). I've heard that after Satya took over, MSFT hosts a one-week hackathon once a year (which is a very new thing for MSFT).
Microsoft is a global company. Bing was a globally distributed organization. I worked with people in Brazil, Canada, India, UK, and Norway quite frequently.
Do you think Microsoft will make future Windows versions POSIX compliant a la OSX?
I'm not really sure about this. Today, Microsoft is a lot different and open to integrations and open-source work. They're definitely doing whatever they can to stay relevant. My understanding is that POSIX was for UNIX subsystems. They have even integrated a Linux terminal in Windows, which is something nobody would have imagined ten years ago. They will most likely have to rewrite the OS to be POSIX compliant because it's not something they can do with the current OS.
What is the difference with small company?
Microsoft is like the Titanic. It cannot be maneuvered quickly. However, they have the luxury of large amounts of sitting cash, ability to leverage their existing ecosystem to launch new products and have a huge army of developers and lawyers at their disposal. Most things take years to develop because of the large teams.
Small companies, on the other hand, have the ability to pivot very quickly and are often able to disrupt an existing industry without affecting an existing ecosystem that they offer. However, they have opposite problems like not enough cash to experiment with stuff and they might need to work harder at retaining talent.
Where is the gap between bing and google? Why is Google search more satisfying?
An earlier comment may be relevant to your question.
The gap between Bing and Google is quite narrow. They have engineers cycling between the same set of companies and they're solving the same problems. Each company's approach to how they implemented search varies and that's what distinguishes them from each other.
Google is generally more satisfying because their results are usually more relevant compared to Bing. At the end of the day, you need instant answers and Google does quite well on that. Google had a headstart in Search, so it managed to disrupt the industry and continue to stay on the forefront.
Microsoft, on the other hand, has been playing catchup to Google.
What collaboration tools did you use with your team on Microsoft? Anything in the line of Trello, Asana or Jira?
Every product team had preferred tools. Most of the time similar tools were used across Microsoft. For bug tracking, we used an internal tool called Product Studio.
You can read more about that here.
For source control, we used Source Depot. Not sure if it is still being used. I heard Windows moved to Git a few years ago.
Often internal products turn into external products. The culture has improved significantly since Satya took over.