Hi guys! So I am starting up on a new application and want to discuss which programming language should I choose to build my server side infrastructure on.
The architecture will be based on containers and Kubernetes for future scalability. The idea of the application is very simple with a few CRUD endpoint. It involves video storage, voice transcriptions, video processing and natural language processing for mood and opinion mining and a lot of server side text manipulations.
Now I know if there is ML involved I should probably choose Python and call it a day but that is exactly what I want to discuss. A lot of these services are provided by cloud solutions and we want to use them initially and then build our own solutions once we have a good amount of datasets in the coming years.
So, my question is how viable is Golang for purposes like string manipulation and AI? I am leaning towards Golang because of its seamless integration in a scalable cloud infrastructure. I have heard a lot of good things about it being super fast and saving deployment costs in cloud infrastructure that charge for computation time.
For more context, I have already built backend applications with Node and Python combined in the past and achieve good results with it but it wasn't built to scale and was a concept rather than a full fledged startup with investment.
Looking forward to your valuable input.
Oldest comments (4)
Since you used the word "we" and it seems that this app needs to make money at some point I would say choose the language that at least one of you knows well, better if one the majority knows well :) Compatibly with the needs of this new project.
To get back to your questions:
You talk about saving infrastructure cost and scalability which is true in theory but would it be fine if to choose one language you had to spend hundreds if not thousands of man hours to develop tools similar to existing ones that have been battle tested in the other language?
I like both languages and I'm not saying Go is not the right choice, but I think what matters the most is the libraries and tools around them.
You want to deal with natural language processing, ML and AI. Spend a day or two checking the state of those libraries in the Go community vs those in the Python community. I honestly have no idea how good or bad ML tools are in Go.
Make a shopping list of all the tools you'd need to incorporate or write from scratch for your MVP, you can learn Go in a day, play with those tools and you'll know which is the right choice.
A couple of days spent now will save you trouble down the line, instead of going all in with one language or another without knowing what's available.
Aha, gotcha. I probably asked this question because I want discover what Golang can offer. I can use an experts opinion on this but I guess you're right. Everyone has their own preferences and there is no silver bullet. Discovering the pitfalls my self through documentation and trial and error is a good way forward and I am currently doing that.
What I found is that Node would be the best option for me to build a quick and dirty CRUD app as I have the most experience there. Once I want to do ML I can write services in Python querying the same database. Heck if there is a lot of traffic and things are getting slower/expansive I can slowly migrate to Golang in the future.
Ok my expert opinion: if you're satisfied with external ML services build everything with Node since you know it best, otherwise build everything with Python (both the server and the business logic). With Python you have tools like Numpy, Pandas, TensorFlow, Jupyter, web frameworks, official support by big companies to deploy in the cloud and so on.
Once you have your monolithic MVP you can circle back on what to do next. You might not even need to rewrite anything. If you have to, you can pick apart the monolith one endpoint at the time.
Start simple :)
At the end of the day, languages aren't going to matter too much unless you have corporate policy that dictates you must use some ancient version of language X. A language is just a tool, and for the most-part, the majority of languages will be able to produce a solution; some will be easier than others and some will be more elegant, some will offer you no community support and some of them will have a vibrant community willing to help you out. Pick what you are most comfortable with.
We can buy more hardware to deal with slower languages, or implement cache services. Over time, you can refactor parts of the application into microservices if you need to use one specific language for some unique functionality.
It sounds like you are writing a pretty cool app that will involve a lot of moving pieces. There are a lot of design questions you need to answer and the good news is that you don't have to answer them in a computer language! Working code is the most valuable kind of code out there. Use whatever it takes to produce that. It's great that you have thought ahead and identified possible bottlenecks, but you'll never have a chance to encounter them if you get stuck with analysis paralysis!