Maxi Contieri

Posted on Jun 27, 2022 • Originally published at maximilianocontieri.com

GPT-3 Training Programmers for the Present (and the Future)

#programming #gpt3 #machinelearning #beginners

I wrote a paper in Spanish and asked "someone else" to rewrite it

Last year, I wrote a paper in Spanish about the future of programmers.

TL;DR: Instead of manually translating my paper, I decided to rewrite it completely with GPT-3.

In the same way, The Guardian asked GPT-3 when it was in private beta.

https://www.theguardian.com/commentisfree/2020/sep/08/robot-wrote-this-article-gpt-3

When I asked it to translate the article, it decided the title was not good enough.

So it changed it to present AND future

😱Scary, isn't it?

Let's move on.

So this is the original paper rewritten by GPT-3:

The Paper

// Translate this into English and convert it to a technical article:

Summary

The current market is looking for programmers to stack bricks (1) using their trendy languages. Such professionals are trained in specific tools that have very short life cycles and then become obsolete.

There are currently alternatives to carry out computer developments using intelligent people, capable of making their own decisions and participating in the creative process.

The programmers of the future will have to be excellent and declarative (2). Professionals who know about algorithms and complexity and who know-how

Introduction

Most developers in the world are making changes to old systems or have to use complex preexisting libraries or frameworks that they have little control over and few opportunities to modify. (3)

In today's software industry, it is very unlikely to find the need to develop from scratch, with a completely blank page.

The specific lifespan of a programmer age along with the fashionable language. This period of time is usually less than a decade, so the industry discards professionals as obsolete before ten years of training. (4)

In the last decades, they were fashionable as supposed silver bullets (5), languages like Visual Basic, C ++, Java, Php, Perl, Javascript, Ruby, Python, and GoLang.
Some of them are already ceasing to be used, leaving their place for new fashions. (6)

The general concepts (which Frederick Brooks calls essential (7)) are better absorbed in our first academic stage. We must focus on forming these concepts so that professionals can easily migrate from one technology (which Brooks calls accidental) to another technology (also accidental and, therefore, quickly obsolete).

The threat

By 2021 there are already numerous alternatives to artificial intelligence and machine learning, capable of carrying out low-level programming and algorithmic tasks (8) (9).
Modern virtual machines (those that interpret the high-level language in which we program and that exist, for example, in web browsers) optimize the code for us. (10)

As technology advances, low-level programmers will no longer be required in the same way that no one is looking for librarians today (11). Similar to what happens with other automatable and obsolete professions. (12)

Our profession is still based on teaching low-level imperative programming as if we were facing the problems of the 1960s or 1970s when the specialized hardware was punch cards and open tape machines.

We educate our students to carry out absurd and obsolete algorithmic optimizations, very close to the language of machines and very far from the modeling of entities of the problem that we are representing in our computer solution.

We train professionals to understand how a computer works and to lower themselves to speak in its language, forming low-level mental models and trying to reason in the algorithmic way in which a Turing machine processes.

However, today we can develop semantic models with several layers of abstraction above as we approach declarative mental models we move away at the same time from the computable abstractions of machines. The most declarative programming languages allow us to reason and infer rules about our models. Against this trend, our students write their instructions based on ones and zeros.

Moore's law (14) tells us about the speed of change of the hardware and its exponential growth in processing power. With respect to software, we grow linearly limited by the realities of many decades ago and the inertia of trainers. In analogy with Malthusian economic theory (15) we need to catch up with the hardware curve to be able to release all our potential before the machines do it for us.

The lack of abstraction and understanding of reality is a symptom of an inertial mentality, based on the time restrictions of processors and storage of the 60s and 70s. These restrictions are limited today to very specific domains and are not representative of the problems we have to solve today as software engineers in most of our jobs. (16)

The alternative

We need to train generalists, but not theorists. People who understand the basics and foundations of software engineering and can adapt to accidental implementations based on current tools, but also to future ones that we do not yet know today.

Concepts such as software design, life cycle models, teamwork and building a shared theory (17), continuous integration and deployment or architecture, are much more important than learning to minimize CPU usage, to use React, the Vue framework, or the Keta library.

"It is very difficult to make predictions, especially about the future". This phrase, attributed to the great Niels Bohr, tells us that we should be careful when trying to anticipate the future. Paraphrasing the 2004 Turing Award (18) Alan Key, said: "The best way to predict the future is to invent it".

We do not know for sure what the path of machine automation will be, but we can predict that the most creative and engineering part of our profession should be one of the last bastions to fall before automation.

The next generation of software developers will have to focus their skills on design, modeling the real world, and creating abstractions that evolve along with the problem domain to avoid building obsolete and legacy software (19).

This strategic approach is not limited only to development careers. In data science there are similar problems: data scientists focused on optimizing existing algorithms instead of training generalist professionals, able to understand the academic and commercial problems of our country with a battery of solutions and good criteria to determine which to choose in each situation. They should be trained to model the real problem and evaluate different possible solutions.

Data science professionals have an incredible variety of tools to adjust their models. We are still far from finding the second "super algorithm" (20) capable of solving specific problems with a generic solution. Our future professionals have the only general-purpose computer that can reasonably solve various specific problems: Their brilliant minds.

Our responsibility is to feed and stimulate those brains with real problems and not with toy solutions that computers solve (already in 2020) much more efficiently.

Paradoxically, wasting our brains by asking them to perform low-level optimizations for machines is the worst way to optimize our most expensive resource.

Historically, we have privileged the development of very specific and theoretical tools. This is very common and desirable in science because theoretical advances usually precede concrete implementations and uses by decades.

However, in software development, discoveries and findings are mostly found in the private sector before academia. Our professionals must be trained in methodology and concepts above accidental trendy technologies and languages. This generates tension between the market that wants "implementers expert in a tool" to discard them when it fulfills its fashion cycle of 5 or 10 years.

Our professionals should not be disposable or recyclable. We must train them in techniques and they must keep themselves constantly updated, as happens in other professions such as medicine, physics, or biotechnology.

What do we need to teach?

In addition to "soft" skills such as construction and teamwork (since software arises from a collective activity) (17), we must teach design and prototyping techniques to validate our high-level solutions.

As for software, it is imperative to teach solution design, focusing on the behavior of our models and, paraphrasing Donald Knuth (21), the historical author of most of the algorithms we use today, avoiding premature optimizations because we want to play a game that machines dominate much better than us.

The opportunity

Training talent is an accessible option for any country with a good academic level, such as Argentina.

Investing in training excellent software engineers is a strategic decision and an opportunity for take-off that has already been exploited by many other countries such as Estonia, Ireland, Israel and India. The Sadosky Foundation is currently working in this direction. (22)

In Argentina, we have excellent teachers, a good level of English, an unbeatable time zone to dialogue with USA and Europe, and a culture compatible with the most developed countries.
We need to prioritize information technologies and, within them, train intelligent and declarative engineers rather than mechanized programmers and low-level optimizers.

What should we teach our engineers?

Our professionals must have basic knowledge of programming, algorithms, complexity, and databases.

Above all, they must learn to make designs based on continuous integration and continuous deployment models, with automated tests, using agile techniques such as Test-Driven Development. (23)

The software produced must be declarative and based on the desired behavior (and specified in the automated functional tests); we must stop thinking in the reigning paradigm of the 60s and 70s, based on data types and file and string manipulations, to focus on high-level models that accompany the simulation of any aspect of the real world that we want to represent to solve a certain problem. (24)

The techniques of design based on behavior are agnostic with respect to the accidental technology of fashion and this allows an engineer trained with these concepts 30 years ago to be able to make concrete developments even today.

Unfortunately, such a situation is not replicated by programmers who dominated some fashionable language, which practically has no use and that makes them not find good options in the labor market. The paradox is that a trade with full employment discards these professionals for not having been able to adapt. (25)

The change, currently, is even more vertiginous. Technologies last much less and obsolescence stalks us, unless we are intelligent and broad, and have the appropriate training.

Future Work

This is an opinion piece. As future work to support the current thesis, we should carry out a quantitative study including figures on employee turnover (25), average time in each job according to age and studies completed, etc.
To do this we must use techniques related to the social sciences under a multidisciplinary approach.

Conclusions

The future has already arrived. We don't have much idea of what a programmer's job will be like in 5 or 10 years, but we have strong indications that it will not be related to writing algorithms based on data structures. We must train professionals who quickly understand a real-life problem and know how to build simulators with very small conceptual leaps, so that they can evolve alongside the problems we are solving today.

Acknowledgments

Part of the ideas in this article was born from the teaching work in the Software Engineering course in the Computer Science degree at the Faculty of Exact Sciences, University of Buenos Aires.

The translation is accurate and beautiful.

I have checked it with Grammarly and Hemmingway app and found just a few grammatical mistakes.

As an example:

Trabajo Futuro (Spanish for Future Work)

Este es un artículo de opinión. Como trabajo futuro para apoyar la presente tesis deberíamos realizar un trabajo cuantitativo incluyendo cifras de rotación de empleados (25), tiempo promedio en cada trabajo según la edad y los estudios realizados etc.
Para ello debemos utilizar técnicas relacionadas a las ciencias sociales bajo un enfoque multidisciplinario.

Translates in GPT-3 as:

This is an opinion piece. As future work to support the current thesis, we should carry out a quantitative study including figures on employee turnover (25), average time in each job according to age and studies undertaken etc.
To do this we must use techniques related to the social sciences from a multidisciplinary perspective.

While Google Translate yields:

Future work

This is an opinion article. As future work to support this thesis, we should carry out quantitative work including employee turnover figures (25), average time in each job according to age and studies carried out, etc.
For this we must use techniques related to the social sciences under a multidisciplinary approach.

Model calibration

I used text-davinci-002 model with Temperature 0 (Randomness) and Maximum Length 2000 tokens

GPT-3 Opinion

What is your opinion on this article?

I think that this article makes some valid points about the future of programming and the need for programmers to be adaptable to change. I agree that the industry is moving towards more automated solutions and that this will eventually lead to a decrease in the need for programmers. However, I think that there will always be a need for programmers who are able to understand and solve complex problems.

😱(Scary Again!!)

Meta Conclusion

I will definitively, keep using GPT-3 to write technical articles to improve them.

DEV Community