What & Why of programmable proxy

Question which gets asked too often is "When you say programmable proxy, what is a programmable proxy, and why do you need a programmable proxy?". This article tries to answer this question from different perspective. We will start with brief definition of what is a proxy? then move forward our discussion of evolution, proxy went through its development process, explaining the need for those stages and what benefits each stage provide and why such stage improvements were required. In end we will discuss several aspects of "programmability" and provide a summary of "why we need programmable proxy".

A proxy server is usually deployed in the middle of two isolated networks where it has access to both sides of the network and transfers data from one side to the other to achieve network connectivity. Proxy is a kind of serial network device, which has existed since the birth of computer networks. Proxy acts not only as a network connector, but it also opens a path to new functionality and usage scenarios like:

• Routing. The proxy forwards data to different destinations according to the characteristics of the data defined.
• Load balancing. During forwarding, data is distributed to different destinations to improve throughput and avoid destination single point of failure. Load balancing is becoming an area of proxy segmentation functionality
• Fail over. In the forwarding process, when the destination fails, the proxy can forward the data to an alternate destination, providing uninterrupted service to the requester
• Access control. Proxy can decide what traffic is allowed through and what needs to be blocked. Web Application Firewall (WAF) is a typical proxy application example
• Identity management. Access control is often based on identity information, so proxies often have identity management capabilities as well
• Network acceleration. proxy accelerates network access by caching data
• Metrics collection. Proxy captures data statistics and summarizes it to NPM software for network optimization and network planning
• Information security. In addition to access control, proxies can also be used for security auditing, TLS/SSL offloading, and data encryption to meet security requirements

In addition to proxying, routers provide the functionality of bridging two networks. Routers work at layer 3 of the network while Proxies work at layers above 3, like layer 4 and layer 7.

Proxy servers have gone through following evolutions during its development:

1. Configurations file era. Proxy software (mostly open source software), which makes up the majority of network infrastructure software, provides different functions in different domains, such as proxy for different protocols, proxy for load balancing, proxy for cache acceleration. This generation of proxy is all configuration based. User sets parameters, configures rules in the configuration file, and then starts the service process to execute the rules
2. Configuration DSL. Static configuration files are difficult to express complex logic, so many proxies introduced thin scripting capabilities on top of configuration, commonly referred to as "configuration languages" or Domain specific languages (DSL for short), such as Haprxoy's ACL or Varnish's VCL
3. Scripting language era. As the logic becomes more complex, it becomes more difficult to express that via configuration languages; At the same time, when the number of configuration languages reaches a certain extent, the management of configuration languages itself becomes difficult. Shell scripts, for example, one can write simple logic, but when the shell code reaches a certain level, it is often a step toward more structured scripting languages like Perl or Python. Proxy supports scripting language, which has both the convenience of scripting language and the structural advantage of programming language. Examples of this are OpenResty (Nginx + Lua) and Nginx Plus(Nginx + NJS). These examples also include proxy servers implemented by a number of application programming languages, such as Node.js based StrongLoop and Spring Gateway, which often have scripting capabilities of their own
4. Cluster age. Script language solves the difficulties in modularization and structuring of complex logic in proxy server. A further requirement at this point is to integrate proxies with other administrative control tools, hence the need for a REST or alike interface. The external control plane can dynamically update the logic in the script through the REST or alike interface. At the same time, the use of proxies has moved from single instances to cluster of proxies, so proxy software like Envoy and OpenResty based Kong often support clustering capabilities themselves, implementing clustering capabilities in some centralized or shared way, while providing REST interfaces. For proxies of type #4 mentioned above, cluster management is generally possible through configuration management; Configuration management tools can also expose REST interfaces. For example, the Ansible + Nginx solution implements similar capabilities to #5. In contrast, scheme #4 requires more components to form the scheme, while scheme #5 is more convergent
5. Cloud era. In this era, proxies are deployed in a distributed manner. The most common scenario is to deploy one proxy for each application process e.g Sidecar Proxy pattern. Adoption of distributed proxies, opened the doors for using different rules and policies for different upstream services, that is, multi-tenant capability. The various upstream services not only have logically independent rules and policies; Physical isolation is also provided, allowing granular management at the process and interface level. If we consider the control plane and data plane of the service mesh as a whole, then the service mesh is representative of this domain, typical examples are Istio+Envoy, Linkerd + Linkerd proxy. Pipy is a product of this age and falls under this category.

In the above evolution stages, each stage is an improvement over the previous one:

• #2 adds basic scripting capabilities over #1. This basic scripting capability is augmented by dynamic capabilities on top of configuration files. For example, get the characteristics of the request at run time (such as HTTP headers), and then make dynamic logical decisions based on those characteristics to perform specific actions
• #3 adds full scripting capabilities over #2, allowing scripting logic to be structured and modular. In the #2 stage, when logic becomes complex, the volume of scripts increases dramatically, and structured scripting capabilities become a necessity
• #4 has more REST interfaces and clustering capabilities than #3. To horizontally expand the proxy capacity, you need to cluster multiple proxy instances. Instances in the cluster share configurations and scripts, and users can manage configurations and scripts through the REST interfaces
• #5 has more distributed capabilities than #4, mainly because different instances within the same cluster run different scripts and configurations. In #4 mode, there are usually different configurations and policies for different upstream services, such as different authentication modes and access control mechanisms. As upstream services grow, the configurations of these different upstream services are logically separated but physically run in the same proxy process. This logically separated configuration and policy scripts running in the same physical process has some disadvantages: the more logic running in one process brings the more complexity; different upstream services share resources such as CPU and memory, affecting each other. If a script of an upstream service has a security vulnerability, the configurations of other upstream services may be leaked, resulting in security risks. #5 stage is an improvement over the #4 in that the proxy processes for each upstream service are independent and isolated from each other. They are managed by the same cluster manager, but the configuration and scripts at work are separated and isolated. This isolation is a strong requirement in a multi-tenant environment -- different upstream services belong to different tenants, and tenants should not affect each other or know each other's configurations. #5 can be thought of as the extreme mode of "#4 multi-clustering "-- at its most extreme, each process has its own configuration

Lets take another look at the evolution of proxies -- The Evolution of Requirements.

1. The first generation of proxies mainly implemented proxy functionality and provided basic configurable capabilities. At that time, network equipment, especially Serial network equipment, required proxy to be highly reliable; The real-time transmission of massive data requires high throughput, low latency and low resources. Like all software, proxy also required to support modularity and extensibility. Proxy in this stage was mainly developed in C language, and the corresponding development of extension modules were also developed in C language. Extension modules get loaded dynamically at the start of the process. To summarize, proxy requirements at this stage are: connectivity (network capabilities), ease of use (configurable via configuration files), reliability (requirements for cross-linking devices), high performance, scalability
2. The second generation of proxies got further improvements in extensibility and flexibility, such as some dynamic data acquisition and making logical judgment. The introduction of scripts further enhanced the ease of use. Support for combinatorial logic and dynamic data retrieval provided flexibility while improving scalability
3. The main improvements of the 3rd generation proxies over the 2nd generation proxies are manageability, developer friendliness and programmability. Scripting capabilities are widely used, mainly because it is difficult to develop and maintain extensions using C language, while scripting languages are easier to learn and provide fast turnarounds in developments when compared to compiled languages counterparts. Developers' development efficiency and the difficulty of massive script maintenance require this generation of proxies to use a more structured scripting language, and to maintain performance, resource utilization and other core capabilities of the previous generation. The use of structured and modular scripting language, opened the era of programmable proxies and required proxies to provide two level of possibilities, one is to use C language for development of core modules, and scripts to be used to program dynamic logic; In other words programmable includes the meaning of giving its user the power of development of core modules and dynamic logic.
4. The fourth generation of proxies starts with cluster support capabilities, which are improvements to manageability. With the support of REST interfaces, proxies become part of network infrastructure implementation, and a landing place for infra as code. REST interface capabilities improved proxies ability to be managed and considered a part of the ease of administration. External interfaces are also an important feature of programmable, and REST, as the most common form of interface, is also widely used in the proxy server world. At this point, programmable consists of three layers: programmable core modules as described in #3, programmable dynamic logic , and programmable external interfaces. The emergence of proxy server cluster reflects the change of scalability from function expansion to resource expansion. The emergence of REST interfaces provides the technical foundation for further self-service and managed services
5. The evolution of the fifth generation of proxy is driven by the popularity and rapid development of cloud computing. The elasticity, self-service, tenant, isolation, and metering of the cloud require that the proxy service software have the best capability of the cloud native. If the fourth generation of agents is for system administrators, the fifth generation of agents is for cloud services. While fully maintaining the characteristics of previous generations of proxy software, Cloud Ready is further incorporated. With the expansion of cloud computing to the edge, the fifth generation of proxies are also toward the development of heterogeneous hardware, heterogeneous software, low energy consumption, so this generation of proxies began to show the ability to optimize the integration of cloud and edge. The fifth generation of proxies in the programmable further evolution, from the core module, dynamic logic, external interface, increased cloud ability; Supports distributed, multi-tenant, and metering. Metering is a derivative requirement of multi-tenancy, which requires isolation on the one hand, and resources that can be measured at the smallest possible granularity on the other

Let's summarize the above discussion into a tabular format, with the first column identifying a specific requirement that the proxy meets; Header row represents proxies at different stages, with the typical examples or known software in parentheses. In each cell, we use * to indicate whether such capabilities are available and to what extent (1-5 *, 5 * for full support, and 1 * for basic support).

SN	Requirement	Configuration (squid, httpd, nginx)	Configuration Language (varnish, haproxy)	Scripting support (nginx+lua, nginx+js)	Clustering (kong, envoy)	Cloud (istio+envoy, linkerd, pipy)	Remarks
1.	Connectivity	* * * * *	* * * * *	* * * * *	* * * * *	* * * * *	Connectivity in the cloud era began using kernel technologies such as iptables and ebpf; Previously, there was only user Space process mode
2.	reliability	* * * * *	* * * * *	* * * * *	* * * * *	* * * * *	Reliability has always been the most important fundamental capability of proxy server
3.	high performance	* * *	* * * *	* * * * *	* * * * *	* * * * *	Performance includes throughput, latency, error rate, and deviation from the mean. Latency metrics P99, P999 and other. Early proxy software has a long tail effect, so the indicators above P99 are not as good as later software. Proxy with high-performance scripts often perform better than their predecessors when returning the same content. Agents using proactive technology are more stable (with less deviation from the mean) while providing the same performance
4.	flexibility	*	* *	* * *	* * * *	* * * * *	Compared with the fourth generation, the fifth-generation proxies significantly enhances the multi-protocol support capability, so we give this generation a five-star evaluation. Moreover, the processing model of the fifth generation can adapt to various protocols and has universality, which is better than that of the fourth generation
5.	scalability	*	* *	* * *	* * *	* * * *	Similar to Flexibility, the 5th generation proxies supports multiple protocols in addition to core function extension development, 7-layer logic extension development, so we give it one star more than the 4th generation
6.	hardware compatibility	* * * *	* * * *	* * * *	* * * *	* * * *	Proxies developed in C or C++ generally have better hardware compatibility and a more active community to migrate applications to new hardware architectures. Proxies developed using Rust, Go, and Lua have been relatively slow to migrate for hardware compatibility
7.	system compatibility	* * *	* * *	* * * *	* * * *	* * * * *	system mainly includes two aspects, one is the operating system, the other is the cloud platform. In terms of operating system compatibility, each generation of proxies is similar; However, in terms of cloud platform compatibility, both the fourth and fifth generation proxies have more fully considered and realized cloud compatibility. In contrast, the significant difference between the fifth generation and the fourth generation is the ability to support multi-tenant
8.	ease of management	* *	* *	* *	* * *	* * * *	ease of management is a function of the system operations & administrator roles. The first and second generations mainly use configuration files, based on which the use of configuration management tools to achieve automatic and batch management. In the third generation, in addition to managing configuration files, we need to further manage source files. But in essence, there is no significant difference from the first and second generation of ease of management. The fourth generation provides REST interfaces, which greatly improves the ease of management. In addition to REST, the fifth generation usually provides a control plane for the cloud to manage proxies; It also provides multiple external interfaces to meet other management requirements, such as monitoring, auditing, and statistics
9.	ease of use	*	*	*	* *	* * *	The primary users of the first three generations of proxies are ops & sys administrators. In the fourth generation, administrators began to provide some functions to users, and the as-a-service mode began to appear. The fifth generation takes into account more user scenarios and provides more rent-oriented capabilities
10.	ease of development	*	* *	* * *	* * * *	* * * * *	The development around the proxies includes two aspects, one is inside the proxy to achieve the functionality, another is outside the proxy to achieve the management ability of the proxy. The first three generations provide an interface for internal development; The latter two generations provide both internal and external interfaces. The significant improvement of the fifth generation compared with the fourth generation is the cloud interface
11.	core interface is programmable	*	*	*	*	*	Each generation of proxies provides the ability to extend core interfaces, but these interfaces are too low-level and difficult to master
12.	functionality extension is programmable	*	* *	* * *	* * * *	* * * * *	Providing the ability to extend functions more efficiently is part of the process that gets better with each generation of proxies. It is the core metric of programmable proxy
13.	protocol extensions are programmable				* *	* * *	The first three generations are mainly for single agreements, or fixed agreements. Starting with the fourth generation, users began to seek support for multiple protocols and custom protocols. In the fifth generation, protocol extension is considered as a core capability in the design
14.	modular scripting			* * *	* * * *	* * * *	Third generation proxies are beginning to pay more attention to script structuring; The fourth and fifth generations attempt to prepare for more structured programming, such as Envoy's attempt to provide multilingual support through WASM; Pipy introduces high-performance JS scripts for better structuring
15.	configuration management is programmable				* *	* * *	The first three generations of proxy configuration are mainly for ops management personnel, and external configuration management tools are based on this premise. The fourth generation supports REST management interfaces. The fifth generation further provides a standard cloud interface for configuration management
16.	resource extensions are programmable	*	*	*	* *	* * * *	In the first three generations, proxy capacity expansion is mainly to increase the number of threads or processes. The fourth generation provides scale-out capability for processes, known as clustering capability. On the basis of the fourth generation, the fifth generation on the one hand provides horizontal expansion capabilities, on the other hand provides capabilities under smaller resources to support more fine-grained metering and billing; That is, it not only supports incremental scaling, but also provides the ability to reduce scaling, and all of these capabilities provide programming interfaces
17.	tenant extension programmable					* * *	The cloud is something that has emerged at the same time as the fourth generation of proxy, and tenant, as a core feature of the cloud, is not well supported in the fourth generation. The fifth generation is designed on the premise of cloud, considering and providing tenants with the possibility of programming their own extensions

rows #11 to #17 in the table above are the specific aspects of Programmable Proxy. These aspects also constitute the answer to the question of Why Programmable Proxy:

• The internal functions of the proxy need to be expanded, including the extension of the underlying core capabilities, the extension of supporting more protocols, and Layer7 processing capabilities (forwarding, routing, judgment, access control, etc.); These Layer7 processing power require a more convenient way of programming, that is, scripting and structured programming ability
• Proxies need to provide interfaces externally to integrate into larger management systems (such as cloud platforms), including configuration management, resource management, and so on
• Proxies need to provide extensible capabilities for different roles, including operations, administrators, resource providers, and tenants, all of which require the capability of Programmable to some extent
• Also, like any programmable component, Programmable Proxy requires accompanying documentation, development manuals, code management, dependency management, build and deployment tools, and preferably a visual development and debugging environment. Only when these requirements are fully met can users better manage network traffic and the services carried on the traffic