Pipy 0.50.0 is released!

Pipy 0.50.0 is now available. This release consists of 320+ commits and comes with improvements to several areas including support for protocols like HTTP/2, GRPC, UDP, WebSockets, and more, support for anonymous pipelines, enhanced observability support, new filters, optimization is done for PipyJS and code readability, better documentation, and some bug fixes. The Pipy Runtime API has expanded for new pipeline types, pipeline lifecycle callbacks, support for transparent mode proxy, and a few other optimizations.

This release was truly a community effort and could not have been made possible without all of the hard work from everyone involved in active discussions and the Pipy project on GitHub.The Pipy community provides code submissions covering new functionality and bug fixes, documentation improvements, quality assurance testing, continuous integration environments, bug reports, and much more. Everyone has done their part to make this release possible! If you’d like to join this amazing community, you can find it on GitHub, Slack, and the Pipy discussion groups.

Below we list the most remarkable changes in the Core API and PipyJS. For detailed change log refer to Pipy Release Page

Core

Pipelines

• New pipeline type for reading from files (or stdin)
• Added onStart()/onEnd() pipeline lifecycle callbacks
• Support idle timeout for inbound and outbound connections
• Transparent mode port listening
• Socket reading performance optimization

Metrics

• Collect metrics on a pipy repo node from all workers connected to it
• Record metric history on a repo node for up to 5 minutes for charts showing on the Admin UI
• Expose metrics from a pipy repo node to Prometheus with gzip compression
• Added new metrics for inbound/outbound data and connection times

Logging

• Send real-time log streams from workers to the pipy repo

Codebases

• Send pipy repo codebase update notifications to workers via WebSocket to reload script with no delay
• Support codebase deletion of a pipy repo

PipyJS

• New nesting syntax for in-place anonymous sub-pipelines
• Added pipy.solve() to evaluate script from external files
• Added -e option for evaluating PipyJS script from the command line
• Support function argument default values
• String size upper limit raised to 256MB
• Performance optimization of object literal construction

Protocols

HTTP/2 & gRPC

• Support full HTTP/2 specification except for server push
• Extended demuxHTTP()/muxHTTP() to support HTTP/2 protocol negotiation

HTTP/1.x

• Support HTTP message trailers
• Proper handling of responses with status code 100
• Detect plain HTTP/1 traffic by detectProtocol() filter
• HTTP/1.x decoder performance optimization

MQTT

• Added a new pipeline communication model for MQTT-like protocols using input()/output() filters
• Standard-conformance bug fixes

WebSocket

• Added WebSocket codecs
• Added a communication channel between the worker and the repo via WebSocket

UDP

• Added UDP listeners

TLS

• Support ALPN in acceptTLS() and connectTLS() filters
• OpenSSL version bumped to 1.1.1q

API

• New API for general purpose logging
• Added algo.Quota as a replacement of the old "accounts" used by throttling filters
• Added Data.shiftWhile() and shiftTo() for simple data format parsing in PipyJS
• Added LoadBalancer.next() with builtin connection pools
• Added algo.uuid() for UUIDv4 generation
• JSON.encode() performance optimization

Filters

• New filter branch() that supports conditional embedded anonymous sub-pipelines
• New filter depositMessage() for buffering large messages in a temp file
• New filter deframe() for general purpose protocol deframing using PipyJS
• New filter compressMessage() for deflate/gzip compression
• New filter decodeMultipart() for decoding a MIME multipart message into multiple messages
• Support caps on the maximum queue size per session for all muxer filters
• Support Brotli algorithm in decompressMessage() filter
• Support writing to stdout using tee()
• Support timeout in filter wait()

Documentation

Tremendous growth in documentation coverage has been done since the last release in English, Chinese and Japanese.

We would like to thank each and every contributor who was involved in this release.