I had been working some on HiTycho lately. I see strong use cases for HPX clustering in Geospatial work. For example, one could pass off map tiles in a large map for merging satellite imagery to independent tasks running on separate nodes and machines in a HPC cluster. Processing video feeds by splitting frames, VR simulations, and forms of AI processing may all be valid use case for this kind of scaling. Naturally I think about carrier scale telephony, too.
I have also seen the future, the promised land for true scalable processing, both in horizontal distributed computing and vertical clustering in C++. This is why I am interested in combining HPX with Areg. This to me could make existing things like Temporal feel like legacy code.
The main features in this weekends drop of HiTycho for HPC was in-memory buffer streaming and fork injections I borrowed from Busuto. Busuto has also been better aligned with HiTycho. This will make it easier to test and develop locally with Busuto and later migrate applications to HiTycho for true massive parallel scaling. That is my deeper vision for both of these libraries and why I do want to eventually make Areg work with both.
For further reading:
https://github.com/aregtech/areg-sdk
https://github.com/dyfet/busuto.git
https://github.com/dyfet/hitycho.git
Top comments (3)
Perhaps less noticed is Kakusu (github.com/dyfet/kakusu) which I now use as a generic almost stand-alone C++ header-only front-end and common API to various back-end crypto toolkits (wolfssl, libsodium, openssl, minicrypt, etc). To glue that together and handle key data, digests, etc. it uses a byte_array class. It also includes C++ templates for making consistent hash rings which are commonly used to distribute requests for hosts dynamically in distributed computing.
@aregtech
Interesting project. Security will be a big focus for the next release. I can see how Kakusu’s approach could complement at the data/security layer. I’d really value your input on security topics, while keeping in mind that embedded/IoT efficiency stays the top priority :)
Indeed, that is why Kakusu can use backends like minicrypt and wolfssl, which are both ideal for embedded cryptographic work, as well as libsodium and openssl. Kakusu is pretty stand-alone, it has no real platform requirements of its own, it is header-only to avoid runtime library linking since it can use different backends, and is something that could potentially be dropped in and then made the basis of a more complete crypto framework all by itself. That is actually why I had pointed it out.