DEV Community: Megabit

Simple and easy-to-use JSON parser in C

Megabit — Tue, 12 Sep 2023 12:25:40 +0000

This article introduces the JSON parser of the open source C library Melon.

I believe many readers have heard of or even used the cJSON. So this article will compare cJSON and Melon’s JSON component.

Let’s take a look together below.

Encode

Suppose we want to build the following JSON:

{
    "name": "Awesome 4K",
    "resolutions": [
        {
            "width": 1280,
            "height": 720
        },
        {
            "width": 1920,
            "height": 1080
        },
        {
            "width": 3840,
            "height": 2160
        }
    ]
}

So, let’s first take a look at the cJSON version:

#include <stdio.h>
#include <cjson/cJSON.h>

//NOTE: Returns a heap allocated string, you are required to free it after use.
char *create_monitor_with_helpers(void)
{
    const unsigned int resolution_numbers[3][2] = {
        {1280, 720},
        {1920, 1080},
        {3840, 2160}
    };
    char *string = NULL;
    cJSON *resolutions = NULL;
    size_t index = 0;

    cJSON *monitor = cJSON_CreateObject();

    if (cJSON_AddStringToObject(monitor, "name", "Awesome 4K") == NULL)
    {
        goto end;
    }

    resolutions = cJSON_AddArrayToObject(monitor, "resolutions");
    if (resolutions == NULL)
    {
        goto end;
    }

    for (index = 0; index < (sizeof(resolution_numbers) / (2 * sizeof(int))); ++index)
    {
        cJSON *resolution = cJSON_CreateObject();

        if (cJSON_AddNumberToObject(resolution, "width", resolution_numbers[index][0]) == NULL)
        {
            goto end;
        }

        if (cJSON_AddNumberToObject(resolution, "height", resolution_numbers[index][1]) == NULL)
        {
            goto end;
        }

        cJSON_AddItemToArray(resolutions, resolution);
    }

    string = cJSON_Print(monitor);
    if (string == NULL)
    {
        fprintf(stderr, "Failed to print monitor.\n");
    }

end:
    cJSON_Delete(monitor);
    return string;
}

int main(void)
{
    char *p;
    p = create_monitor_with_helpers();
    printf("%s\n", p);
    return 0;
}

Next, let’s take a look at the Melon’s:

#include <stdio.h>
#include "mln_json.h"
#include "mln_log.h"

static mln_string_t *generate(void)
{
    mln_json_t j;
    mln_string_t *ret;

    mln_json_init(&j);

    mln_json_generate(&j, "{s:s,s:[{s:d,s:d},{s:d,s:d},{s:d,s:d}]}", \
        "name", "Awesome 4K", "resolutions", "width", 1280, "height", 720, \
        "width", 1920, "height", 1080, "width", 3840, "height", 2160);
    ret = mln_json_encode(&j);

    mln_json_destroy(&j);

    return ret;
}

int main(void)
{
    mln_string_t *p;
    p = generate();
    mln_log(none, "%S\n", p);
    return 0;
}

Decode

We have the following JSON:

{
    "name": "Awesome 4K",
    "resolutions": [
        {
            "width": 1280,
            "height": 720
        }
    ]
}

Let's take a look at the decoding, cJSON first:

#include <stdio.h>
#include <cjson/cJSON.h>

int supports_full_hd(const char * const monitor)
{
    const cJSON *resolution = NULL;
    const cJSON *resolutions = NULL;
    cJSON *monitor_json = cJSON_Parse(monitor);
    if (monitor_json == NULL)
        return -1;

    resolutions = cJSON_GetObjectItemCaseSensitive(monitor_json, "resolutions");
    cJSON_ArrayForEach(resolution, resolutions)
    {
        cJSON *width = cJSON_GetObjectItemCaseSensitive(resolution, "width");
        return width->valuedouble;
    }

    cJSON_Delete(monitor_json);
    return -1;
}

int main(void)
{
    char p[] = "{\"name\":\"Awesome 4K\",\"resolutions\":[{\"width\":1280,\"height\":720}]}";
    int i = supports_full_hd(p);
    printf("%d\n", i);
    return 0;
}

Next is the Melon's:

#include <stdio.h>
#include "mln_json.h"
#include "mln_log.h"

static int handler(mln_json_t *j, void *data)
{
    return (int)mln_json_number_data_get(j);
}

static int parse(mln_string_t *p)
{
    mln_json_t j;
    mln_string_t exp = mln_string("resolutions.0.width");
    mln_json_decode(p, &j);
    return mln_json_parse(&j, &exp, handler, NULL);
}

int main(void)
{
    mln_string_t p = mln_string("{\"name\":\"Awesome 4K\",\"resolutions\":[{\"width\":1280,\"height\":720}]}");
    int i = parse(&p);
    mln_log(none, "%d\n", i);
    return 0;
}

Write at the end

Melon's JSON component mainly provides the following four functions to facilitate users to encode and decode JSON:

mln_json_decode decodes JSON strings into JSON structure nodes
mln_json_parse obtains the corresponding JSON sub-node from the decoded JSON structure based on the given expression
mln_json_generate builds a JSON structure based on the given format information
mln_json_encode generates a JSON string based on the generated JSON structure

Melon's JSON component provides a functional interface that is easy to read and use, making it easier for developers to maintain projects.

Welcome to try the open source C library Melon.

Github：https://github.com/Water-Melon/Melon

Step-sharing scheduling Coroutine HTTP Server

Megabit — Thu, 31 Aug 2023 16:18:33 +0000

This article introduces an open source API server named Medge. Medge is an HTTP server, and its authors currently position it as an API server.

In Medge, we can implement API logic through the scripting language Melang. Melang is a step-sharing scheduling coroutine language, in which each script task is regarded as a coroutine running inside the same single thread, and is scheduled according to the execution step. Therefore, during the development of upper-level logic, developers do not need to consider the problem of coroutine scheduling, nor do they need to consider when to call certain functions to give up CPU execution rights.

Now let’s get to the point and see a complete simple usage example.

After we have installed Medge, we can execute the following command to start it.

medge

We can also use the following parameters to start Medge.

medge -p 8080 -d /opt/medge/ -w 1

-p is used to specify the TCP port number that the HTTP server listens to, if not set, the default is 80. In this example HTTP listens on 0.0.0.0:8080.
-d is used to specify the unified entry directory path of the API service, in this case it is /opt/medge. We will explain the structure of this directory later. If this parameter is not specified, it defaults to /opt/medge.
-w is used to specify the number of worker processes for HTTP. Defaults to 1 if not specified.

Now, the server has been started, but we have not given the specific processing logic of this API service. Let’s write the API service code.

As mentioned above, the -d parameter of Medge specifies the unified entry directory of the API service. Here is the directory tree under this path:

|- /opt/medge/
    |- service_1/
        |- entry.m
        |- ...
    |- service_2/
        |- entry.m
        |- ...

We can see that there are two services in the directory /opt/medge named: service_1 and service_2, and each service has a script file named entry.m. entry.m is the entry file of the service, that is, when Medge receives an HTTP request and finds the corresponding service through the HTTP header Host, it will immediately execute the entry.m script of the service for processing.

In our example, there is only one API service, so the directory tree in this example is:

|- /opt/medge/
    |- test.com/
        |- entry.m
        |- index.m

Let’s implement entry.m and index.m.

//entry.m
/*
 * Implement a simple controller.
 * There are three variable injected in this script task:
 *   1. Req. Its prototype is:
 *       Req {
 *           method; //string  e.g. GET POST DELETE ...
 *           version; //string e.g. HTTP/1.0 HTTP/1.1
 *           uri; //string e.g. /index/index
 *           args; //an key-value array, e.g. ["key":"val", ...]
 *           headers; //an key-value array, e.g. ["Content-Type":"application/json", ...]
 *           body; //string
 *       }
 *
 *    2. Resp. Its prototype is:
 *        Resp {
 *            version; //same as Req's version
 *            code; //integer  e.g. 200
 *            headers; //same as Req's headers
 *            body; //same as Req's body
 *        }
 *
 *.   3. Basedir. A string of the base directory path. (Not used in this example)
 */

#include "@/index.m"

str = Import('str');
sys = Import('sys');

uri = str.slice(Req.uri, '/');
ctlr = str.capitalize(uri[0]);
o = $ctlr;
if (sys.has(o, uri[1]) != 'method') {
  Resp.code = 404;
} else {
  o.__action__ = uri[1];
  Resp.body = o.__action__();
  Resp.headers['Content-Length'] = str.strlen(Resp.body);
}

//index.m

Json = Import('json');

Index {
    @index() {
        Resp.headers['Content-Type'] = 'application/json';
        return Json.encode(['code': 200, 'msg': 'OK']);
    }
}

There are some comments in front of entry.m, which mentions that each request will inject three variables into the entry.m named Req, Resp and Basedir.

Req contains request-related information, such as: method, version, resource path, parameters, request headers and request body.

Resp contains response-related information, such as: version (default is the same as the request version), response status code (default is 200), response headers and response body.

Basedir is a string whose content is the path specified by the Medge -d parameter.

In this example, entry.m implements a very simple controller in the MVC framework, which finds the corresponding processing class and class method according to the URI, and calls them to process the HTTP request.

In this example, we will send the following HTTP request:

curl -v -H "Host: test.com" http://127.0.0.1:8080/index/index

Therefore, the URI /index/index will be split first in the entry.m, and the first index will be capitalized and used as the class name to find out the corresponding class which is named Index in the current scripting environment.

After the class Index is found and instantiated, call its index (that is, the second index in the URI) method to process the request.

After executing the curl command, we will see its output roughly:

*   Trying 127.0.0.1:8080...
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> GET /index/index HTTP/1.1
> Host: test.com
> User-Agent: curl/7.81.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Length: 23
< Content-Type: application/json
< 
* Connection #0 to host 127.0.0.1 left intact
{"code":200,"msg":"OK"}

The End

At present, Medge is still an experimental project, and there are many functions need to be implemented. For example, it only supports HTTP/1.0 HTTP/1.1 version without keepalive at present, and does not support https yet. In addition, Melang scripts currently only support MySQL databases, and there are no corresponding HTTP library functions or classes for developers to send HTTP request in Melang.

If you are interested in Medge, please visit its Github for more detail.

Thanks for reading!

Preemptive scheduling coroutine API server

Megabit — Fri, 28 Jul 2023 02:53:35 +0000

This article introduces the open source API server Medge. Medge is an HTTP server, and it is positioned as an API server.

In Medge, we can use the scripting language Melang to write and implement API logic. Melang is a preemptive scheduling coroutine language, in which each script task is regarded as a coroutine running inside the same single thread, and is scheduled according to the execution step. Therefore, during the development of upper-level logic, developers do not need to consider the problem of coroutine scheduling, nor do they need to consider when to call certain functions to give up CPU execution rights.

Now let’s see a complete simple usage example.

Example

After we have installed Medge, we can use the command medge to start HTTP server.

In this example, we will execute the command as shown below.

$ medge -p 8080 -d /opt/medge/ -w 1

-p is used to specify the TCP port number that the HTTP server listens to, if not set, the default is 80. In this example the HTTP server listens on 0.0.0.0:8080.
-d is used to specify the uniform entry directory path of the API services, in this case it is /opt/medge. We will explain the tree structure of this directory later. If this parameter item is not specified, it defaults to /opt/medge.
-w is used to specify the number of worker processes. Defaults to 1 if not specified.

At this point, the server has been started, but we have not given the specific processing logic of the API service. Then let’s add API services.

As mentioned above, the -d parameter of Medge specifies the unified entry directory of the API services. Below we give the approximate directory tree structure under this path:

|- /opt/medge/
    |- service_1/
        |- entry.m
        |- ...
    |- service_2/
        |- entry.m
        |- ...

It can be seen that there are two services in the /opt/medge directory, namely: service_1 and service_2, and each service has a script file entry.m with the same name. entry.m is the entry file of the service, that is, when Medge receives an HTTP request and finds the corresponding service, it will immediately execute the entry.m script of the service for processing.

In our example, we only has one API service, so the directory tree in this example is:

|- /opt/medge/
    |- 127.0.0.1:8080/
        |- entry.m
        |- index.m

Here, 127.0.0.1:8080 is a directory. The reason for this name is that the service name in Medge is named and distinguished by the header field Host of the HTTP request. And in this example, HTTP server does not have a domain name, so the directory name here is in the form of such an IP:port.

Below, we give the script content of entry.m and index.m:

//entry.m
/*
 * Implement a simple controller.
 * There are three variable injected in this script task:
 *   1. Req. Its prototype is:
 *       Req {
 *           method; //string  e.g. GET POST DELETE ...
 *           version; //string e.g. HTTP/1.0 HTTP/1.1
 *           uri; //string e.g. /index/index
 *           args; //an key-value array, e.g. ["key":"val", ...]
 *           headers; //an key-value array, e.g. ["Content-Type":"application/json", ...]
 *           body; //string
 *       }
 *
 *    2. Resp. Its prototype is:
 *        Resp {
 *            version; //same as Req's version
 *            code; //integer  e.g. 200
 *            headers; //same as Req's headers
 *            body; //same as Req's body
 *        }
 *
 *.   3. Basedir. A string of the base directory path. (Not used in this example)
 */

#include "@/index.m"

str = Import('str');
sys = Import('sys');

uri = str.slice(Req.uri, '/');
ctlr = str.capitalize(uri[0]);
o = $ctlr;
if (sys.has(o, uri[1]) != 'method') {
  Resp.code = 404;
} else {
  o.__action__ = uri[1];
  Resp.body = o.__action__();
  Resp.headers['Content-Length'] = str.strlen(Resp.body);
}

//index.m

Json = Import('json');

Index {
    @index() {
        Resp.headers['Content-Type'] = 'application/json';
        return Json.encode(['code': 200, 'msg': 'OK']);
    }
}

It can be seen that entry.m contains some comment information in front of it, which mentions that each request will inject 3 variables into the entry.m script of its corresponding service:

Req contains information about the request, such as: method, version, resource path, parameters, request headers and request body.
Resp contains information about the response, such as: version (default is the same as the request version), response status code (default is 200), response header and response body.
Basedir is a string containing the path specified by the Medge -d parameter.

entry.m in this example implements a very simple controller in the MVC framework, which finds the corresponding processing class and class method according to the resource path, and calls method to process the HTTP request.

In this example, we will use the following command to send an HTTP request:

$ curl -v http://127.0.0.1:8080/index/index

Therefore, in entry.m, the resource path /index/index will be sliced at first, and the first letter of it will be capitalized after obtaining the first index. Then use this capitalized string as the class name to find if there is a class called Index in the current scripting environment. After finding it, call its index (that is, the second index of the resource path) method to process the request.

After executing the curl command, we will see that its output is:

*   Trying 127.0.0.1:8080...
* Connected to 127.0.0.1 (127.0.0.1) port 8080 (#0)
> GET /index/index HTTP/1.1
> Host: 127.0.0.1:8080
> User-Agent: curl/7.81.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Length: 23
< Content-Type: application/json
< 
* Connection #0 to host 127.0.0.1 left intact
{"code":200,"msg":"OK"}

At the end

At present, Medge is still an experimental project, and there are still many functions to be implemented. For example, it only supports HTTP/1.0 HTTP/1.1 version at present, and does not support https yet. In addition, Melang scripts currently only support MySQL databases, and there are no corresponding http library functions or classes for developers to use.

For more details, please visit its Github repository.

Thanks for reading!

Next generation dynamic tunnel proxy service

Megabit — Mon, 20 Feb 2023 07:35:12 +0000

Overview

Speaking of tunnel proxy, familiar friends will definitely think of frp, which is a well-known open source tunnel proxy. And this product has also built its own ecosystem around the tunnel agency service.

This article is to throw out an idea and use an experimental open source project for demonstration.

As early as a few years ago, some people may have noticed that some software has begun to try to support dynamic updates, especially in cloud-native environments. For example: Nginx's dyups module, Nginx's Unit service, Kong's location hot update, etc.

Therefore, for tunnel proxy services such as frp, I also want to try the hot update mode. So there is this experimental project - Menet.

This is an experimental project, the experiment lies in two aspects:

Use Melang language development to verify the ability and stability of Melang language
Supports dynamic update of configuration to realize the dynamic establishment and disconnection of upstream, downstream and tunnels

This project is not recommended for production environments, this article is only used to demonstrate the dynamic update effect.

Usage

To use Menet, you only need to install the Melang script interpreter, and then execute Menet.

$ melang menet.m

Here, there are two configurations that need to be given before startup:

Management port, used to deliver configuration to this proxy service
Tunnel port, the tunnel listening port of each service is determined at startup, and the remaining listening ports or remote connections are dynamically created at runtime

In the default example, the content of the configuration file is as follows:

//conf.m

conf = [
     'admin': [ //HTTP API listen address
         'ip': '0.0.0.0',
         'port': '1234'
     ],
     'tunnel': [ //tunnel server listen address
         'ip': '0.0.0.0',
         'port': '4321'
     ],
];

You can edit the conf.m file if these two addresses should be modified.

After the service starts, you will see that two listening addresses are output to the terminal, which means that the service is ready.

Example

Below we give an example, assuming we have the following network structure:

                  |---------------|                      |------------------|
    service1      |192.168.1.2    |        tunnel1       |192.168.1.3       |   service1
----------------> |8080    Menet  |--------------------->|4321      Menet   |-------------->192.168.1.3:80
                  |admin port:1234|                      |admin port:1234   |
                  |---------------|                      |------------------|

Briefly describe:

We have a real 80 service running on 192.168.1.3, and we expect to use the tunnel proxy to access this service from port 8080 of 192.168.1.2.

We can use the curl command to issue 5 HTTP requests to complete this topology deployment, and we will explain them one by one:

   $ curl -XPOST -d '{"name":"tunnel1", "dest":["192.168.1.3", "4321"]}' http://192.168.1.2:1234/tunnel

This request is sent to Menet of 192.168.1.2, the purpose is to let 1.2 establish a TCP connection to port 4321 (tunnel port) of 1.3, so that a tunnel (tunnel1) is established. *Note: This is just an example. In practice, you can also send a request to 1.3 to establish a TCP to the tunnel port of 1.2. *

   $ curl -XPOST -d '{"name":"service1", "key":"UHI@&s8sa*S", "type": "local", "addr":["0.0.0.0", "8080" ]}' http://192.168.1.2:1234/service

This request is used to tell the 1.2 Menet service to establish a local port 8080 for listening (type is local), and name the listening service service1, and the tunnel uses RC4 encryption, so key is the encryption key.

   $ curl -XPOST -d '{"name":"service1", "key":"UHI@&s8sa*S", "type": "remote", "addr":["192.168.1.3", "80" ]}' http://192.168.1.3:1234/service

This request is used to tell the 1.3 Menet service to establish a connection to the 192.168.1.3:80 port and forward it when receiving service1 data from the tunnel.

   $ curl -XPOST -d '{"tunnel": "tunnel1", "service": "service1", "type": "local"}' http://192.168.1.2:1234/bind

This request is to inform the Menet service of 1.2 to establish a mapping relationship between tunnel1 and service1.

   $ curl -XPOST -d '{"tunnel": "tunnel1", "service": "service1", "type": "remote"}' http://192.168.1.3:1234/bind

This request is to inform the 1.3 Menet service to establish a mapping relationship between tunnel1 and service1.

After performing these 5 steps, we can use curl to access 192.168.1.2:8080 to indirectly access the 192.168.1.3:80 service.

User space dynamic tracing and feedback control

Megabit — Fri, 03 Feb 2023 16:53:11 +0000

Preface

Based on the latest features of the open source C language library Melon, this article describes the use of the library for user space dynamic tracing, processing the tracing data, and feeding back the results to the program.

When it comes to dynamic tracing, your first impression may be bpf, dtrace, systemtap, etc., but the dynamic tracing introduced in this article does not depend on these. The functions provided in Melon are more inclined to allow the program to complete its own dynamic tracing in the user space, without relying on the kernel, nor on uprobe or usdt.

Regarding the Melon library, this is an open source C language library, which has the advantages of out-of-the-box, no third-party dependencies, simple installation and deployment, and complete documentation.

Github: https://github.com/Water-Melon/Melon

Implementation principle

The dynamic tracing of the Melon library is implemented by adding tracing points in the program.
In Melon, an application is divided into two layers (but both are run in the same process):

C code layer
Melang script code layer

The two code levels can run in the same thread, or in different threads, but they need to run in the same process.

In other words: *The tracing point data will be thrown at the C layer and passed to the specified script task, and then the script task will receive the data and process it. *

This seems to be no different from logging in one program and reading the log in another program for processing, so what are the benefits of doing so?

Advantage:

No need to parse the format of the log, you can directly get the corresponding type of data
Remote transmission or storage can be performed after script side processing (script library function guarantee)
Even under the same thread, the script task execution will not interrupt the C layer logic for a long time, and the script and C logic are automatically time-sharing scheduling
Using the feedback API on the script side, the calculation result can be sent back to the C layer, so that the execution logic of the C code layer can be changed according to the result, for example: service degradation

Example

Let's look at a very simple example:

#include <stdio.h>
#include "mln_log.h"
#include "mln_core.h"
#include "mln_trace.h"
#include "mln_conf.h"
#include "mln_event.h"

int timeout = 100;

static void timeout_handler(mln_event_t *ev, void *data)
{
    mln_trace("sir", "Hello", getpid(), 3.1);
    mln_event_timer_set(ev, timeout, NULL, timeout_handler);
}

static int recv_handler(mln_lang_ctx_t *ctx, mln_lang_val_t *val)
{
    timeout += val->data.i;
    return 0;
}

int main(int argc, char *argv[])
{
    mln_event_t *ev;
    struct mln_core_attr cattr;

    cattr.argc = argc;
    cattr.argv = argv;
    cattr.global_init = NULL;
    cattr.main_thread = NULL;
    cattr.master_process = NULL;
    cattr.worker_process = NULL;

    if (mln_core_init(&cattr) < 0) {
       fprintf(stderr, "Melon init failed.\n");
       return -1;
    }

    if ((ev = mln_event_new()) == NULL) {
        mln_log(error, "event new error\n");
        return -1;
    }

    if (mln_trace_init(ev, mln_trace_path()) < 0) {
        mln_log(error, "trace init error\n");
        return -1;
    }
    mln_trace_recv_handler_set(recv_handler);

    mln_event_timer_set(ev, 1000, NULL, timeout_handler);

    mln_event_dispatch(ev);

    return 0;
}

Briefly describe the program flow:

Initialization of the Melon library (mln_core_init)
Initialize the event object
Initialize the tracing script
Set the function used to process the data sent by the script layer
Set the timeout event
Event dispatching, timeout event will be triggered

In the timeout processing function timeout_handler, we use mln_trace to send three different types of data to the script task, and then continue to set the timeout event.

The timeout period is a global variable timeout, which is initially 100 milliseconds.

When the script layer sends data, here we agree that the script layer must send an integer, then in the receiving function recv_handler, we accumulate this value and timeout as the subsequent timeout period.

From this, it can be guessed that the amount of data delivered to the script layer per second in the program will be less and less.

The script layer code is given below:

sys = Import('sys');

Pipe('subscribe');
while (1) {
     ret = Pipe('recv');
     if (ret) {
         for (i = 0; i < sys. size(ret); ++i) {
             sys.print(ret[i]);
         }
         Pipe('send', 100);
     } fi
     sys.msleep(1000);
}
Pipe('unsubscribe');

A brief description of the logic of the script layer is to receive a batch of data from the C layer every second, and then output it to the terminal, and after the output, send an integer 100 to the C layer.

Let's take a look at the results of running the program:

...
[Hello, 72173, 3.100000, ]
[Hello, 72173, 3.100000, ]
[Hello, 72173, 3.100000, ]
...

You will see a lot of the above output, but if you run it yourself, you will find that the number of output lines per second will be less and less, which is consistent with our program logic.

Conclusion

As can be seen from this example, we can not only trace and process on the script side, but also use the processing results to feedback and control the C layer. And doing so has three advantages:

There is no need to add additional statistical variables and structures in the C layer
The C layer and the script layer are separated on the code management aspect
Both levels of code run in the same thread

In order to simplify the demonstration code, the above example does not show network communication and data storage at the script layer, but all these functions are supported by Melang scripts. Interested readers can refer to Melang official website.

Finally, readers interested in the Melon library can visit its Github repository for more details.

Thanks for reading!

Open Source C Library Melon: Doubly Linked List

Megabit — Fri, 03 Feb 2023 05:04:41 +0000

This article introduces the use of the double-linked list of the open source C library Melon.

Readers who are interested in the open source C library can visit its Github repo.

Introduction to Linked List

First briefly introduce what is a doubly linked list. You can refer to the following figure:

Simply put, a linked list is to connect nodes one by one through pointers. In the doubly linked list, each node not only records the pointer to the next node, but also records the pointer to the previous node.

Advantages of the doubly linked list:

The time complexity of node insertion and deletion operations is O(1), so it is very suitable for frequent insertion and deletion scenarios.

Use of doubly linked list

Let’s define a type first:

typedef struct test_s {
 int val;
} test_t;

In the subsequent introduction, what we have to do is to use Melon’s linked list component to transform this structure and build it into a doubly linked list.

In Melon, there are two implementations of doubly linked lists. Both implementations have their own advantages, so readers can choose to use them according to the needs.

The first implementation

#include <stdio.h>
#include <stdlib.h>
#include "mln_defs.h"

typedef struct test_s {
    int val;
    struct test_s *prev;
    struct test_s *next;
} test_t;

MLN_CHAIN_FUNC_DECLARE(test, test_t, static inline void, );
MLN_CHAIN_FUNC_DEFINE(test, test_t, static inline void, prev, next);

int main(void)
{
  int i;
  test_t *head = NULL, *tail = NULL, *t;

  for (i = 0; i < 10; ++i) {
    t = (test_t *)malloc(sizeof(test_t));
    if (t == NULL) {
        fprintf(stderr, "malloc failed.\n");
        return -1;
    }
    t->val = i;
    t->prev = t->next = NULL;
    test_chain_add(&head, &tail, t);
  }

  for (t = head; t != NULL; t = t->next) {
    printf("%d\n", t->val);
  }
  return 0;
}

In this code, the main is very simple. It uses a for loop to malloc 10 test_t structures, and then fill their val with the integer i. Then use the test_chain_add to link these nodes into a list. Finally, uses a for loop to traverse the nodes and print the value of val.

Here comes the question, where did test_chain_add come from?

We see that there are two MLN_CHAIN_FUNC_xxx macros before main, these two macros are defined in the header file mln_defs.h in Melon, and are used to define and declare the insertion and deletion functions of the linked list.

MLN_CHAIN_FUNC_DECLARE

The first parameter: function name prefix
The second parameter: custom structure type
The third parameter: function return value and function type
The fourth parameter: function parameter restriction attribute (not used in this example)

MLN_CHAIN_FUNC_DEFINE

The first parameter: function name prefix
The second parameter: custom structure type
The third parameter: function return value and function type
The fourth parameter: the name of the pointer member used to access the previous node in the custom structure
The fifth parameter: the name of the pointer member used to access the next node in the custom structure

These two macros will define and declare two functions named: test_chain_add and test_chain_del. The prototypes of these two functions are similar to the following:

void (*chain_op)(type **head, type **tail, type *node);

The first parameter is the address of the first pointer of the doubly linked list, the second parameter is the address of the last pointer of the doubly linked list, and the third parameter is the node pointer to be operated.

Features

The advantage of this implementation is that the insertion and deletion functions can be defined as inline, and the names of the forward and backward pointers can be changed at will. And when traversing the linked list, you only need to access these two pointers of the custom node.

The disadvantage is that users need to know the member names of the forward and backward pointers in the custom structure. And if many doubly linked lists are defined, many insertion and deletion functions need to be defined either, which will increase the size of the executable program.

The second implementation

#include "mln_list.h"
#include "mln_defs.h"
#include <stdlib.h>

typedef struct {
    int        val;
    mln_list_t node;
} test_t;

int main(void)
{
    int i;
    test_t *t;
    mln_list_t sentinel = mln_list_null();

    for (i = 0; i < 3; ++i) {
        t = (test_t *)calloc(1, sizeof(*t));
        if (t == NULL)
            return -1;
        mln_list_add(&sentinel, &t->node);
        t->val = i;
    }
    for (t = mln_container_of(mln_list_head(&sentinel), test_t, node); \
         t != NULL; \
         t = mln_container_of(mln_list_next(&t->node), test_t, node))
    {
        printf("%d\n", t->val);
    }
    return 0;
}

The behavior of the main function is exactly the same as the code of the first implementation. The difference lies in the following aspects:

Include mln_list.h
Instead of adding the forward and backward pointers to test_t, a member (node) of a fixed type (mln_list_t) is added.
The head and tail pointers of the doubly linked list are changed to a mln_list_t type variable named sentinel.
The linked list insertion function is no longer a custom prefix function, but a function with a fixed name called mln_list_add.
To access the first node of the linked list use the macro mln_list_head
To obtain the pointer to the host structure where the linked list node is located (ie test_t in this example), you need to use the macro mln_container_of defined in mln_defs.h.

Features

Advantages: Users do not need to care about the implementation details of the linked list, they only need to add mln_list_t type members into the custom type. And the insertion and deletion operations of the function are fixed names, respectively: mln_list_add and mln_list_remove.

Disadvantage: Access to the host node (test_t) to which the linked list node belongs requires the help of the macro mln_container_of, which does not seem as intuitive as the first implementation. And the linked list operation functions are all non-inline, so the overhead of frequent calls will be higher than the first implementation.

In fact, readers may notice that the second implementation (mln_list.c) is implemented by the first doubly linked list implementation.

End

Thanks for reading.

Readers can visit the Melon library for more details.

Melon is an open source C language library that is free of dependencies and out of the box, and complete documentation.

Simple HTTP server implemented in C: new trace technology

Megabit — Wed, 11 Jan 2023 10:46:53 +0000

This article mainly introduces a tracing technology in the Melon library, and use of an HTTP server as an example to illustrate.

Regarding the Melon library, this is an open source C language library. This library does not depend on any libraries, so it is easy to install and can be used out of the box. Github repository: repo.

Let’s go directly to the code

// http.c
#include <stdio.h>
#include <stdlib.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <errno.h>
#include <arpa/inet.h>
#include <sys/time.h>
#include "mln_core.h"
#include "mln_log.h"
#include "mln_http.h"
#include "mln_iothread.h"
#include "mln_trace.h"


static void *mln_thread_entry(void *args);
static void mln_accept(mln_event_t *ev, int fd, void *data);
static int mln_http_recv_body_handler(mln_http_t *http, mln_chain_t **in, mln_chain_t **nil);
static void mln_recv(mln_event_t *ev, int fd, void *data);
static void mln_quit(mln_event_t *ev, int fd, void *data);
static void mln_send(mln_event_t *ev, int fd, void *data);
static int mln_http_send_body_handler(mln_http_t *http, mln_chain_t **body_head, mln_chain_t **body_tail);

static void mln_accept(mln_event_t *ev, int fd, void *data)
{
    mln_tcp_conn_t *connection;
    mln_http_t *http;
    int connfd;
    socklen_t len;
    struct sockaddr_in addr;

    while (1) {
        memset(&addr, 0, sizeof(addr));
        len = sizeof(addr);
        connfd = accept(fd, (struct sockaddr *)&addr, &len);
        if (connfd < 0) {
            if (errno == EAGAIN) break;
            if (errno == EINTR) continue;
            perror("accept");
            exit(1);
        }

        connection = (mln_tcp_conn_t *)malloc(sizeof(mln_tcp_conn_t));
        if (connection == NULL) {
            fprintf(stderr, "3No memory.\n");
            close(connfd);
            continue;
        }
        if (mln_tcp_conn_init(connection, connfd) < 0) {
            fprintf(stderr, "4No memory.\n");
            close(connfd);
            free(connection);
            continue;
        }

        http = mln_http_init(connection, NULL, mln_http_recv_body_handler);
        if (http == NULL) {
            fprintf(stderr, "5No memory.\n");
            mln_tcp_conn_destroy(connection);
            free(connection);
            close(connfd);
            continue;
        }

        if (mln_event_fd_set(ev, \
                             connfd, \
                             M_EV_RECV|M_EV_NONBLOCK, \
                             M_EV_UNLIMITED, \
                             http, \
                             mln_recv) < 0)
        {
            fprintf(stderr, "6No memory.\n");
            mln_http_destroy(http);
            mln_tcp_conn_destroy(connection);
            free(connection);
            close(connfd);
            continue;
        }
    }
}

static void mln_quit(mln_event_t *ev, int fd, void *data)
{
    mln_http_t *http = (mln_http_t *)data;
    mln_tcp_conn_t *connection = mln_http_get_connection(http);

    mln_event_fd_set(ev, fd, M_EV_CLR, M_EV_UNLIMITED, NULL, NULL);
    mln_http_destroy(http);
    mln_tcp_conn_destroy(connection);
    free(connection);
    close(fd);
}

static void mln_recv(mln_event_t *ev, int fd, void *data)
{
    mln_http_t *http = (mln_http_t *)data;
    mln_tcp_conn_t *connection = mln_http_get_connection(http);
    int ret, rc;
    mln_chain_t *c;

    while (1) {
        ret = mln_tcp_conn_recv(connection, M_C_TYPE_MEMORY);
        if (ret == M_C_FINISH) {
            continue;
        } else if (ret == M_C_NOTYET) {
            c = mln_tcp_conn_remove(connection, M_C_RECV);
            if (c != NULL) {
                rc = mln_http_parse(http, &c);
                if (c != NULL) {
                    mln_tcp_conn_append_chain(connection, c, NULL, M_C_RECV);
                }
                if (rc == M_HTTP_RET_OK) {
                    return;
                } else if (rc == M_HTTP_RET_DONE) {
                    mln_send(ev, fd, data);
                } else {
                    fprintf(stderr, "Http parse error. error_code:%u\n", mln_http_get_error(http));
                    mln_quit(ev, fd, data);
                    return;
                }
            }
            break;
        } else if (ret == M_C_CLOSED) {
            c = mln_tcp_conn_remove(connection, M_C_RECV);
            if (c != NULL) {
                rc = mln_http_parse(http, &c);
                if (c != NULL) {
                    mln_tcp_conn_append_chain(connection, c, NULL, M_C_RECV);
                }
                if (rc == M_HTTP_RET_ERROR) {
                    fprintf(stderr, "Http parse error. error_code:%u\n", mln_http_get_error(http));
                }
            }
            mln_quit(ev, fd, data);
            return;
        } else if (ret == M_C_ERROR) {
            mln_quit(ev, fd, data);
            return;
        }
    }
}

static int mln_http_recv_body_handler(mln_http_t *http, mln_chain_t **in, mln_chain_t **nil)
{
    mln_u32_t method = mln_http_get_method(http);
    mln_trace("s", method == M_HTTP_GET? "GET": "OTHERS");
    if (method == M_HTTP_GET)
        return M_HTTP_RET_DONE;
    mln_http_set_error(http, M_HTTP_NOT_IMPLEMENTED);
    return M_HTTP_RET_ERROR;
}

static void mln_send(mln_event_t *ev, int fd, void *data)
{
    mln_http_t *http = (mln_http_t *)data;
    mln_tcp_conn_t *connection = mln_http_get_connection(http);
    mln_chain_t *c = mln_tcp_conn_get_head(connection, M_C_SEND);
    int ret;

    if (c == NULL) {
        mln_http_reset(http);
        mln_http_set_status(http, M_HTTP_OK);
        mln_http_set_version(http, M_HTTP_VERSION_1_0);
        mln_http_set_type(http, M_HTTP_RESPONSE);
        mln_http_set_handler(http, mln_http_send_body_handler);
        mln_chain_t *body_head = NULL, *body_tail = NULL;
        if (mln_http_generate(http, &body_head, &body_tail) == M_HTTP_RET_ERROR) {
            fprintf(stderr, "mln_http_generate() failed. %u\n", mln_http_get_error(http));
            mln_quit(ev, fd, data);
            return;
        }
        mln_tcp_conn_append_chain(connection, body_head, body_tail, M_C_SEND);
    }

    while ((c = mln_tcp_conn_get_head(connection, M_C_SEND)) != NULL) {
        ret = mln_tcp_conn_send(connection);
        if (ret == M_C_FINISH) {
            mln_quit(ev, fd, data);
            break;
        } else if (ret == M_C_NOTYET) {
            mln_chain_pool_release_all(mln_tcp_conn_remove(connection, M_C_SENT));
            mln_event_fd_set(ev, fd, M_EV_SEND|M_EV_APPEND|M_EV_NONBLOCK, M_EV_UNLIMITED, data, mln_send);
            return;
        } else if (ret == M_C_ERROR) {
            mln_quit(ev, fd, data);
            return;
        } else {
            fprintf(stderr, "Shouldn't be here.\n");
            abort();
        }
    }
}

static int mln_http_send_body_handler(mln_http_t *http, mln_chain_t **body_head, mln_chain_t **body_tail)
{
    mln_u8ptr_t buf;
    mln_alloc_t *pool = mln_http_get_pool(http);
    mln_string_t cttype_key = mln_string("Content-Type");
    mln_string_t cttype_val = mln_string("text/html");

    buf = mln_alloc_m(pool, 5);
    if (buf == NULL) {
        mln_http_set_error(http, M_HTTP_INTERNAL_SERVER_ERROR);
        return M_HTTP_RET_ERROR;
    }
    memcpy(buf, "hello", 5);

    if (mln_http_set_field(http, &cttype_key, &cttype_val) == M_HTTP_RET_ERROR) {
        mln_http_set_error(http, M_HTTP_INTERNAL_SERVER_ERROR);
        return M_HTTP_RET_ERROR;
    }

    mln_string_t ctlen_key = mln_string("Content-Length");
    mln_string_t ctlen_val = mln_string("5");
    if (mln_http_set_field(http, &ctlen_key, &ctlen_val) == M_HTTP_RET_ERROR) {
        mln_http_set_error(http, M_HTTP_INTERNAL_SERVER_ERROR);
        return M_HTTP_RET_ERROR;
    }

    mln_chain_t *c = mln_chain_new(pool);
    if (c == NULL) {
        mln_http_set_error(http, M_HTTP_INTERNAL_SERVER_ERROR);
        return M_HTTP_RET_ERROR;
    }
    mln_buf_t *b = mln_buf_new(pool);
    if (b == NULL) {
        mln_chain_pool_release(c);
        mln_http_set_error(http, M_HTTP_INTERNAL_SERVER_ERROR);
        return M_HTTP_RET_ERROR;
    }
    c->buf = b;
    b->left_pos = b->pos = b->start = buf;
    b->last = b->end = buf + 5;
    b->in_memory = 1;
    b->last_buf = 1;
    b->last_in_chain = 1;

    if (*body_head == NULL) {
        *body_head = *body_tail = c;
    } else {
        (*body_tail)->next = c;
        *body_tail = c;
    }

    return M_HTTP_RET_DONE;
}

static void *mln_thread_entry(void *args)
{
    mln_event_dispatch((mln_event_t *)args);
    return NULL;
}

int main(int argc, char *argv[])
{
    mln_event_t *ev;
    mln_iothread_t t;
    struct sockaddr_in addr;
    int val = 1, listenfd;
    int nthread = 1;
    mln_u16_t port = 1234;
    mln_s8_t ip[] = "0.0.0.0";
    struct mln_core_attr cattr;
    struct mln_iothread_attr tattr;

    /* init library */
    cattr.argc = 0;
    cattr.argv = NULL;
    cattr.global_init = NULL;
    cattr.master_process = NULL;
    cattr.worker_process = NULL;
    mln_core_init(&cattr);

    /* create event instance */
    if ((ev = mln_event_new()) == NULL) {
        mln_log(error, "event new error.\n");
        return -1;
    }

    /* set listen fd */
    listenfd = socket(AF_INET, SOCK_STREAM, 0);
    if (listenfd < 0) {
        mln_log(error, "listen socket error\n");
        return -1;
    }
    if (setsockopt(listenfd, SOL_SOCKET, SO_REUSEADDR, &val, sizeof(val)) < 0) {
        mln_log(error, "setsockopt error\n");
        return -1;
    }
    addr.sin_family = AF_INET;
    addr.sin_port = htons(port);
    addr.sin_addr.s_addr = inet_addr(ip);
    if (bind(listenfd, (struct sockaddr *)&addr, sizeof(addr)) < 0) {
        mln_log(error, "bind error\n");
        return -1;
    }
    if (listen(listenfd, 511) < 0) {
        mln_log(error, "listen error\n");
        return -1;
    }
    if (mln_event_fd_set(ev, \
                         listenfd, \
                         M_EV_RECV|M_EV_NONBLOCK, \
                         M_EV_UNLIMITED, \
                         NULL, \
                         mln_accept) < 0)
    {
        mln_log(error, "listen sock set event error\n");
        return -1;
    }

    /* init trace */
    if (mln_trace_init(ev, mln_trace_path()) < 0) {
        mln_log(error, "trace init failed.\n");
        return -1;
    }

    /* create threads */
    tattr.nthread = nthread;
    tattr.entry = (mln_iothread_entry_t)mln_thread_entry;
    tattr.args = ev;
    tattr.handler = NULL;
    if (mln_iothread_init(&t, &tattr) < 0) {
        mln_log(error, "iothread init failed\n");
        return -1;
    }

    mln_event_dispatch(ev);
    return 0;
}

In these 330 lines of code, we implemented a simple HTTP1.1 server by the Melon library. The behavior of this server is that it will reply a Hello when it receives a GET request.

In this example, we started two threads (main thread + 1 worker thread), and both threads will operate on the same event object (ev). The socket of our HTTP server depends on this event object ev. In other words, event objects can be dispatched across threads, and the event model helps us shield the problem of thundering herd.

In fact, only single thread can be used here, just to demonstrate and prove to readers that the event model supports multithreading.

Then let’s discuss tracing in Melon. We can see that the mln_trace_init function is called in main to initialize the tracing task. The tracing data in this example is processed by a script task. Although the script task also rely on the event model, but it does not prevent socket events from being fired and processed. Simply speaking, all file descriptor and timer events, and script execution are processed in a time-sharing manner. Infinite loop in the script will not blocked event processing. In addition, if trace configuration is not existent or inactivated, the program will not emit trace information (refer to mln_trace called in mln_accept).

The trace script uses the built-in scripting language Melang in the Melon library. Let’s take a look at the Melang script to this example:

sys = Import('sys');

Pipe('subscribe');
beg = sys.time();
sum = [];
while (1) {
    ret = Pipe('recv');
    if (!ret) {
       now = sys.time();
       if (now - beg >= 3) {
           sys.print(sum);
           beg = now;
       } fi
       sys.msleep(10);
    } else {
       n = sys.size(ret);
       for (i = 0; i < n; ++i) {
           if (!(sys.has(sum, ret[i][0])))
               sum[ret[i][0]] = 0;
           fi
           sum[ret[i][0]]++;
       }
    }
}
Pipe('unsubscribe');

In this code, we will classify and count all requests according to its method and output the statistical results every 3 seconds.

Next, we need to modify the configuration file of Melon, and remove the comments from the trace_mode configuration item.

Note: If you are starting the http server in the Melon source directory, you should modify the Melon/trace/trace.m file. Otherwise, the /usr/local/lib/melang/trace/trace.m file should be modified, because the interpreter will first check whether there is such a file in the current path.

Then let’s run the program

#tty1
$ ./http

#tty2
$ ab -c 100 -n 100 http://127.1:1234/

Then we can see that tty1 will output

...
[]
[]
[100, ]
...

And as ab is called multiple times, the value in the array will also increase accordingly.

Finally, summarize the benefits of doing this.

With this trace mode, you can turn on and off the trace as needed, and you can even call the trace module interfaces in the program to dynamically turn on and off the trace mode.
The Melang script itself also supports communication with MySQL, so the trace data can be summarized and stored in DB from the script layer. So that there is no need to add a large number of statistical variables in the C program.

20-line of code HTTP server

Megabit — Mon, 03 Oct 2022 07:38:06 +0000

Not Javascript, not Python, not Go, not PHP, no encapsulation.

Today I'm going to introduce a new coroutine language named Melang.

Melang is a coroutine concurrent scripting language. It is an interpreted, not compiled language.

In Melang, every script task is a coroutine. And multiple script tasks can be processed in the same thread or multithread at the same time.

This is very similar to Go's coroutines, but the coroutines in Melang do not need to actively give up CPU execution rights or call specific functions to indirectly give up CPU execution rights. Instead, the task execution is automatically switched after the interpreter performs certain operations. This also eliminates the need for developers to consider switching timing.

In Melang, all script code logic is synchronized. But it's actually done asynchronously by the interpreter. This help many developers who are not familiar with asynchronous programming to get started.

Here is a simple HTTP server example:

/* filename: server.m */
net = import('net');
mq = import('mq');

listenfd = net.tcp_listen('127.0.0.1', '80');
for (i = 0; i < 4; ++i)
    eval('worker.mln', i);
while (true) {
    fd = net.tcp_accept(listenfd);
    mq.send('test', fd);
}

/* filename: worker.m */
sys = import('sys');
net = import('net');
mq = import('mq');

while (1) {
    fd = mq.recv('test');
    ret = net.tcp_recv(fd);
    if (ret) {
        net.tcp_send(fd, "HTTP/1.1 200 OK\r\nContent-Length: 1\r\n\r\na\r\n\r\n");
    } fi
    net.tcp_close(fd);
}

The main coroutine (server.m) creates 4 worker coroutines (worker.m), and then enters an infinite loop waiting for the establishment of the client TCP.

The worker coroutine (worker.m) is in an infinite loop, receiving the established TCP socket from the main coroutine, then receiving the request from the TCP, then sending the HTTP response, and finally closing the socket.

Next, we can start the script from the command line with the following command:

$ melang server.m

Once started, it can be accessed using tools like curl or ab.

Introduction To A New Coroutine Language

Megabit — Sun, 02 Oct 2022 08:05:31 +0000

Today's open source project is named Melang.

This is a "new" scripting language that has been around for 6 years.

Now, let me take you into the Melang world.

What is Melang

Melang is a coroutine concurrent scripting language. It is an interpreted, not compiled language.

In Melang, every script task is a coroutine. And multiple script tasks can be processed in the same thread or multithread at the same time.

At the same time, in order to allow developers to start faster, the syntax of the language is very similar to the C, so developers who are familiar with C will easily start developing.

Then let's say Hello, world together!

//hello.m
sys = import('sys');

sys.print("Hello, World!")

execute shell command:

$ melang hello.m

The output is:

Hello, World!

Coroutine Concurrency and Coroutine Pool

In Melang, each script task is isolated from each other. However, sometimes we also need to let different tasks communicate with each other to cooperate, so the message queue module is provided to make this idea possible.

However, based on this mechanism, we can easily derive a new pool model: coroutine pool.

That is, a master coroutine can be used to allocate resources, while a group of worker coroutines can receive resources through the message queue and process them individually. This is very similar to the traditional thread pool model.

Let's take a look at the simplest HTTP server implemented using a coroutine pool:

/* filename: server.m */
net = import('net');
mq = import('mq');

listenfd = net.tcp_listen('127.0.0.1', '80');
for (i = 0; i < 4; ++i) {
    eval('worker.mln', i);
}
while (1) {
    fd = net.tcp_accept(listenfd);
    mq.send('test', fd);
}

/* filename: worker.m */
sys = import('sys');
net = import('net');
mq = import('mq');

sys.print(EVAL_DATA);
while (1) {
    fd = mq.recv('test');
    ret = net.tcp_recv(fd);
    if (ret) {
        net.tcp_send(fd, "HTTP/1.1 200 OK\r\nContent-Length: 1\r\n\r\na\r\n\r\n");
    }fi
    net.tcp_close(fd);
}

The two pieces of code are: the main coroutine (server.m) and the worker coroutine (worker.m).

The main coroutine (server.m) creates 4 worker coroutines (worker.m), and then enters an infinite loop waiting for the establishment of the client TCP.

Next, we can start the script from the command line with the following command:

$ melang server.m -t=2

Here, the meaning of -t=2 is to let the interpreter start two threads to process these 5 coroutines (1-server + 4-worker).

Once started, it can be accessed using tools like curl or ab.

The End

In summary, Melang is a:

Scripting language
Preemptive task scheduling language
Coroutine Concurrency Language
Languages where synchronous code is executed asynchronously

In addition, Melang also incorporates features such as reactive programming, operator overloading, reflection, injection, and more.

Thanks for reading!

Preemptive scheduling coroutine programming language

Megabit — Sun, 10 Jan 2021 10:52:00 +0000

The concept of coroutine must be familiar to everyone now. After all, not only LUA, Go or Kotlin but also C++, they are all support coroutines.
But different languages have different implementations. Such as LUA, coroutines will be scheduled when someone called yield() or resume().
A new language introduced to you today is a scripting language of coroutine preemptive scheduling based on time slices. That is, each coroutine does not need to explicitly call certain functions to give up control, but according to the weight of execution time to control whether to be suspended or run.

Its name is Melang

Official site: https://melang.org/
Repo: https://github.com/Water-Melon/Melang

We will discuss the following:

features
usage scenarios

Features

For a language, it must have flow control, array, dict, structure or class, etc. These in Melang are no different from other languages. So we will focus on the following:

Coroutine preemptive scheduling

In Melang, each script task is an independent coroutine, and coroutines' execution environments are isolated. But they can communicate with each other by the message queue functions.
Let's look at two examples:

running multi-tasks by melang execution file

There are two script files:

 //a.mln
 while (true) {
   @mln_print('a');
 }

 //b.mln
 while (true) {
   @mln_print('b');
 }

Then I execute the following command:

 $ melang a.mln b.mln

After execution, you will see the screen print a and b alternately, as follows:

 ...
 b
 b
 b
 a
 a
 a
 a
 a
 ...

And using top command will find out that only one process (and also only one thread) is performing these two tasks.
Melang scheduling does not occur in the function mln_print, but will try to switch during the processing of each statement.

start up multi-tasks in a Melang task

 //launcher.mln
 @mln_eval('a.mln');
 @mln_eval('b.mln');

We create a new script file named launcher.mln to start up a.mln and b.mln by calling function mln_eval. And then execute:

 $ melang launcher.mln

The output is the same as previous example.

Reflection

Reflection means that the class to be operated is known at runtime, and the complete structure of the class can be obtained at runtime and the corresponding method can be called. -- quoted from a netizen

In Melang, the concept of class is called Set, but it is more similar to the structure in C, but also supports function definitions.
Let's look at a simple example:

Human {
  name;
  age;
  @init (name, age) {
    this.name = name;
    this.age = age;
  }
}

Here we define a Set named Human, which has two member variables: name and age, and a method init.
The general instantiation code is as follows:

me = $Human;

This instantiates a Human object named me.
However, we can also instantiate it like this:

str = 'Human';
me = $str;

The result is the same.

Then we can call the method of the object:

me.init('Tom', 18);

In Melang, not only Set can be obtained by a string, but also functions and object methods can be used in this way.

Examples for function:

@foo ()
{
  @mln_print('foo');
}
a = 'foo';
a();

Here we define a function, and we call it indirectly by a string variable.

Example for object method:

Human {
  name;
  age;
  action;
  @init (name, age) {
    this.name = name;
    this.age = age;
  }
}

me = $Human;
me.action = 'init';
me.action('Tom', 18);
@mln_print(me.name);

As you can see in this example, we call the method of an object by a string member variable.

Injection

For an instantiated object, we can still add (or inject) some members and methods to it, for example:

Human {
}

someone = $Human;
someone.age = 20;
@mln_print(someone.age);

The output is 20. The member variable age was successfully added to the object someone.

Note: The injection is defined for the object, not Set. Therefore, if you instantiate Human again, there is still no age.

The above example is the simplest injection. Melang also provides function set and get similar to other scripts.

Let's look at an example of method injection:

Human {
  age;
}

@printAge()
{
  @mln_print(this.age);
}

someone = $Human;
someone.age = 20;
someone.print = printAge;
someone.print();

This example will still output 20, but this time it is output by mln_print called in the injected method printAge.

Reactive programming

As we can see in web front-end programming, a variable will be monitored by a set of functions. When the value changes, the callback function will be called. (Note that, in Melang, even if the new value is the same as the old one, it also will be called)

Example:

@eventHandler(newdata, userData)
{
  @mln_print(newdata);
  @mln_print(userData);
}
n = 0;
@mln_watch(n, eventHandler, 'this is an useData');
for (; n < 5; ++n)
  ;

Here, variable n will be tracked by calling the function mln_watch. When the value of n changed, the eventHandler function would be called.

We will see the following output:

1
this is an useData
2
this is an useData
3
this is an useData
4
this is an useData
5
this is an useData

Synchronous code, executed asynchronously

In Melang, code is written in a synchronized form. The advantage is logical coherence.

In traditional languages, synchronous writing also means synchronous processing, so the performance is low. But in Melang, the interpreter will execute completely asynchronously.

Example:

//a.mln
while (true) {
  @mln_msgQueueSend('queue', 'a');
  ret = @mln_msgQueueRecv('queue');
  @mln_print(ret);
}

//b.mln
while (true) {
  ret = @mln_msgQueueRecv('queue');
  @mln_print(ret);
  @mln_msgQueueSend('queue', 'b');
}

These two scripts use the message queue functions to send and receive messages from each other and print them out. For each task, its logic is sequential. For example, a.mln must send a message first, then receive the message, and then print. If b.mln does not send a message to a.mln, then a.mln will be suspended while receiving the message.

The above is just a simple example, and it is the same when it comes to network I/O. When data arrives on the socket, the interpreter will receive it, then return to the script layer, and change the script task from the suspended state to the running state.

Usage Scenarios

The current usage scenario of Melang is mainly embedded in other applications, which is basically the same as LUA. The Melang project on Github is a launcher. The launcher will call the script-related functions in the core library for initialization and execution. And of course, the core library is also open source.

Melang's core library provides many interfaces such as algorithms, data structures, event mechanisms, multi-process and multi-threaded frameworks. Based on this library, not only can obtain melang script, but also reduce development costs.

In addition, if an application uses the event processing interface provided by the core library, not only events can be processed, but scripts can also be executed in the same thread.

Melang provides a basic function library. Besides string, file system and network IO operations, it also provides functions for accessing MySQL8 and a series of functions such as encryption, decryption and communication.

Although the core library has been updated and upgraded for several years, the language is still in its infancy. And we still need to make a lot of efforts on it. Looking forward to your attention to this project. Thanks for reading.