DEV Community: Wissance

Beyond fmt: Building a Flexible Text Builder in Go with Wissance/stringFormatter

Ushakov Michael — Tue, 10 Mar 2026 06:18:13 +0000

Introduction

Go is renowned for its performance and simplicity, especially in system programming and web services. Its standard fmt package is a workhorse for string formatting. However, when building complex text output—like dynamic configuration strings, detailed logs, or formatted reports—developers often find themselves writing repetitive code or wrestling with fmt.Sprintf's verb-based syntax.

This article explores the Wissance/stringFormatter project, an open-source Go module that reimagines text building. It introduces a familiar, template-based approach (think C# or Python) that is not only syntactically convenient but, according to its benchmarks, also outperforms the standard fmt package in several scenarios. We will dive into its core features, from positional and named argument formatting to advanced data type handling, and see how it positions itself as a true "flexible text builder."

Core Features: Template Syntax and Performance

at its heart, stringFormatter provides a high-performance alternative to fmt for building strings from templates. Its main appeal is the use of familiar placeholder syntax {0}, {name}, which many developers find more readable for complex templates.

Flexible Formatting Methods

The library offers two primary methods for text construction, catering to different use cases:

Positional Formatting with Format

This function uses indexed placeholders like {0}, {1}. It's straightforward for scenarios where the argument order is clear.

result := stringFormatter.Format("Hello {0}, welcome to {1}!", "User", "GoApp")

// Output: "Hello User, welcome to GoApp!"

Named Formatting with FormatComplex: This method shines when building strings with many parameters or when the template needs to be self-documenting. By passing a map[string]any, you can use descriptive names like {user} or {port}.

params := map[string]any{"user": "admin", "port": 8080, "ssl": true}
result := stringFormatter.FormatComplex("Config: user={user}, port={port}, ssl={ssl}", params)

// Output: "Config: user=admin, port=8080, ssl=true"

This approach makes templates significantly easier to read and maintain.

Advanced Argument Formatting

What elevates stringFormatter from a simple templating tool to a "flexible text builder" is its support for advanced formatting directives directly within the placeholders. You can control the output representation of various data types:

Numeric Formatting: It supports binary (B), hexadecimal (X), and octal (o) representations, complete with padding.

stringFormatter.Format("{0:B8} in hex is {0:X2}", 15)

// Output: "00001111 in hex is 0f"

Floating-Point Precision: Control decimal places or use scientific notation:

stringFormatter.Format("Value: {0:F4} or {0:E2}", 10.456789)

// Output: "Value: 10.4568 or 1.05e+01"

Slice Formatting: Join slices with a custom separator effortlessly using the L directive.

ids := []int{1, 2, 3, 4}
stringFormatter.Format("IDs: {0:L-}", ids)

// Output: "IDs: 1-2-3-4"

Code Style Conversion: Dynamically convert between naming conventions like snake_case, camelCase, and kebab-case using the c directive.

stringFormatter.Format("Converted: {0:c:snake}", "MyVariable")

// Output: "Converted: my_variable"

Beyond Formatting: Utility Functions
The project also packages other useful text utilities, reinforcing its role as a comprehensive text manipulation toolkit:

MapToString: Converts a map into a formatted string, allowing you to define the pattern for each key-value pair.

SliceToString / SliceSameTypeToString: Provides high-performance slice-to-string conversion, which benchmarks show as significantly faster than manual loops using fmt.

Conclusion

The Wissance/stringFormatter project offers a compelling alternative for Go developers who frequently engage in complex string building. By combining a clean, template-based syntax with advanced formatting options and a focus on performance, it successfully creates a more flexible and expressive text builder than what's readily available in the standard library.

Whether you need to generate dynamic configuration files, create human-readable reports, or simply want a faster way to format slices and maps, stringFormatter provides a robust, well-tested, and high-performance solution. It demonstrates that sometimes, looking beyond the standard library's tools can lead to cleaner code and better performance.

Create REST automatically in .Net

Ushakov Michael — Tue, 02 Sep 2025 18:25:08 +0000

What is it and what is it for

Now it is possible to significantly simplify WebAPI development with the NET platform (>6). During the last years I was developing my lib (Give us a STAR please if you find our solution interesting or useful) that allows reducing the amount of code to be written during application development. Among other template processors, this library has the following differences:

Allows the use of any technology (framework) to work with persistent data (i.e., Entity Framework, Dapper, or whatever else) due to separated Core interfaces and one of the built-in implementations; controllers use the IModelManager interface and do not depend on specific technology at all;
Fully Swagger-friendly, and we could display any set of Swagger parameters automatically;
Contains default implementation of Create, Update, and Delete methods with Entity Framework;
Contains implementation of Bulk Create, Bulk Update and Bulk Delete methods for Entity Framework implementation;
Contains an extension that allows you to create any type of controller: ReadOnly (2 GET methods), FullCRUD, or even Bulk with EntityFramework in just one line of code;
Contains implementation and examples how to use Manager classes with GRPC services therefore it is easy to switch from REST to GRPC or to have them both in same time with just one Manager class;

Generate `REST` Controllers in one line

The full example could be find here but very briefly we have the following method that works with ServiceCollection:

private void ConfigureWebApi(IServiceCollection services)
{
    services.AddSwaggerGen();
    ServiceProvider provider = services.BuildServiceProvider();
    Assembly stationControllerAssembly = services.AddSimplifiedAutoController<StationEntity, Guid, EmptyAdditionalFilters>(
                provider.GetRequiredService<ModelContext>(), "Station",
                ControllerType.FullCrud, null, provider.GetRequiredService<ILoggerFactory>());
    Assembly measureUnitControllerAssembly = services.AddSimplifiedAutoController<MeasureUnitEntity, Guid, EmptyAdditionalFilters>(
                provider.GetRequiredService<ModelContext>(), "MeasureUnit",
                ControllerType.ReadOnly, null, provider.GetRequiredService<ILoggerFactory>());
    Assembly sensorControllerAssembly = services.AddSimplifiedAutoController<SensorEntity, Guid, EmptyAdditionalFilters>(
                provider.GetRequiredService<ModelContext>(), "Sensor",
                ControllerType.FullCrud, null, provider.GetRequiredService<ILoggerFactory>());
    Assembly measurementControllerAssembly = services.AddSimplifiedAutoController<MeasurementEntity, Guid, EmptyAdditionalFilters>(
                provider.GetRequiredService<ModelContext>(), "Measurement",
                ControllerType.Bulk, null, provider.GetRequiredService<ILoggerFactory>());

     services.AddControllers().AddApplicationPart(sensorControllerAssembly).AddControllersAsServices();
     services.AddControllers().AddApplicationPart(stationControllerAssembly).AddControllersAsServices();
     services.AddControllers().AddApplicationPart(measureUnitControllerAssembly).AddControllersAsServices();
     services.AddControllers().AddApplicationPart(measurementControllerAssembly).AddControllersAsServices();
}

for the visual indentation process of getting Assembly and load controllers from it is separated on 2 lines.

To implement the following approach, we have to pass:

DbContext (Entity Framework), passing generic DbContext allows you to use any database (Postgres, MySql, SqlServer, …)
Controller name without suffix Controller
One of enum value ReadOnly|FullCrud|Bullk
Filter function (could be null)
ILoggerFactory for making logs in the controllers There is only one requirement — every entity class must implement IModelIdentifiable<T> with property public T Id {get;set;}

What could be done

One-line controllers add simplified logic, but here we lose some control, i.e., we don’t have DTO objects even though Generic IModelManager does. If you would like to add custom methods to your controller, you should manually create a controller class (this is also very simple with ~10 lines of code; see i.e.). Authentication and authorization could be simply added via attributes, but in one-line controllers we don’t have yet (wait for the new releases).

Conclusion

This lib was developed by the author and his LLC. We also offer our services in quality enterprise software development; please don’t forget to GIVE us a STAR on GitHub.

Override Go app configuration with Environment variable

Ushakov Michael — Thu, 12 Dec 2024 21:11:17 +0000

How to make containerized applications more flexible ?

For at least 10+ years, we develop applications to work in containers. I won't be considering advantages and disadvantages of this approach but want to focus on the application flexibility. Almost every dependency, i.e. storage containers like Postgres, MySql, Redis a so on, allows us to override most of the configuration properties via environment variables. Docker containers stimulate us to use environment variables in our containers. But unlike of well-known services programmers develop custom applications on their own approach. I prefer to configure applications using JSON configuration files. But what should I do if in configuration files 100 and more properties, I can't use Environment variable for each property. Instead of this I decided to use JSON config file as a template with working default values and override properties at application start if appropriate environment variables were set.

How to implement such approach in Go application

Nowadays, we don't use a single Docker image; we prefer to have some orchestration, even something simple like docker-compose. In docker-compose we usually create .env-files. As it was mentioned before, environment variables are working well with well-known images like Postgres or MySQL. Consider we are having the following application config (JSON) that is using as an absolutely working template with the default values.

{
    "server": {
        "address": "0.0.0.0",
        "port": 8182
    },
    "logging": {
        "level": "info",
        "http_log": false,
        "http_console_out": false
    }
}

We should be able to override any of this value; consider that we should increase log level to debug and enable HTTP logging. For doing this easy, we just have to create technical env variables that have a special name pattern:

starts with a double underscore __
contains full property path, i.e. for log level __logging.level .

Using this go package, you could do it absolutely easy, all what you have to do:

Read JSON object from file using go_config_extender.LoadJSONConfigWithEnvOverride function (you could see full example in a test)
Set env file like this:

# all previous variables
__logging.level="debug"
__logging.http_log=true

That's all, and please give us a STAR on GitHub

Conclusion

This approach and package could be used not only for containerized applications but for apps running natively too. This package is successfully working on our authorization server.

GO writing and reading files with slices

Ushakov Michael — Thu, 07 Nov 2024 21:10:16 +0000

1. How possibly we should work with files using slices

Solving my programming tasks recently, I was figured out that there **are no methods that allowed to get slice of lines and save processed lines on disk. Of course, I could split a string into slice of strings and use that slice, but I would like to have some package once and use it easily, whenever I need it.

Well, I would like to have the following methods:

ReadAllLines that reads file content and return a tuple []string,error
WriteAllLines to write slice of strings
AppendAllLines to add slice of strings to end of existing file.

2.1 New package that meets all requirements

So after I decided what methods such package should have I wrote gfu (gfu is stands for Go File Utils) package and would like to share them with, see github repo:

2.1 ReadAllText method

This method does the following:
1 Returns tuple ([]string, error) of result with auto-detect line ending (CR, LF or CRLF);
2 Removes line ending symbols from slice items
3 Removes empty lines if omitEmpty argument is set to true

Example:

lines, err := gfu.ReadAllLines("myFile.txt", true)

2.2 WriteAllText method

This method does the following:

Inserts the line separator symbol defined in the 3rd argument of WriteAllLines func
Truncates a file if it exists, if not, a file will be created

Example:

lines := []string{
"{",
"    \"id\": 1,",
"    \"name\": \"Michael Ushakov\"",
"}",
}
file := "write_all_lines_test.txt"
err := gfu.WriteAllLines(file, lines, "\n")

2.3 AppendAllText method

WriteAllLines overwrites file content, but what should we do if we need to add some lines portion to an existing file? We should use the AppendAllLines function that, by signature, is identical to WriteAllLines:

lines := []string{
"{",
"    \"id\": 1,",
"    \"name\": \"Michael Ushakov\"",
"}",
}
file := "append_all_lines_test.txt"
err := gfu.WriteAllLines(file, lines, "\n")

additionalLines := []string{
"{",
"    \"id\": 2,",
"    \"name\": \"Alex Petrov\"",
"}",
}

err := gfu.AppendAllLines(file, lines, "\n")

3. Conclusion

All these functions are quite convenient and combined in a small package, also tests were written on all these functions, so we could consider them as reliable. And I go further on my software development journey. don't forget to give us a star if you find this package helpful.

Format a text in GO better than fmt

Ushakov Michael — Sun, 14 Jan 2024 18:47:23 +0000

Format a text in GO better then fmt

Looking at the article title, we should clarify what means better and what is text formatting. Lets start from the last one from these these theses. Text formatting is an important part of programming, prepared text is using in a various tasks:

description/result of some operations;
detailed log;
as a query for data selection in other systems;
and in many others fields. Better means that sf (wissance.StringFormatter) has features that fmt has't (see chapter 1 to see our text formatting approach).

1. What can do sf aka wissance.stringFormatter

In our earlier article we were writing about sf convinience (convenience is a thing that is subjective to humans; here, I mean convenience based on my own background). But briefly it is more covnenient to format text like:

userNews = stringFormatter.Format("Hi \"{0}\", see latest news: {1}", "john doe", "1. You won 1M$ in a lottery, please give us your VISA/MS card data to receive money.")

then like:

userNews = fmt.Sprintf("Hi \"%s\", see latest news: %s", "john doe", "1. You won 1M$ in a lottery, please give us your VISA/MS card data to receive money.")

Until version 1.2.0 sf was unable to make more precise argument formatting (i.e., use different number notation: bin, hex), starting with 1.2.0 we could do almost all that fmt supports:

Bin number formatting:
- {0:B}, 15 outputs -> 1111
- {0:B8}, 15 outputs -> 00001111
Hex number formatting
- {0:X}, 250 outputs -> fa
- {0:X4}, 250 outputs -> 00fa
Oct number formatting
- {0:o}, 11 outputs -> 14
Float point number formatting
- {0:E2}, 191.0478 outputs -> 1.91e+02
- {0:F}, 10.4567890 outputs -> 10.456789
- {0:F4}, 10.4567890 outputs -> 10.4568
- {0:F8}, 10.4567890 outputs -> 10.45678900
Percentage output
- {0:P100}, 12 outputs -> 12%

sf has 2 string format methods - Format and FormatComplex. Latter also allows passing argument formatting, like for Format method.

Let's consider a minimal example:

We should build text using the following format "Today tempearture is {temp}, humidity is {hum} where {temp} and {hum} should be replaced with an actual sensor values.
We would like to specify {temp} and {hum} output, i.e., {temp} should have 4 digits after the dot, and {hum} must be outputted in percents. After analyzing these requirements, we modified our template as follows: "Today tempearture is {temp:F4}, humidity is {hum:P100}".
Passing 12.3456 and 60 like this:

   sf.FormatComplex("Today tempearture is {temp:F4}, humidity is {hum:P100}", map[string]any {"temp":12.3456, "hum":60})

More examples could be found in a FormatComplex unit test, see in repo and below:

func TestFormatComplexWithArgFormatting(t *testing.T) {
    for name, test := range map[string]struct {
        template string
        args     map[string]any
        expected string
    }{
        "numeric_test_1": {
            template: "This is the text with an only number formatting: scientific - {mass} / {mass : e2}",
            args:     map[string]any{"mass": 191.0784},
            expected: "This is the text with an only number formatting: scientific - 191.0784 / 1.91e+02",
        },
        "numeric_test_2": {
            template: "This is the text with an only number formatting: binary - {bin:B} / {bin : B8}, hexadecimal - {hex:X} / {hex : X4}",
            args:     map[string]any{"bin": 15, "hex": 250},
            expected: "This is the text with an only number formatting: binary - 1111 / 00001111, hexadecimal - fa / 00fa",
        },
        "numeric_test_3": {
            template: "This is the text with an only number formatting: decimal - {float:F} / {float : F4} / {float:F8}",
            args:     map[string]any{"float": 10.5467890},
            expected: "This is the text with an only number formatting: decimal - 10.546789 / 10.5468 / 10.54678900",
        },
    } {
        t.Run(name, func(t *testing.T) {
            assert.Equal(t, test.expected, stringFormatter.FormatComplex(test.template, test.args))
        })
    }
}

It is clear that we've got convenient (for people with C#, Python background) library for text formatting. But we have another advantage - performance, sf makes formatting **FASTER than fmt**.

2. Perforamnce Benefits

There is one more important thing that could distinguish a sf library: it has performance advances, both in formatting cases with argument format specifications and without them. But FormatComplex is twice faster than fmt, see picture below with the results:

3. Conclusion

Today we could say that there is one more text formatting utility/lib that has all fmt features, and this library processes text faster, which in some cases could be important. Please give us a star on GitHub and subscribe.

My C# REST run now 250 times faster

Ushakov Michael — Mon, 29 May 2023 08:33:49 +0000

Intro

This article is about how to make you API work really faster (i've got 250 times faster) using our library, we are going to show it with our well-known example of Weather Control REST API. But in particular, we consider Create and Update operations of a standard CRUD controls. We made previously mentioned library to reduce amount of code and make a result of the same wrapping DTO (standartize the data contract). Today, we need just only < 10 lines of code to create any controller with all standard CRUD methods.

How we could speed up REST API

We could make API to perform faster if we are going to create/update multiple objects at once (bulk api), sometimes it is possible and could be crucial to achieve a higher performance. In the mentioned above example, we are using Controller class with Bulk Api (derives from BasicBulkCrudController) could be easily created as follows:

namespace Wissance.WeatherControl.WebApi.Controllers
{
    public class MeasurementsSeriesController: BasicBulkCrudController<MeasurementsDto, MeasurementsEntity, int>
    {
        public MeasurementsSeriesController(MeasurementsManager manager)
        {
            Manager = manager;  // this is for basic operations
            _manager = manager; // this for extended operations
        }

        private MeasurementsManager _manager;
    }
}

This code automatically contains 4 mehods:

GET /api/MeasurementsSeries for getting multiple measurements with paging
GET /api/MeasurementsSeries/{id} to get one measurements by id
POST /api/bulk/MeasurementsSeries for creating multiple objects (bulk API);
PUT /api/bulk/MeasurementsSeries for updating multiple objects (bulk API);
DELETE /api/bulk/MeasurementsSeries/{idList} for removing multiple object (bulk API), could be called like ~/api/bulk/MeasurementsSeries/id=1&id=2&id=5 A full example of Manager class with all bulk operations you could see here. Here i show non-optimal but working example with EntityFramework, even this impl make my code working 250 times faster for creating 10 items (see performance check below method code). So i just add array of MeasurementsEntity and add them to DbSet and after all make one Save instead of multiple for each object:

public override async Task<OperationResultDto<MeasurementsDto[]>> BulkCreateAsync(MeasurementsDto[] data)
        {
            try
            {
                IList<MeasurementsEntity> measurements = data.Select(d => MeasurementsFactory.Create(d)).ToList();
                await _modelContext.Measurements.AddRangeAsync(measurements);
                int result = await _modelContext.SaveChangesAsync();
                if (result >= 0)
                {
                    return new OperationResultDto<MeasurementsDto[]>(true, (int)HttpStatusCode.Created, null,
                        measurements.Select(m => MeasurementsFactory.Create(m)).ToArray());
                }
                return new OperationResultDto<MeasurementsDto[]>(false, (int)HttpStatusCode.InternalServerError, "An unknown error occurred during measurements creation", null);
            }
            catch (Exception e)
            {
                return new OperationResultDto<MeasurementsDto[]>(false, (int)HttpStatusCode.InternalServerError,
                    $"An error occurred during measurements update: {e.Message}", null);
            }
        }

Because we claim that we made our API working faster, let’s check performance using Python. For a very rough view we made comparison of time elapse for query single bulk endpoint vs sequential call of non-bulk endpoint (worst case) for POST new items as a result we’got ~250x time performance raise (see Python test here), see picture below

I wrote simple tests using Python that shows following results:

Elapsed time in Non-Bulk REST API with EF is 0.9759984016418457 secs.
Elapsed time in Bulk API with EF is 0.004002094268798828 secs.

So just using such a simple technique we made API working faster in a better case (for non-bulk API example) all request could be made parallel in time, but this isn’t solving bottleneck issue with persistence manager (EntityFramework in this current case).

Conclusion

Therefore, we could conclude that REST resources that allow to use bulk operations could be made working faster, and we have prepared boilerplate reusable solution (Wissance.WebApiToolkit). We would be very appreciated if you could give us a star on Github.

Make you Go code work 1.5x faster OR even more

Ushakov Michael — Wed, 12 Oct 2022 04:55:31 +0000

Introduction

Performance is a key thing of everything, we (humans) don't like to wait and waste our time. Therefore, sometimes quick solution as most of the managers suppose are better than slow but with good engineering and design. But today we are not speaking about management, but about code performance. We have a small text formatting library that allows us to format text using a template in convenient on our view way. These not only formatting function do what fmt.Sprintf does in more convenient way but also provide an additional features. Previous versions of our module were loose in performance to fmt.Sprintf, but since 1.0.1 we are better. And today we are going to tell how to make any golang code work faster.

Parameters and return values

There are two options to pass either arguments and/or get function result - by pointer and, by value, consider the following example:

func getItemAsStr(item *interface{}) string   // 1st variant
func getItemAsStr(item interface{})  string   // 2nd variant
func getItemAsStr(item *interface{}) *string  // 3rd variant
func getItemAsStr(item interface{})  *string  // 4th variant

According to our performance tests, we could conclude the following:

We've got small performance rise when we pass arguments by pointer because we've got rid of arguments copying.
We've got performance decrease when we return a pointer to function local variable

Therefore, the most optimal variant is the 1st variant.

Strings

Strings are immutable like in many other programming languages, therefore string concatenation using + operator is a bad idea, i.e. code like this is a very slow:

var result string = ""
for _, arg := range args {
    result += getItemAsStr(&arg)
}

Every "+" creates a new string object as a result memory usage rise, performance significantly decreases due to spending sufficient time on allocations on new variables.

There is a better solution for string concat - use strings.Builder, but for a better performance you should initially enlarge buffer (using Grow function) to prevent it from some / all re-allocs, but if buffer size will be very large there will be a penalty to performance due to initial memory allocation will be slow, therefore you should choose initial buffer size wisely, i.e.:

var formattedStr = &strings.Builder{}
formattedStr.Grow(templateLen + 22*len(args))

Cycles

There are 2 ways to iterate over collection:

using range expression

  for _, arg := range args {
      // do some staff here
  }

using traditional for with 3 expressions:

  for i := start; i < templateLen; i++ {
      // do some staff here
  }

2nd variant is better by performance. But there are additional advantages of usage for with 3 expressions:

reduce number of iterations, more iteration code is slower, therefore, if you could affect your loop value, do it. This can't be achieved with range;
set initial value for iteration much higher if it is possible, i.e. in our case:

  start := strings.Index(template, "{")
  if start < 0 {
     return template
  }

  formattedStr.WriteString(template[:start])
  for i := start; i < templateLen; i++ {
      // iterate over i
  }

Conclusion

Using such simple techniques we made our code run 1.5 times faster than it was, and now it works faster even than fmt.Sprintf, see our performance measurements:

. We also suggest you to use our library instead of fmt.Sprintf, because if you prepare strings for sql queries or something like this, it is important to make them as faster, as it possible. We would be thankful if you give us a star on Github and start to follow our organization here and on Github too.

Ferrum is a NextGen Auth Server (almost)

Ushakov Michael — Thu, 29 Sep 2022 12:09:49 +0000

Authorization and Why we created `Ferrum` project

Nowadays, almost all distributed, web or client-server systems have authorization to restrict access only to those users who have rights,OAuth2.0 and OpenId-Connect are most popular and convenient technologies for users authorization. There are some well-known authorization servers, i.e. KeyCloak.

Personally, I like KeyCloak and use it where it is possible. About 2 years ago I had to expand my technology stack and I started to write apps using Golang. These apps are WEB API that use Authorization on KeyCloak server. When API grows and become more complicated, it is very important to make automated integration tests. For authorization we using separate from demo/production server, and it is important to make code portable to any machine, therefore the most convenient way is to start all infrastructure before any test and stop when test ends with any persistent data cleanup at the end. Such approach can't be easily and fast implemented using Keycloak. Therefore, we created Ferrum authorization server that could be run and stopped from code and with API compatible with KeyCloak (with similar behavior and endpoints). Ferrum could be used not only for tests purposes but also like a full-fledged authorization server.

This is a tutorial how easily (in 1-2 minutes) add authorization to your solutions or how to write integrational tests on golang web api also in 1-2 min.

Build, configure, and run authorization server

To use Ferrum as a standalone authorization server, you need:

Build it with go build (optionally re-generate SSL Certificates using go generate);
Edit very simple config.json file and specify application endpoint;
Configure you server realm, clients and users data (for more details see next paragraph) in data.json file (in future we'll add store data in NO SQL and user register feature);
Run executable ./Ferrum, you authorization server is up. That's all.

Configure server data

Let's consider data configuration more detailed. Data is similar to what we are having on Keycloak. We have 3 types of objects for version 0.1.0:

realm = organization or solution
client = project / application inside realm;
user;

Every realm could contain as many clients and users as you wish. Realm structure is:

{
    "name": "myapp",
    "token_expiration": 330,
    "refresh_expiration": 200,
    "clients": [
     {
         "id": "d4dc483d-7d0d-4d2e-a0a0-2d34b55e5a14",
         "name": "test-service-app-client",
         "type": "confidential",
         "auth": {
                     "type": 1,
                     "value": "fb6Z4RsOadVycQoeQiN57xpu8w8wplYz"
         }
     } ],
     "users": [
      {
          "info": {
              "sub": "667ff6a7-3f6b-449b-a217-6fc5d9ac0723",
              "email_verified": false,
              "roles": [
                  "admin"
               ],
               "name": "admin sys",
               "preferred_username": "admin",
               "given_name": "admin",
               "family_name": "sys"
          },
          "credentials": {
               "password": "1s2d3f4g90xs"
          }
     }
     ]

Restrictions:

For data compatibility with Keycloak we must have "info" struct in every user with preferred_username property as a login;
We must have credentials struct and password property for a password check, in future we will store only password hashes, and won't store opened passwords.

Advantages:

We could have userinfo of any complexity (info structure in every user, see upper json) and we should not make some changes in the code. I.e. we like to add phone number, easy:

   {
       "info": {
           "phone": "+79996663322",
           // other props ...
       },
       "credentials": {
            // other props ...
       }
   }

Simple configuration, it enough 1-2 minutes to prepare server data and use;
Could be running from other application for tests purposes or as part of development / production solution.
API compatible with KeyCloak, you could easily switch to one technological stack in future when Ferrum will have all necessary features.

Ferrum REST API

For version v0.1.0 we have only 2 endpoints:

For issuing a new token {baseUrl}/auth/realms/{realm}/protocol/openid-connect/token/
For getting user info that issued this token {baseUrl}/auth/realms/{realm}/protocol/openid-connect/userinfo/

All these REST API

Write integrational tests with go

A full working example could be found in our integation test here
Minimal code example:

// 1. init app, appConfig, testServerData and testKey are variables
app := CreateAppWithData(appConfig, &testServerData, testKey)
// 2. start the app
res, err = app.Start()
// 3. issue new token (taken from func)
// baseUrl, realm and other token issue params are variables
tokenUrlTemplate := "{0}/auth/realms/{1}/protocol/openid-connect/token/"
tokenUrl := stringFormatter.Format(tokenUrlTemplate, baseUrl, realm)
getTokenData := url.Values{}
getTokenData.Set("client_id", clientId)
getTokenData.Set("client_secret", clientSecret)
getTokenData.Set("scope", "profile")
getTokenData.Set("grant_type", "password")
getTokenData.Set("username", userName)
getTokenData.Set("password", password)
response, err := http.PostForm(tokenUrl, getTokenData)
// 4. get a token and use
responseBody, err := io.ReadAll(response.Body)
var result dto.Token
err = json.Unmarshal(responseBody, &result)
stationEndpointTemplate := "{0}/api/station/"
getStationsUrl := stringFormatter.Format(stationEndpointTemplate, baseServiceUrl)
client := http.Client{}
request, err := http.NewRequest("GET",getStationsUrl, nil)
    request.Header.Set("Authorization", "Bearer "+ result.AccessToken)
response, err := client.Do(request)
// todo(UMV): parse response
// 5. stop app
app.Stop()

Conclusion

Thanks for reading, please help our project grow, please add us a github star
Our project will grow anyway, we are planning to add:

data store in one of NoSQL databases;
secure credential store;
integrations with external users provider like LDAP;
integrations with social providers like google;
possibility to authorize using biometrical data.

REST API with .Net Easily

Ushakov Michael — Fri, 24 Jun 2022 08:01:09 +0000

1. A short introduction to how to build Web Applications

REST API is a well-known approach to designing and structuring Web Applications. REST is not a protocol or framework it is a set of concepts. Although there are other approaches i.e. it could be used stateful services with one HTTP method verb (SOAP) or use specific full duplex GRPC or WebSockets connection. REST is a very popular approach: it allows us to use the same methodology for different resources (which usually maps on persistent objects, entities). As engineers, we try to optimize our work: we attempt not to repeat ourselves (DRY), simplify (KISS) and so on. Therefore, today i, as a part of the Wissance team, would like to share yet another boilerplate to write REST API easily see our WebAPI Toolkit.

2. What is REST

REST is a set of concepts that allows us to build a robust API (for example see this article).
REST operates with the term "resource" which usually has persistence nature i.e. mapped to database table and so on. Let's see if we have a user as a resource.
In REST we have separate HTTP methods to perform different operations:

CREATE this operation implements with POST HTTP method to resource i.e. POST {base_url}/api/resource (i.e. if {base_url} is http://10.0.0.10/myservice and resource is user full uri could look like: http://10.0.0.10/myservice/api/user) During creation we are passing user data as DTO in request body and getting back created user with set identifier and other generated on server fields.
READ this operation impelements with GET HTTP method and it usually contains 2 independent endpoints to get collection of entity items and to get one by id i.e. for user resource:
- GET {base_url}/api/user to get a collection
- GET {base_url}/api/user/{id} to get a single item by {id}
UPDATE this operation implements with PUT HTTP method (in this article PATCH method don't be considered). During a update we are passing resource as DTO through request body i.e. for user resource via url PUT {base_url}/api/user/{id}
DELETE this operation implements with DELETE HTTP method i.e for user resource via url DELETE {base_url}/api/user/{id}

Implementing REST API Service one by one i finally understood what could be made common. We created open source library with this coomon code and it helps us ti significantly reduce amount of any project code. This is our (Wissance LLC) project WebApiToolkit

3. Our approach and concepts

First of all we have following REST application architecture concepts:

All resources are storing in Entity classes that have required property Id (IModelIdentifiable<T> interface in our repo).
Controllers are just facades with some preliminary input data processing and validation, all application logic is implementing in appropriate Manager classes (Those we could use the same Managers for i.e. Signalr too not only for a REST API). Manager is an entry point to all complicated operation i.e. integration with other Web services and applications, with persistent storages like Databases and Queues and many others and so on is carrying on Manager classes. Controllers could only set Http Status codes and other things like setting some HTTP headers.
End user interacts with Web API using DTO which could have differences in number of fields in comparison with Entity i.e. we don't like to expose some fields or we do calculation for database fields during processing DTO by Manager.
We have Factory classes that allow to get DTO from Entity and vise versa.
All DTO classes are isolated from Entities and others application projects into separate project/assembly this allows us to easily distribute data contract as set of DTO as a Nuget package or directly by Add assembly manually to project.

We analyzed all OUR Net Core projects with REST API and created package with common code. If you like our package please don't forget to give us a star, it is important for us.

4. Let's see how it works in details

Lets consider that we are having a Weather station REST application. A full working code example is here, please don't forget to give a star to a this project too. In this application we have 2 REST resources that associated with 2 database tables:

weather station (meta info about where measurements were taken)
measurements (a set of weather control values: temperature, humidity, atmosphere pressure, e.t.c.) with set of sensors and sensors itself. Lets see how our controllers looks like and what could us give the Wissance.WebToolkit.

4.1 Controllers

In our Toolkit we have 2 base Controller classes:

For read-only resources - BasicReadController;
For resources that could be changed or deleted (Full CRUD) - BasicCrudController; In mentioned upper demo project we used BasicCrudController as a base class for Weather station and Measurements:

namespace Wissance.WeatherControl.WebApi.Controllers
{
    [ApiController]
    public class StationController : BasicCrudController<StationDto, StationEntity, int>
    {
        public StationController(StationManager manager)
        {
            Manager = manager;  // this is for basic operations
            _manager = manager; // this for extended operations
        }

        private StationManager _manager;
    }
}

As a result we have got:

Lightweight controllers, less code as it is possible;
We don't care about method declaration, routing, status codes processing;
Create and Update operations wraps response in OperationResultDto which allow to pass message what was wrong;
Get resource collection allowes to use pagination
We could use different identitifer type (int, string, guid) becuse it passed to controller class as a generic type.

This controller expose 5 endpoints:

GET ~/api/station/[?page={page}&size={size}] to get collection of station where page and size are optional parameters if they not provided default values is used (page=1, size=25);
GET ~/api/station/{id} to get one station by {id};
POST ~/api/station/ to create new station;
PUT ~/api/station/{id} to update station with id = {id};
DELETE ~/api/station/{id} to delete station with id = {id};

4.2 Managers

In our library we have base abstract class ModelManager that implements IModelManager interface. To work with BaseCrudController we should use Manager classes (they should implement IModelManager interface) i.e. StationManager looks like:

 public class StationManager : ModelManager<StationEntity, StationDto, int>
    {
        public StationManager(ModelContext modelContext, ILoggerFactory loggerFactory) : base(loggerFactory)
        {
            _modelContext = modelContext;
        }

        public override async Task<OperationResultDto<IList<StationDto>>> GetAsync(int page, int size)
        {
            return await GetAsync<int>(_modelContext.Stations, page, size, null, null, StationFactory.Create);
        }

        public override async Task<OperationResultDto<StationDto>> GetByIdAsync(int id)
        {
            return await GetAsync(_modelContext.Stations, id, StationFactory.Create);
        }

        public override async Task<OperationResultDto<StationDto>> CreateAsync(StationDto data)
        {
            try
            {
                StationEntity entity = StationFactory.Create(data);
                await _modelContext.Stations.AddAsync(entity);
                int result = await _modelContext.SaveChangesAsync();
                if (result >= 0)
                {
                    return new OperationResultDto<StationDto>(true, (int)HttpStatusCode.Created, null, StationFactory.Create(entity));
                }
                return new OperationResultDto<StationDto>(false, (int)HttpStatusCode.InternalServerError, "An unknown error occurred during station creation", null);

            }
            catch (Exception e)
            {
                return new OperationResultDto<StationDto>(false, (int)HttpStatusCode.InternalServerError, $"An error occurred during station creation: {e.Message}", null);
            }
        }

        public override async Task<OperationResultDto<StationDto>> UpdateAsync(int id, StationDto data)
        {
            try
            {
                StationEntity entity = StationFactory.Create(data);
                StationEntity existingEntity = await _modelContext.Stations.FirstOrDefaultAsync(s => s.Id == id);
                if (existingEntity == null)
                {
                    return new OperationResultDto<StationDto>(false, (int)HttpStatusCode.NotFound, $"Station with id: {id} does not exists", null);
                }

                // Copy only name, description and positions, create measurements if necessary from MeasurementsManager
                existingEntity.Name = entity.Name;
                existingEntity.Description = existingEntity.Description;
                existingEntity.Latitude = existingEntity.Latitude;
                existingEntity.Longitude = existingEntity.Longitude;
                int result = await _modelContext.SaveChangesAsync();
                if (result >= 0)
                {
                    return new OperationResultDto<StationDto>(true, (int)HttpStatusCode.OK, null, StationFactory.Create(entity));
                }
                return new OperationResultDto<StationDto>(false, (int)HttpStatusCode.InternalServerError, "An unknown error occurred during station update", null);

            }
            catch (Exception e)
            {
                return new OperationResultDto<StationDto>(false, (int)HttpStatusCode.InternalServerError, $"An error occurred during station update: {e.Message}", null);
            }

        }

        public override async Task<OperationResultDto<bool>> DeleteAsync(int id)
        {
            return await DeleteAsync(_modelContext, _modelContext.Stations, id);
        }

        private readonly ModelContext _modelContext;
    }

it is not small as Controller class but it still quite small. We also should mention that we have in ModelManager abstract classes following additional methods:

public async Task<OperationResultDto<TRes>> GetAsync(DbSet<TObj> dbSet, TId id, Func<TObj, TRes> createFunc)
public async Task<OperationResultDto<IList<TRes>>> GetAsync<TKey>(DbSet<TObj> dbSet, int page, int size, Func<TObj, bool> filter, Func<TObj, TKey> sort, Func<TObj, TRes> createFunc)
public async Task> DeleteAsync(DbContext context, DbSet dbSet, TId id)

Methods that we mentioned above did not used as a IModelContext methods implementations for one particular reason - we cannot implement univesal authorization to perform all sets of operations in the abstract class due to real cases could very complicated, therefore additional methods were created to help us to implement all required operations in any custom manager class like StationManager.

ModelManager abstract class additional features:

collection of objects could be filtered using delegate param - Func<TObj, bool> filter of GetAsync method;
collection of objects could be sorted using delegate param - Func sorted of GetAsync method.

We still have what to do (this is not the final version) and in one future release there will be a possibility to pass filter & sort params to appropriate controller and manager methods.

5. Additional Wissance.WebToolkit features

In our example project we did not consider following important things and we are trying to describe them here briefly (if this articles get 50 likes we will make second chapter):

authentication and authorization;
additional controller methods that expands controller interface over a just CRUD controller.

5.1 Authentication and authorization

We could use IHttpContextAccessor _httpContext to check whether user has access to resource or not, we could get Claim value like this where claimTypeId is a const like Username and so on:

private string GetClaim(string claimTypeId)
{
    ClaimsPrincipal principal = _httpContext.HttpContext.User;
    Claim claim = principal.Claims.FirstOrDefault(c => c.Type == claimTypeId);
    if (claim == null)
         return null;
    return claim.Value;
}

Don't forget to inject IHttpContextAccessor:

services.AddSingleton<IHttpContextAccessor, HttpContextAccessor>();

5.2 Additional controller methods

There is nothing complicated about extending controllers, if i.e. we have to search weather station in pseudo code we should add in controller something like this:

    [HttpGet]
    [Route("api/[controller]/search")]
    public async Task<IList<StationDto>> SearchInTaxServiceAsync([FromQuery]string query, [FromQuery]int page)
   {
       IList<StationDto> result = await _manager.SearchAsync(query, page);
       if (result == null)
       {
           HttpContext.Response.StatusCode = (int)HttpStatusCode.InternalServerError;
        }

        return result;
    }

6. Conclusion

Thank for reading bro, please rise you star on our github projects (WebApiToolkit && WeatherControl) if you find that they are helpful for you. As a conclusion we also would like to say just a fact: we significantly reduce amount of code and copy-paste code using our WebApiToolkit.

Easy Websockets testing

Ushakov Michael — Mon, 28 Mar 2022 17:57:16 +0000

Introduction

Nowadays surfing the web we are used to variety of modern conveniences, one of which is an ability to contact customer support in real time using online chat. The chat allows you to ask a question about certain product or service, request status and details of your order, consult a technician or file a complaint. All that without ever having to leave the web site.
Of course, such a thing was possible long before the introduction of the WebSocket protocol. Technically you could achieve something like that using Comet, which is a term for a bunch of hacky techniques to implement two-way interaction such as hidden iframe and http polling. Let’s just say this way of doing it was far from ideal. Fortunately, we got WebSocket, which unlike Comet was specifically designed for persistent two-way communication. In this article we are going to briefly consider the protocol’s intricacies and eventually learn how to test it. One more significant addition: this tools should work from private networks, therefore online services are not suitable for us. Dear reader, if you would like to support our cause, please:

Subscribe to our Medium channel;
Star our WebSocket interface testing application’s repository (https://github.com/Wissance/wstester); Ok, let’s proceed!

The difference between WebSocket and HTTP

In the very beginning the web relied entirely on HTTP which was designed to be stateless and as such work in the request-response fashion. Basically what it means is, every time you send a request your browser establishes a new connection with the server and once you receive the response the connection gets terminated. So in case you need to receive notifications or updates from the server there is no way to retain the connection. Your only option is or more accurately was so-called HTTP polling, which is a technique, where your browser repeatedly sends the HTTP request to the server(polls the server) to determine if there was some kinda change. The server either responds immediately or holds the connection until there is an update, the former is known as HTTP long polling.
If you have an online chat on your web site which for whatever reason employs any kinda polling, the client (your browser) has to constantly send HTTP requests in order to check for new messages. Such an approach generates a lot of unnecessary traffic and affects the server load which worsens its performance.

Now that WebSocket is around you no longer have to build such a monstrosity. Unlike HTTP WebSocket allows you to establish a persistent full-duplex(both ways) TCP connection which is gonna be alive for as long as you need. So, you can use WebSocket to send and receive messages from the server as if it was a regular TCP connection, which it is.

The testing tools

So how do we test it?
WebSocket requires an open TCP connection, besides, often frontend and backend are being developed separately and sometimes by different teams. I was tasked with WebSocket interface development many times and as there was no tool to test it I had to do it manually. I always wanted some kinda universal utility I could use regardless of the backend.
Actually it’s possible that I wasn’t looking hard enough for these things. But I assure you, there aren’t a lot of them out there to date. Recently I found out that new versions of Postman support WebSocket protocol. Sounds great, but unfortunately, Postman, just like any tool really got few disadvantages which make it not necessarily ideal for every situation:

There is something wrong with the way Postman stores its collections. One day something weird happened and I lost all of mine. All the requests just disappeared.
Postman is a single window application (what if i want to establish many (10+) connections is it possible or not, i have not check);
Postman is bloated and heavy; We developed our solution long before Postman added WebSocket support and considering all the aforementioned Postman flaws it’s still great in its own way.

Wstester, our tool to test WebSocket

When we were designing it we decided to make it as simple as possible. You could find it here

The key features are:

No need for bundlers, package managers and so on;
The code has only one dependency, which is JQuery and iy could be easily removed;
No heavy CSS frameworks like Bootstrap, only one css file (Pure.css);
Application automatically prepends WebSocket protocol ws:// to the address you enter;
Intuitive UI;
Portable (could be used in any private networks)

This is what the web interface looks like (to test it open tester.html file in your browser):
Here is a little video tutorial (Ru language) with a bit of theory and the testing process demonstration.

Conclusion

The application could use some work. There are few minor issues with the interface a*nd it lacks certain features such as request storage and so on. If there were enough people interested in this project (at least 30 stars on github*) we would be willing to spend our time on its further development. Thanks for reading and good luck!

DEV Community: Wissance

Beyond fmt: Building a Flexible Text Builder in Go with Wissance/stringFormatter

Introduction

Core Features: Template Syntax and Performance

Flexible Formatting Methods

Positional Formatting with Format

Advanced Argument Formatting

Conclusion

Create REST automatically in .Net

What is it and what is it for

Generate REST Controllers in one line

What could be done

Conclusion

Override Go app configuration with Environment variable

How to make containerized applications more flexible ?

How to implement such approach in Go application

Conclusion

GO writing and reading files with slices

1. How possibly we should work with files using slices

2.1 New package that meets all requirements

2.1 ReadAllText method

2.2 WriteAllText method

2.3 AppendAllText method

3. Conclusion

Format a text in GO better than fmt

Format a text in GO better then fmt

1. What can do sf aka wissance.stringFormatter

2. Perforamnce Benefits

3. Conclusion

My C# REST run now 250 times faster

Intro

How we could speed up REST API

Conclusion

Make you Go code work 1.5x faster OR even more

Introduction

Parameters and return values

Strings

Cycles

Conclusion

Ferrum is a NextGen Auth Server (almost)

Authorization and Why we created Ferrum project

Build, configure, and run authorization server

Configure server data

Ferrum REST API

Write integrational tests with go

Conclusion

REST API with .Net Easily

1. A short introduction to how to build Web Applications

2. What is REST

3. Our approach and concepts

4. Let's see how it works in details

4.1 Controllers

4.2 Managers

5. Additional Wissance.WebToolkit features

5.1 Authentication and authorization

5.2 Additional controller methods

6. Conclusion

Easy Websockets testing

Introduction

The difference between WebSocket and HTTP

The testing tools

Wstester, our tool to test WebSocket

Conclusion

Generate `REST` Controllers in one line

Authorization and Why we created `Ferrum` project