DEV Community: Obum

How to improve Flutter Forms

Obum — Sat, 05 Aug 2023 21:52:22 +0000

A Form is how we collect input from users in our apps. Forms are unique points of user interactivity. Examples of where you can find forms include:

Auth Screens (register or login),
Checkout Screens (cart contents),
Item Creation/Updating Screens (like todos), etc.

Most apps and websites have a form at some point. Forms are part of what defines a user's experience on our apps. We should improve forms to give users a good experience.

The forms we build are peculiar to a product or feature. There is no one-fits-all rule for forms. What should be a best practice could have exceptions. What we are saying is, wherever you are building a form, there is room for improvement.

Following are tips that you can apply to improve your forms in Flutter.

1. Overall Form UI
2. Form Validation
3. Password Fields
4. Other Input Types
5. Keyboard Types
6. Autofill
7. Disabling, Loading, and Submitting.
Conclusion

1. Overall Form UI

We can't talk about User Experience without appreciating the User Interface (UI). The UI is the first point of contact with users. It defines the "love at first sight" of our apps.

You want to use/build good UIs that will retain the user. In most cases, you should already have a pre-designed UI (predesigned with Figma or other tools) that you will just implement in Flutter.

You can implement any UI in Flutter. The idea is to implement beautiful UIs, whether you created the UI yourself or it was predesigned by someone else.

The unique combination of box decorations, fonts, colors, icons, sizes, spacings, shadows, etc. all constitute what your form UI looks like. Go for the best UI implementation to give your users a good experience.

Use descriptive labels and placeholders for form fields.

^{Image Source}

2. Form Validation

Don't trust Users. Don't trust Developers.

Yes, our code should never trust anybody. "Trust" here is assuming that the user or the developers will do what we expect. The code we write has to enforce boundaries on the data it manipulates.

Many apps use data. This data has to be checked to ensure the smooth flow of the system.

For example,

If you are building a registration form, you want to ensure that all emails or usernames are unique.
If you are building a booking/reservation system, you want to ensure that users can only book slots for resources in the future.
If you are building an online exam, you want to be sure the user has answered all questions before submitting. And so on.

Data validation involves checking user data before accepting it into the system. We should do data validation on both client-side and server-side code.

On client-side code, we don't trust the users. Or rather, we want to help the users to give us the type of data we expect and to respect constraints.

On server-side code, we don't trust the front-end developers. Or rather we want to be sure that anyone calling our APIs respects the constraints we expect.

Flutter is a client-side tool, so we will focus on client-side data validation. You can do data validation in client code anywhere you take user input in your app (not everything is a form).

Form validation is checking user-inputted data in a form. Validate forms to ensure data constraints.

Consider validating input fields on user interaction and "onSubmit" of the form.

Use descriptive messages to explain to the user was is the issue with each field.

^{Image Source}

Also, show validation messages next to each form field rather than showing a generic error at the top or bottom of the form.

^{Image Source}

To take input text from users in Flutter, use TextFields. To take input text while validating the input in Flutter, use TextFormFields.

TextFormFields accept a validator function parameter that should return null if the content of the input is okay or the error message as a string if the input's content (what the user filled) has issues.

If an input is required, the validator function should equally return some text explaining that that field is required.

Asides from required, you can check for other properties on input fields like mininum or maximum length of text, email validity, only digits, or against any RegExp Pattern.

Apply any validation logic you want inside the validator function of your input fields.

For email validation, you can use Regex yourself or use packages like email_validator on pub.dev.

TextFormFields also accept a maxLength integer to help you enforce maximum length on input field values (without the validator).

They also accept an inputFormatters list of TextInputFormatters for more form validation properties.

Form widgets (like every widget in Flutter) accept a key parameter. You can use the form's key to validate form fields before submitting form data.

Flutter Forms also accept an autovalidateMode enum with values of always, disabled, and onUserInteraction.

To reset the validation state of a Flutter Form, reassign the value of the form's key and empty the contents of inputs through their controllers.

3. Password Fields

Passwords are usually in auth-related screens: either registration or login.

It is good practice to obscure the user's password against preying eyes. However, the user sometimes might want to view the password. We shouldn't deny our users this feature.

As you obscure passwords, please add a button to toggle viewing and obscuring the password. Flutter TextFormFields have an obscureText boolean property. They also permit you to specify the obscuringCharacter.

It is also common practice to double the password fields in registration forms. Doing this helps users to be sure of the password they are creating.

The controller of a TextField tells its content. We can use the controller of a password field in the validator of the confirm password field to know whether passwords match.

Still on passwords, you might want users to use complex passwords. Maybe minimum length of 8 characters, mixing alphabets, numbers, and symbols, not using common names, etc.

You can all such logic in the validator function of your password field. You can also add extra UI validation feedback yourself to tell the user the strength of their passwords.

4. Other Input Types

Text is not the only collectible data. When we want users to select one or more options, we have to use appropriate widgets for those cases.

Use radio buttons when at least one and only one option can be chosen. Use checkboxes when multiple options can be chosen.

DropdownButtons are also usable in forms. Use a dropdown if only one option is to be selected from a list of more than 3 options. If the list is long (more than 10), consider adding "search" functionality for the user to easily locate their choices.

These selector widgets can also be involved in form validation. You might have to add and style extra widgets to show their error messages next to them. Their errors will mainly be "required"-related errors.

In such case, in your submit function, when checking if the form's values are valid with formKey.currentState!.validate(), also check if the values of selector fields have been set.

There is a DropdownButtonFormField widget that presents as a dropdown inside a form field. It takes a decoration property and a validator function, just like TextFields. Use it where necessary.

If you want to show a custom widget as a field, you can wrap it in the InputDecorator widget and it will appear as you want.

You can also have fields for Date and Time. Use pickers for these fields. Flutter has the showDatePicker function to open a Date picker. We also have the showTimePicker function.

If you want more customization, there are packages on pub.dev to suit your needs. Or you can code one yourself (it's harder).

You can have a field that will display the selected date/time. But this field will not be "editable". Like the user can't type text into it. On tap of the field, open a picker, and in the callback, set the value of the field to the chosen date.

You can have validators in the field too. You can validate chosen date or time as you wish in the picker's callback or in your validator function.

5. Keyboard Types

Still in the world of optimizations, we can open specific keyboard types for each TextField in Flutter.

Set the keyboardType on the TextField to any of the available TextInputType and you are good to go. The types include emailAddress, phone, number, etc.

You can set a keyboard type for each field to match the demands of that field.

For example

If you set the keyboard type to TextInputType.emailAddress, the mobile operating system keyboard will have a prominent @ symbol for emails.
If you set it to TextInputType.phone, the user will see mainly digits on the keyboard.
If you set it to TextInputType.multiline, there will be an enter key, and so on.

textInputAction is also a valid setting for text fields in Flutter. It takes any value of the TextInputAction enum like done, search, send, etc.

These will show a corresponding icon or text at the end of the opened keyboard in each mobile operating system. It helps to tell the user what the field does in that context.

6. Autofill

Autofill or autocomplete is when browsers or operating systems fill forms in apps or websites on behalf of users.

If a user opts in for autofill, the system saves user data that has been filled in previous forms. In the next form, the system will help the user fill out the form (autofill).

The user can then edit what the system auto-filled to suit their purpose.

Autofill info could also be gotten from the user's device directly. Like the signed-in email addresses, phone numbers of sim cards, or personal names that the user has set on their phone.

Autofill makes life easy. It makes users spend less time on forms and makes them happy.

We as developers have to make our forms compatible with autofill to help users that opt-in for it. We have to tell the browser or OS what data our form fields are collecting. This will make the system fill the right data into the right field.

TextFields in Flutter accept an autofillHints property. It takes a list of AutoFillHints enums. This enum has values like email, name, newPassword, password, telephoneNumber, username, etc. There are many options. Set the right ones on each field.

Autofill is also helpful with passwords. Many users use password managers. These managers rely on autofill to know when to fill in passwords in apps or websites for the users.

When autofill is properly set, these managers can also suggest new passwords for the user's account on your app (which is what you want).

You can wrap related fields (like email and password fields) in an AutofillGroup widget. This widget tells the system that the fields are linked. This is a further useful feature for password managers.

Set autofill in your fields. The platform where Flutter is running will collaborate with your form and help the user as you expect.

7. Disabling, Loading, and Submitting.

Your form should usually have a button towards its end for submitting all the user-inputted data. This button should call a submit function. The submit function should validate the form before you submit its values.

// in submit function
if (formKey.currentState!.validate()) {
  // submit the form
} else {
  // toast "resolve all errors"
}

If your Flutter code is running on the web, you may want to set the onFieldSubmitted property on the TextFormFields to the submit function. This will make the form submit when the user presses the enter key while on a desktop browser. Web users are used to this on websites so let's give them the same experience.

Most of the time, submitting your form is an asynchronous process. You would be submitting this form to your backend. You want to take care of the user flow after submission. You can navigate the user to another waiting screen or show a loader to tell the user to wait.

If errors occur during submission (maybe network error), you want to reinstate the form with the values the user submitted (or just deactivate the loading), so that the user can retry submitting.

You don't want the user to change any of the values of fields while the form is been submitted. You may want to have an isSubmitting boolean. Then set the enabled property on all form fields to !_isSubmitting (that's when not submitting). This will make disable the fields when _isSubmitting is true.

The submit function should toggle the value of _isSubmitting. It will make it true before it starts and make it false after processing.

// in submit function
if (formKey.currentState!.validate()) {
  setState(() => _isSubmitting = true);
  try {
    // submit form data 
  } catch (e) {
    // toast the error ("e")
  } finally {
    setState(() => _isSubmitting = false);
  }
} else {
  // toast "resolve all errors"
}

You can use Snackbar or any custom toast in pub.dev to toast error feedback from the backend after submission.

Supposedly, if the user's submission was successful, your UI should change. The user shouldn't see the form again. Depending on your app, after successful form submission, you could still toast successful (or a useful message), then navigate the user to another screen or close the form.

Conclusion

Forms are how we collect users' data. Forms should not be difficult for users.

The above are tips to improve your forms in Flutter. You can combine them or use all of them.

Build forms that make life easy for the user. Your objective should be to give your users a nice experience while they are in your app.

Cheers!

How to Restore Poison Messages (or Dead Letters) from Memphis broker vs RabbitMQ

Obum — Thu, 22 Sep 2022 16:39:59 +0000

In this article, you will understand possible the causes of poison messages in a message broker. You will also learn how to restore them in both Memphis.dev and RabbitMQ.

What is Service Oriented Architecture?
What is a message broker?
About Memphis.dev
About RabbitMQ
What are poison messages?
How to Restore Poison Messages from Memphis
How to Restore Poison Messages from RabbitMQ
Summary

What is Service Oriented Architecture?

Service Oriented Architecture (SOA) involves splitting different parts of your application into services. That way you can have services for Authentication, Batch-Processing, Managing Videos, Notifications, etc.

Separating core functionalities into services help your platform to scale appropriately. You also get to enjoy the benefit of the whole system not breaking down when one service malfunctions. In addition, you can improve each service without tampering others.

Such backend features usually carry out asynchronous tasks. These asynchronous activities don’t complete at the same time. If a user is interacting on your platform, you don’t want the user to wait for long processes to complete. You want to continue interactivity. At the same time, you want a service to tell you when it is done with its job. This is where message brokers come in.

What is a message broker?

A message broker is a communication tool used within service-oriented platforms. A message broker (or message queue) is a man-in-the-middle for services in your backend.

With message brokers, some services can send messages to the broker whereas some can receive. Services that send messages are called producers. On the other hand, services that receive messages are called _consumers. _

A “message” in message broker stands for what the service sends. It could be an event, piece of data, a task, etc. A “message” represents what the services communicate.

As a broker, a message broker is an intermediary for messages. As a queue, a message broker permits you to stock up messages till consumers get them. Message brokers complete the scaling of Service Oriented platforms. Message brokers ensure that services appropriately communicate with each other.

Memphis.dev and RabbitMQ are example message brokers. They are both open source. Both Memphis.dev and RabbitMQ have a dashboard (or UI) to visualise and manage “messages”.

You can install both of them on any Operating System with Docker. If you don’t have Docker installed, install Docker for your Operating System here.

About Memphis.dev

Memphis.dev is a real-time data processing platform. It is an open-source, real-time data processing platform for in-app streaming use cases.

Run the following command in your terminal to install Memphis.

curl -s https://memphisdev.github.io/memphis-docker/docker-compose.yml -o docker-compose.yml && docker compose -f docker-compose.yml -p memphis up

When the command completes, open localhost:9000 in the browser, it should show you the Memphis UI. Execute the following steps to get started:

Enter your email, full name, and set a password. Your email address is important for account recovery. Create account.
Enter hello for both factory and station names. These are just for demo purposes. Click Next.
Enter hello for app username. Click next.
Click or skip through the next welcome onboarding code and steps. Then launch the dashboard.

About RabbitMQ

RabbitMQ is the most widely deployed open source message broker. With tens of thousands of users, RabbitMQ is one of the most popular open source message brokers.

Run the following command in your terminal to install RabbitMQ.

docker run -it --rm --name rabbitmq -p 5672:5672 -p 15672:15672 rabbitmq:3.10-management

When the command completes, open localhost:15672 in the browser, it should show you the RabbitMQ UI. Sign in with guest as both username and password. These are the default credentials when you set up RabbitMQ with Docker.

What are poison messages?

Poison messages are those messages that have not reached their destination. The message broker had received them but it found it hard to deliver them. At times, it could not deliver them because the consumer may be offline. So the broker will keep sending the messages till they finally become poison.

Another reason why poison messages could occur is because of acknowledgment. In a normal setup, when a consumer receives and uses up a message from the broker, the consumer acknowledges that message. Acknowledging a message tells the broker that that message has been received. Acknowledging is important. It is the broker’s way of ensuring proper communication across services.

Asides acknowledging, a consumer can reject a message (negative acknowledgment). When a consumer receives a message and rejects it or does not acknowledge it, the message broker will keep on retrying. It keeps retrying because it wants to ensure that the message gets across to the consumer. After all, that’s why it is a broker in the first place!

Also, there is usually a predefined number of times a broker would keep retrying messages before those messages become poison. You can also set this “number of times”. Also, you can instead configure a message broker to keep retrying but for a period of time (like 5mins, 2hours, 7days, etc.)

We call these failed messages poison because they introduce unintended effects to the system. If it was a user-initiated task, this user may never get the intended value delivered. If the value were a paid feature, such failure could cause a terrible customer experience (and loss of money). On the broker’s side, it could reduce platform fidelity. As it could seem that the message broker is not reliable for its job.

Poison messages are also called dead letters. “Dead letters” suit properly for poison messages for two reasons:

Messages are letters in the context of the broker’s transactions. A letter has a destination likewise a message.
The letters are “dead” because they did not reach their appropriate destination.

Rather than discarding failed or unacknowledged messages, message brokers keep retrying as stated above. If retry time or number of retries exceed the preset maximum, message brokers store these poison messages (or dead letters). You then need to restore these poison messages to ensure smooth running of your system. How you restore poison messages (or dead letters) depend on the message broker you are using.

How to Restore Poison Messages from Memphis

To restore poison messages in Memphis, resend the poison messages and acknowledge them.

To see this in action, let’s use NodeJS to create a simple “Hello World” usecase of Memphis. We will write code that sends and receives “Hello World” messages to and from Memphis. However, the consuming part will not acknowledge the messages. This way Memphis will keep sending till they become poison messages.

Create a new memphis-poison folder, install memphis, and run the code for the above setup. Run the following commands in your terminal to do the above.

mkdir memphis-poison 
cd memphis-poison 
npm init -y
npm install --save memphis-dev

Create a new index.js file inside the newly created memphis-poison folder. Paste the following JavaScript code inside.

const memphis = require('memphis-dev');

(async function () {
  try {
    await memphis.connect({
      host: 'localhost',
      username: 'hello',
      connectionToken: 'memphis'
    });

    const consumer = await memphis.consumer({
      stationName: 'hello',
      consumerName: 'hello_consumer1',
      consumerGroup: 'hello_consumer_group',
    });
    consumer.on('message', (message) => {
      console.log(message.getData().toString());

      // commented out the following line to produce poison messages
      message.ack();

    });
    consumer.on('error', (error) => {
      console.log(error);
    });

    const producer = await memphis.producer({
      stationName: 'hello',
      producerName: 'hello_producer1'
    });
    for (let index = 0; index < 10; index++) {
      await producer.produce({
        message: Buffer.from('Hello world')
      });
      console.log('Message sent');
    }
    console.log('All messages sent');
  } catch (ex) {
    console.log(ex);
    memphis.close();
  }
})();

The above code connects to Memphis broker. It produces 10 “Hello World” messages and consumes them. Run the file by running the following command in the terminal. Ensure you are within the memphis-poison folder inside the terminal.

node index.js

Notice that the terminal keeps receiving the “Hello World” messages. This is because the message.ack(); line is commented out. That code is supposed to acknowledge the messages but as it is commented out, the messages become poison messages.

The default Time-To-Live (TTL) for messages in Memphis is 30 seconds. So the code will keep receiving “Hello World”s for up to half a minute before it stops and becomes poison.

Check the Memphis UI dashboard to see the poison messages. It will create a Dead Letter Station (DLS) and keep the poison messages there.

To restore the poison messages, simply restore and acknowledge them.

Select all of the poison messages and click on resend.
Get back to the code, uncomment the message.ack(); line, and re-run the file.

This will re-produce 10 “Hello World”s, then consume them, and also restore the poison messages from the previous run. And that’s how to restore poison messages from Memphis broker.

How to Restore Poison Messages from RabbitMQ

To restore poison messages in RabbitMQ, create a Dead Letter Exchange (DLX) for those messages.

To see this in action, let’s use NodeJS to create a simple “Hello World” use case of RabbitMQ. We will write code that sends and receives “Hello World” messages to and from RabbitMQ. However, the consuming part will negatively acknowledge the messages. This way RabbitMQ will keep sending till they become poison messages.

Create a new rabbitmq-poison folder, install rabbitmq, and run the code for the above setup. Run the following commands in your terminal to do the above.

mkdir rabbitmq-poison 
cd rabbitmq-poison 
npm init -y
npm install --save amqplib

Create a new index.js file inside the newly created rabbitmq-poison folder. Paste the following JavaScript code inside.

const amqp = require('amqplib/callback_api');

amqp.connect('amqp://localhost', function (error0, connection) {
  if (error0) throw error0;

  const queue = 'hello';
  const msg = 'Hello World!';

  connection.createChannel(function (error1, channel) {
    if (error1) throw error1;

    channel.assertQueue(queue, {
      durable: false,
      messageTtl: 10000
    });

    for (let index = 0; index < 10; index++) {
      channel.sendToQueue(queue, Buffer.from(msg));
      console.log(` [*] Sent ${msg}`);
    }
    console.log('All messages sent');

    console.log(` [*] Waiting for messages in ${queue}. To exit press CTRL+C`);
    let receiveCount = 1;
    channel.consume(queue, (msg) => {
      console.log(` [${receiveCount}] Received ${msg.content}`);
      receiveCount++;

      // the following negatively acknowledges the message to make it poison
      channel.nack(msg);
    });
  });
});

The above code connects to RabbitMQ broker. It produces 10 “Hello World” messages and consumes them.

In channel.assertQueue, we’ve set the messageTtl (Messages’ Time To Live - TTL) to 10000 (10 seconds actually). We are setting this so that RabbitMQ can stop retrying after 10 seconds. RabbitMQ doesn’t set a default TTL. So we can say that RabbitMQ’s default TTL for messages is infinity. We are setting this TTL to ensure that the messages quickly become poison messages.

In channel.consume, we are keeping count of how many times the consumer received messages from RabbitMQ. The receiveCount quickly gets huge under 10 seconds and finally stops counting (the consumer stops receiving). It gets this high to tell how much RabbitMQ keeps retrying messages to ensure they get delivered.

Still in channel.consume, the channel.nack(msg); line causes the messages to be negatively acknowledged. Given that the messages have not been acknowledged and that they have a 10-second time-to-live, they become poison messages.

Run the file by running the following command in the terminal. Ensure you are within the rabbitmq-poison folder inside the terminal.

node index.js

To restore the poison messages, create a Dead Letter Exchange (DLX) for those messages. To create this DLX, you will need to setup a different channel for consuming these dead letters (or poison messages). This DLX will consume the dead letters immediately after they get negatively acknowledged.

Create a different dlx.js file within the rabbitmq-poison folder. Paste the following code inside

const amqp = require('amqplib/callback_api');

amqp.connect('amqp://localhost', function (error0, connection) {
  if (error0) throw error0;

  const queue = 'helloDlx';
  const msg = 'Hello World!';

  connection.createChannel(function (error1, channel) {
    if (error1) throw error1;

    channel.assertQueue(queue, { deadLetterExchange: 'dlx' });

    let sendCount = 1;
    const interval = setInterval(() => {
      channel.sendToQueue(queue, Buffer.from(msg));
      console.log(` [${sendCount}] Sent ${msg}`);

      if (sendCount > 10) {
        clearInterval(interval);
        console.log('All messages sent');
      } else {
        sendCount++;
      }
    }, 500);

    console.log(` [*] Waiting for messages in ${queue}. To exit press CTRL+C`);
    let receiveCount = 1;
    channel.consume(queue, (msg) => {
      console.log(` [${receiveCount}] Received ${msg.content}`);
      receiveCount++;

      // makes poison messages and send them to the dead letter exchange above
      channel.nack(msg);
    });
  });

  connection.createChannel(function (error2, channel) {
    if (error2) throw error2;

    channel.assertExchange('dlx', 'direct');

    let receiveCount = 1;
    channel.consume(queue, (msg) => {
      console.log(` [${receiveCount}] Restored Poison ${msg.content}`);
      receiveCount++;
    });
  });
});

The above code connects to RabbitMQ broker. It uses a different queue (helloDlx) from the queue in index.js (hello) to show the restoring process. The main difference between this code and the code index.js is that this code introduces the Dead Letter Exchange (DLX).

In this example, the name of the DLX is simply 'dlx'. You should use more meaningful DLX names in your codebases. DLX are backup kind of services to ensure that the poison messages get properly handled.

When creating the DLX (inside the second channel), we used channel.assertExchange('dlx', 'direct');. However, when setting the primary queue for the messages (inside the first channel), we indicated the DLX in the options parameter with channel.assertQueue(queue, { deadLetterExchange: 'dlx' });.

The above code also sends a message every 0.5 seconds till they reach 10 messages. This speed reduction is to see the restoring process in action.

Run this dlx.js file in your terminal:

node dlx.js

This consumes the dead letter messages immediately. Notice that the “Hello World” messages get restored after their first negative acknowledgment.

Summary

To restore poison messages in Memphis, simply resend and acknowledge them. To restore poison messages in RabbitMQ, setup a dead letter exchange for them.

The manner of creating poison messages in this article was mainly for demo purposes. In real life, you will encounter poison messages from unforeseen circumstances. Consuming services could get faulty and hence cause unacknowledged messages. Either way, you’ve learned how to restore poison messages from Memphis broker and RabbitMQ.

Cheers!

Helpful Resources

How to build a real-time app with NestJS and Memphis broker

Obum — Wed, 24 Aug 2022 10:46:00 +0000

Introduction

Major systems have all become digital. Usage of applications keeps increasing. Scaling your apps is now implicit.

Usually, your application will consist of multiple services. These services will each be in charge of doing something different. Let's say you're building a chat application. You could have a messages service, a notification service, a status/stories service, a call service, an authentication service, etc.

One or more of these services have to communicate with each other. You want your app to scale properly. You'll choose the best communication method for your platform's services. You need a message broker.

A message broker is an intelligent middleman for your application's services. A message broker receives "messages" from one or more services. It then delivers these "messages" to other services that want to consume them. This way, your platform scales and meets your users' demands.

A message broker is made up of 3 parts. A queue, producers, and consumers. The queue is the manager of messages. The producers send messages to the queue while the consumers receive them.

What is Memphis Broker?

Memphis is a modern, powerful, and open-source message broker. Memphis takes away the excess work for data engineers and developers. Memphis broker gives you good developer experience. It reduces your development time by 90%.

Memphis is unique. It comes with a UI and a CLI. You can use the UI to visualize the movements of data. You can use the CLI to run important commands on the broker. There is also the sandbox at sandbox.memphis.dev for interacting to see what it looks like without installing it.

Memphis is available for Go, Java, NodeJS, and Python. More platforms will be supported soon. In Memphis, a queue is called a station. And many stations can be found in a factory. When you start using Memphis for the first time, you will at least need to create a factory and a station inside that factory.

How to set up Memphis broker using Docker

Docker is a popular tool for executing and managing software. It provides containers for software execution. It guarantees that a container will have the required dependencies for a given program to run.

You can use Docker to install Memphis. The advantage of using Docker is that you are assured that Memphis will work properly everywhere. Memphis will be the same in your local development and when your app is in production.

To install Memphis, you need to have Docker installed. If you don’t have Docker installed, install Docker for your Operating System here. Start Docker and ensure it's running.

Run the following command in your terminal to install Memphis. It'll fetch the Docker compose file for Memphis set up and set up Memphis appropriately.

curl -s https://memphisdev.github.io/memphis-docker/docker-compose.yml -o docker-compose.yml && docker compose -f docker-compose.yml -p memphis up

If the command completes successfully, Memphis should be up and running in Docker. If you encounter any issues, consider checking your internet connection.

This is a one-time installation of Memphis. In consequent times, when you need to use Memphis, simply launch Docker Desktop and ensure that the Memphis containers are running.

Carry out the following instructions to complete the Memphis setup in the UI:

Open localhost:9000 in the browser, it should show you the Memphis UI.
Sign in with root and memphis. These are the default credentials of the “root” user when you set up Memphis with Docker.
Enter chat as the factory name. Enter chat as the station name. Click “Next”.
Enter chat as the application user name. Click “Create app user”. It should show you “memphis” as the connection token (password). This is the default.
Skip through the next steps and access the dashboard.

Note: You can use a different factory, station, and username instead of chat above. We are using chat to keep things simple. Use names that are meaningful to your project. The Memphis UI permits you to manage factories, stations, and users. You can create more users with varying passwords/tokens. This ensures security.

What we will be building

Many applications now need to have a real-time ability. Real-time event-driven architectures use message brokers for many-to-many inter-service communication. Examples of real-time apps include online games, admin dashboards, chat apps, etc.

Let's build a simple chat app. Because it is a simple app, it will have only two services for now. One for all messages and the other for sending messages. The app should have more services but these two are enough to show how to use Memphis.

Let's call this chat app FastChat. In this article, you'll learn how to build FastChat using Memphis broker and NestJS. To keep things simple, FastChat won't save chat messages to any database. It'll keep them in memory while the application is alive. FastChat mainly focuses on showing you how to use Memphis broker.

The final codebase for FastChat is here on GitHub.

What is NestJS?

NestJS is an open-source JavaScript framework for server-side applications. It leverages TypeScript's features to build robust apps. You can structure your Nest project around controllers, modules, and services to properly modularize the code.

FastChat has two services: AllMessages and SendMessage service. These two services will intercommunicate using Memphis broker. We will group them into a Nest module (ChatMessagesModule) because they relate to the same entity.

FastChat also has a BrokerModule. This module will configure Memphis broker. It'll import the MemphisModule and connect the Nest app to Memphis. The BrokerModule will also expose a BrokerService through which other parts of the application can access Memphis.

How to set up NestJS

To install NestJS, you need to have NodeJS installed. If you don't have NodeJS installed, download and install it from here.

To install NestJS, open a new terminal instance and run the following command:

npm install -g @nestjs/cli

Now create a new NestJS project. We will use fast-chat for the project name. Run the following command:

nest new fast-chat -p npm

The -p flag is use npm as package manager. Open the newly created fast-chat project in your favorite code editor.

How to set up the ConfigModule in NestJS

Next, we have to set up Memphis broker inside NestJS. But during the setup, we have to connect NestJS to Memphis broker. For such connection, we need at least three properties:

host: The domain name or IP address where Memphis is found, e.g. localhost
username: The user that NestJS will act on their behalf, e.g. root
connectionToken: The password (token actually) of the above user.

The above three are sensitive information. For security purposes, they shouldn’t be added to source control. Rather, we need to access them through environment variables.

NestJS provides a ConfigModule for providing environment variables. It uses dotenv internally. Create a file with .env as name directly inside the fast-chat project folder. Paste the following inside:

MEMPHIS_HOST=localhost
MEMPHIS_USERNAME=chat
MEMPHIS_TOKEN=memphis

These are the default credentials. You should use more secure credentials in production. That is the very reason of use the ConfigModule in the first place.

Add the NestJS’ ConfigModule to the fast-chat project. Run the following command to install it.

npm install --save @nestjs/config

Import the ConfigModule in AppModule. NestJS auto-created the src/app.module.ts file among other files when we created the project. This file contains the declaration of AppModule. Replace the contents of AppModule with the following:

import { Module } from '@nestjs/common';
import { ConfigModule } from '@nestjs/config';
import { AppController } from './app.controller';
import { AppService } from './app.service';

@Module({
  imports: [ConfigModule.forRoot({ isGlobal: true })],
  controllers: [AppController],
  providers: [AppService],
})
export class AppModule {}

Here, we are importing the ConfigModule and setting the isGlobal to true. Doing this will auto-import the ConfigModule in all other modules we wil create (which is what we want). In consequent files that need any values from the .env file, you will only access them from ConfigService (coming later on).

How to setup Memphis broker in NestJS
Run the following command to install Memphis in FastChat:

npm install --save memphis-dev

Generate the BrokerModule (to set up Memphis) with the following command:

nest generate module broker

This will create a new broker.module.ts file inside a new broker folder. Delete the contents of this new broker.module.ts file and paste the following:

import { Module } from '@nestjs/common';
import { MemphisModule } from 'memphis-dev/nest';

@Module({
  imports: [MemphisModule.register()],
  exports: [BrokerService],
})
export class BrokerModule {}

The above imports and registers the MemphisModule. It also exports a BrokerService (which we will create right away. This BrokerService connects the app to Memphis (when NestJS starts up). This BrokerService is what other parts of the Nest app will use to interact with Memphis.

Run the following command to create the BrokerService:

nest generate service broker

This will create a new broker.service.ts file inside the broker folder. Delete its contents and paste the following:

import { Injectable, OnModuleInit } from '@nestjs/common';
import { ConfigService } from '@nestjs/config';
import { MemphisService } from 'memphis-dev/nest';

@Injectable()
export class BrokerService implements OnModuleInit {
  consumer;
  producer;

  constructor(
    private configService: ConfigService,
    private memphisService: MemphisService,
  ) {}

  async onModuleInit(): Promise<void> {
    try {
      await this.memphisService.connect({
        host: this.configService.get<string>('MEMPHIS_HOST'),
        username: this.configService.get<string>('MEMPHIS_USERNAME'),
        connectionToken: this.configService.get<string>('MEMPHIS_TOKEN'),
      });
      this.consumer = await this.memphisService.consumer({
        stationName: 'chat',
        consumerName: 'chatConsumer',
        consumerGroup: 'chatConsumers',
      });
      this.producer = await this.memphisService.producer({
        stationName: 'chat',
        producerName: 'chatProducer',
      });
    } catch (error) {
      console.error(error);
      this.memphisService.close();
    }
  }
}

The BrokerService imports the ConfigService and the MemphisService. NestJS injects these services using its automatic dependency injection.

The BrokerService implements the OnModuleInit lifecycle method. This is the right place to connect Memphis broker. It is the right place because NestJS calls this method when initializing dependencies. Notice that it connects to Memphis using environment variables from the ConfigService.

The BrokerService also exposes references to the producer and consumer entities of Memphis broker. Other services will use these references to produce or consume "messages" (or events).

Note: Use meaningful consumer and producer names. Use meaningful names that can distinguish which project a particular producer is producing from or from which is a consumer consuming.

Now we've successfully set up Memphis broker. We now need to create the ChatMessages services and link them with Memphis.

But Before we head over to creating services let's review another useful tool. Servers usually expose REST APIs that clients will call. However, there are alternative methods for server-to-client communication. One of such methods is gRPC.

What is gRPC?

RPC means Remote Procedure Call. It means calling a function that is located remotely (in the server).

gRPC is an open-source RPC framework for client-server communication. gRPC exposes server functions or microservices to clients. The clients too use gRPC to access data from the server(s). gRPC is available for all programming languages.

gRPC is faster than REST APIs. It is faster because it uses Protocol Buffers to exchange data (rather than JSON or XML). Protocol Buffers (ProtoBuf) is a neutral way of expressing structured data. ProtoBuf is easy to understand.

FastChat uses gRPC because gRPC is the go-to way for client-server communication in a Service Oriented Architecture (SOA).

How to set up gRPC in NestJS

To set up gRPC, run the following command while inside the fast-chat directory:

npm i --save @grpc/grpc-js @grpc/proto-loader @nestjs/microservices

It adds necessary dependencies for using gRPC in NestJS. Next, create the protobuf (.proto) file for the services in FastChat.

Create a new folder in the src folder, name this new folder chat-message. Create a new file inside this new folder. Name the new file chat-message.proto. Paste the following into this src/chat-message/chat-message.proto

syntax = "proto3";

package ChatMessage;

message Empty { }

message ChatMessage {
  string author = 1;
  string text = 2;
  int64 time = 3;
}

service SendMessageService {
  rpc send (ChatMessage) returns (Empty) {}
}

service AllMessagesService {
  rpc all (Empty) returns (stream ChatMessage) {}
}

This is what protobuf looks like. It is language agnostic. The first line is mandatory and it tells the version of protobuf in use. In this case, "proto3", (the latest version).

message in protobuf refers to a custom type we create. ChatMessage contains an author (a string), the message’s text (string) and when the time the message was sent. int64 is a safe way of representing time as a timestamp.

This protobuf file is not a TypeScript file. By default, NestJS includes all TypeScript files when building the server. If a given NestJS project needs to monitor or include non-TypeScript files into the building process, Nest has to be aware of those files.

The src/chat-message/chat-message.proto above file is an example. To tell Nest about it, add it to the nest-cli.json file. This file is directly inside the fast-chat project folder. Include the following in the JSON file:

  "compilerOptions": {
    "assets": ["**/*.proto"],
    "watchAssets": true
  }

One more config step for gRPC involves adding gRPC as a microservice into NestJS. Delete the contents of src/main.ts file and paste the following:

import { NestFactory } from '@nestjs/core';
import { MicroserviceOptions, Transport } from '@nestjs/microservices';
import { join } from 'path';
import { AppModule } from './app.module';

async function bootstrap() {
  const app = await NestFactory.create(AppModule);
  app.connectMicroservice<MicroserviceOptions>({
    transport: Transport.GRPC,
    options: {
      package: 'ChatMessage',
      protoPath: join(__dirname, 'chat-message/chat-message.proto'),
    },
  });
  await app.startAllMicroservices();
  await app.listen(3000);
  console.log(`Application is running on: ${await app.getUrl()}`);
}
bootstrap();

How to create a service with NestJS

Now that gRPC is properly set up, let's implement our AllMessages and SendMessage services. These services will interact with the ChatMessage type or interface. We need an equivalent of these type in TypeScript (separate from what we declared in protobuf).

Create a new chat-message.interface.ts file inside the chat-message folder. Paste the following inside.

export interface ChatMessage {
  author: string;
  text: string;
  time: number;
}

Next, create a ChatMessagesModule that will contain the AllMessages and SendMessage services. Run the following command to create the module.

nest generate module chat-message

This will create a new chat-message.module.ts file inside the chat-message folder. Given that we will use Memphis broker in the services, import the BrokerModule into this newly generated ChatMessageModule. Delete the contents of the chat-message.module.ts file and paste the following:

import { Module } from '@nestjs/common';
import { BrokerModule } from 'src/broker/broker.module';

@Module({
  imports: [BrokerModule]
})
export class ChatMessageModule {}

gRPC requires that NestJS code implments the methods and services defined in the protobuf file. To implement these services, we will use a controller. A controller in NestJS holds logic for API endpoints. But given that FastChat uses gRPC, the controller will rather hold gRPC functions and services.

Create a new ChatMessageController. Run the following command:

nest generate controller chat-message

It will create a new chat-message.controller.ts file inside the chat-message folder. Replace the contents of this file with the following:

import { Controller } from '@nestjs/common';
import { GrpcMethod } from '@nestjs/microservices';
import { Observable, ReplaySubject } from 'rxjs';
import { BrokerService } from 'src/broker/broker.service';
import { ChatMessage } from './chat-message.interface';

@Controller('chat-message')
export class ChatMessageController {
  private readonly chatMessages$ = new ReplaySubject<ChatMessage>();

  constructor(private brokerService: BrokerService) {}

  onModuleInit(): void {
    this.brokerService.consumer.on('message', (message) => {
      this.chatMessages$.next(
        JSON.parse(message.getData().toString()) as ChatMessage
      );
      message.ack();
    });
  }

  @GrpcMethod('AllMessagesService', 'all')
  all(): Observable<ChatMessage> {
    return this.chatMessages$.asObservable();
  }

  @GrpcMethod('SendMessageService', 'send')
  async send(chatMessage: ChatMessage): Promise<void> {
    await this.brokerService.producer.produce({
      message: Buffer.from(JSON.stringify(chatMessage))
    });
  }
}

ChatMessageController declares a chatMessages$ ReplaySubject. This is an entity that we can push values to and listen to values from.

ChatMessageController obtains the BrokerService via dependency injection. In the OnModuleInit lifecycle hook, we consume “messages” from the BrokerService’s consumer, and emit them to chatMessages$ ReplaySubject. message.ack(); tells Memphis that the “message” is acknowledged (so that it won’t send that “message” again).

ChatMessageController also implements the two communicating services of this real-time app. SendMessageService takes a ChatMessage and produces it to the BrokerService. In turn, AllMessagesService provides as chatMessages$ ReplaySubject as an Observable.

Production and Consumption of broker messages can only be done with strings. This is why we convert ChatMessages to and fro strings when sending and receiving them from the BrokerService’s producer and consumer.

At this point, we’ve successfully setup a simple real-time app with NestJS and Memphis broker.

[Optional]: Test the Real-time chat app with gRPC client

With the way gRPC works, you technically don't make REST API requests. You have to access your backend services from another gRPC SDK (client or server).

We can test the above chat services. One way is to create another gRPC client inside the NestJS project folder. This client will make calls to the services. In turn, you can observe their action in the Memphis Broker UI.

Update the AppModule to import the gRPC ClientsModule. Replace the contents of src/app.module.ts with the following:

import { Module } from '@nestjs/common';
import { ConfigModule } from '@nestjs/config';
import { ClientsModule, Transport } from '@nestjs/microservices';
import { join } from 'path';
import { AppController } from './app.controller';
import { AppService } from './app.service';
import { BrokerModule } from './broker/broker.module';
import { ChatMessageModule } from './chat-message/chat-message.module';

@Module({
  imports: [
    ConfigModule.forRoot({ isGlobal: true }),
    BrokerModule,
    ChatMessageModule,
    ClientsModule.register([
      {
        name: 'CHAT_MESSAGE',
        transport: Transport.GRPC,
        options: {
          package: 'ChatMessage',
          protoPath: join(__dirname, 'chat-message/chat-message.proto')
        }
      }
    ]),
  ],
  controllers: [AppController],
  providers: [AppService]
})
export class AppModule {}

Generate a new ClientController that will call the gRPC methods. Run the following command in the terminal:

nest generate controller client

This will create a new client.controller.ts inside a new client folder. Replace the contents of this new file with the following:

import {
  Body,
  Controller,
  Inject,
  OnModuleInit,
  Post,
  Res
} from '@nestjs/common';
import { ClientGrpc } from '@nestjs/microservices';
import { Response } from 'express';
import { firstValueFrom, Observable } from 'rxjs';
import { ChatMessage } from '../chat-message/chat-message.interface';

interface AllMessagesService {
  all(): Observable<ChatMessage>;
}

interface SendMessageService {
  send(chatMessage: ChatMessage): Observable<void>;
}

@Controller('client')
export class ClientController implements OnModuleInit {
  private allMessagesService: AllMessagesService;
  private sendMessageService: SendMessageService;

  constructor(@Inject('CHAT_MESSAGE') private readonly client: ClientGrpc) {}

  onModuleInit(): void {
    this.allMessagesService =
      this.client.getService<AllMessagesService>('AllMessagesService');
    this.sendMessageService =
      this.client.getService<SendMessageService>('SendMessageService');

    this.allMessagesService.all().subscribe((chatMessage: ChatMessage) => {
      console.log(chatMessage);
    });
  }

  @Post('')
  async sendMessage(
    @Body() chatMessage: ChatMessage,
    @Res() res: Response
  ): Promise<void> {
    await firstValueFrom(this.sendMessageService.send(chatMessage));
    res.json({ status: true, message: 'Acknowledged' });
  }
}

This ClientController subscribes to the all method of the AllMessagesService and prints every ChatMessage to the console. It also exposes a /client POST endpoint to receive messages and send them to Memphis broker (through the send method of SendMessageService).

Run nest start to start the NestJS application.

Use any API tester (cURL, Postman, ThunderClient extension on VS Code, etc.) to send a post request to localhost:3000/client. The body of the post request should reflect a valid ChatMessage like:

{
  "author": "UserA",
  "text": "Test Chat Message",
  "time": 1661096069756
}

You should receive an acknowledgement feedback. Check the Memphis UI at localhost:9000, you should notice the sent message (event). Check the NestJS logs and notice that it printed your ChatMessage. Send more POST requests and notice more messages and logs.

The final codebase for FastChat is here on GitHub.

Why did we use Memphis broker?

In a message broker (like Memphis), messages are like events. They are what services use to communicate.

FastChat above is a simple usecase of a message broker. You might not see the essence of producing and consuming messages if this is the only thing you want to build. Besides we placed the two services in the same .proto file. We also implemented both services with the same controller. These were because of simplicity.

In a big project, you will separate your services. They might be in different NestJS projects and deployed from different servers. Yet, they will be communicating via Memphis broker. If you were building a chat infrastructure, other services like notifications, authentication, security, etc. are reasons why you should use Memphis broker.

You will use a Memphis broker because it ensures many-to-many (N:N) communication across your application’s services. It also ensures that every service receives the events or “messages”.

Summary

Memphis broker stands out in its speed and ease-of-use among other message brokers. You can install it with Docker. It has a UI and CLI through which you can manage your projects. You can also manage factories, stations, and users with Memphis broker.

The above FastChat is a simple real-time app to get you started with Memphis broker in NestJS. In summary, to build a real-time app with NestJS and Memphis broker:

Connect to Memphis broker using the MemphisService and the connection parameters.
Then produce and consume messages as you need in your application.

Happy Coding!

How to build a Flutter Form for managing questions and their answers

Obum — Wed, 19 Jan 2022 20:07:10 +0000

What we will be building ...

A one-screen flutter app containing a form. The form can be used to create/update a question and its options (and the correct option too). This interface is for setting questions that could be used elsewhere like an online quiz. The whole question data can then be saved to a database like Firebase Firestore. The screen looks like this:

Prerequisites

Have basic understanding of flutter and probably programming as a whole.
Have installed flutter and its working.

Steps

1. Setup

Initialize the flutter project by running the following command in your terminal

flutter create questions

2. Coding

a. Open the project in your favorite editor.
b. Delete the contents of lib/main.dart file, paste the following code, and save the file.

3. Run

Run the code with the following command

flutter run

The app should run in your attached device and it should show something similar to the above UI.

Final Thoughts

Flutter is a good framework for mobile development. Developer experience with flutter is smooth and the lines of code are shorter compared to other frameworks.

Forms are an important part of many apps as they help in collecting data. In our flutter app, we used a form to collect data about a question been set/created. This Flutter Question Form may not do much because the data is not been submitted anywhere. But then, we have successfully built a UI that can serve for setting questions that could be saved to some database.

Up Next

We will use this form to build an app that can edit many questions to be set for an online quiz.

DEV Community: Obum

How to improve Flutter Forms

Table of Contents

1. Overall Form UI

2. Form Validation

3. Password Fields

4. Other Input Types

5. Keyboard Types

6. Autofill

7. Disabling, Loading, and Submitting.

Conclusion

How to Restore Poison Messages (or Dead Letters) from Memphis broker vs RabbitMQ

Table of Contents

What is Service Oriented Architecture?

What is a message broker?

About Memphis.dev

About RabbitMQ

What are poison messages?

How to Restore Poison Messages from Memphis

How to Restore Poison Messages from RabbitMQ

Summary

Helpful Resources

How to build a real-time app with NestJS and Memphis broker

Introduction

What is Memphis Broker?

How to set up Memphis broker using Docker

What we will be building

What is NestJS?

How to set up NestJS

How to set up the ConfigModule in NestJS

What is gRPC?

How to set up gRPC in NestJS

How to create a service with NestJS

[Optional]: Test the Real-time chat app with gRPC client

Why did we use Memphis broker?

Summary

How to build a Flutter Form for managing questions and their answers

What we will be building ...

Prerequisites

Steps

1. Setup

2. Coding

3. Run

More on the Question Form

Iteration/Looping

Validation

Feedback to user with SnackBar

Customizing the SnackBar

Final Thoughts

Up Next