DEV Community: Jodamco

Flutter liveness: 300% performance enhance

Jodamco — Wed, 26 Jun 2024 12:28:51 +0000

Stepping forward!!

Hello there again! I'm glad to be here sharing results and great improvements. If you're short of time and only want to know how to improve your app performance I recommend you peek at the code here and check out this Flutter doc about Isolates. On the other hand, if you want to know the full story, I invite you to sit comfortably and drop your comments around.

The UI has changed

In the last article from this series, we ended up with a simple screen with just a Widget live streaming the camera and a custom painter to draw a square where the face was located. I decided to enhance it a bit and provide a few things:

Start and Stop buttons (or anything similar) so I could start and top the process without hot-reloading the app. This was needed mainly because constant camera usage and processing consumes a lot of battery and I had to recharge my phone a few times during the development 😅.
Remove painters and provide something less visually invasive.
Export statistics and data from the face detection layer to the parent so it may decide what to do with it, meaning which widgets to paint or display on top of the camera live stream.

I planned it a bit (the sketch above is only here as proof) and decided to use routes to control start and stop by navigating between the live data stream page and the home page. Also, as a substitute for the square around my face, I decided to use a cube positioned with the angles captured by the face detector.

With some sort of design in hand I started to code the changes and get to the result with few difficulties. The interaction between the face detector layer and its parent was made using callbacks. This made things simpler than using any other state manager with the drawback of making the parent have a specified callback (hence, no drawbacks hehehe). Once I had the data in a different layer I just needed to share it with other children.

For the cube, I used some code from stack overflow 🙃 (you can check the final result here). I code the graph using FlChart (which I found way too complicated for nothing) and the rest of the widgets were made using Text and spacing. The new UI ended up like this

Time to understand the performance-related problems of the app.

Performance: the problem

Just as I finished the refactor of the UI and started to test the app, I noticed I had a "latency" of 700ms to run all the needed steps to generate the data from the live stream. Initially, I had a single Isolate being spaned with Isolate.run() running three main steps:

Parse of the ImageData from camera stream to InputImage, including the conversion from the yuv_420_888 format to n21
Instantiate the faceDetector object since I had problems passing it down from the main Isolate to the spawned one.
Run the detection using the generated data.

This was doing great, but with this huge time I would lose too much head movement and the lack of data could compromise the decision around whether the face being detected is a person or just a photo moving around in front of the camera. Since the first version had some issues I had to work them out.

Performance: first approach

My first thought was: split to conquer, divide the work with more isolates. With this in mind I refactored the code and the core change was refactoring _processImage into this



  Future<void> _runAnalysisPipeline(CameraImageMetaData metaData) async {
    if (_isBusy) return;
    _isBusy = true;

    final (analysisData) = await _processImage(metaData);
    if (analysisData != null && widget.onAnalysisData != null) {
      widget.onAnalysisData!(analysisData);
    }

    _isBusy = false;
  }

  Future<(InputImage, List<Face>)?> _processImage(
    CameraImageMetaData metaData,
  ) async {
    final inputImage = await parseMetaData(metaData);

    if (inputImage == null ||
        inputImage.metadata?.size == null ||
        inputImage.metadata?.rotation == null) return null;

    final faceList = await runFaceDetection(inputImage);
    if (faceList.isEmpty) return null;

    return (inputImage, faceList);
  }

  Future<InputImage?> parseMetaData(CameraImageMetaData metaData) async {
    RootIsolateToken rootIsolateToken = RootIsolateToken.instance!;
    return await Isolate.run<InputImage?>(() async {
      BackgroundIsolateBinaryMessenger.ensureInitialized(rootIsolateToken);

      final inputImage = metaData.inputImageFromCameraImageMetaData(metaData);
      return inputImage;
    });
  }

  Future<List<Face>> runFaceDetection(InputImage inputImage) async {
    RootIsolateToken rootIsolateToken = RootIsolateToken.instance!;
    return await Isolate.run<List<Face>>(() async {
      BackgroundIsolateBinaryMessenger.ensureInitialized(rootIsolateToken);

      final FaceDetector faceDetector = FaceDetector(
        options: FaceDetectorOptions(
          enableContours: true,
          enableLandmarks: true,
          enableTracking: true,
          enableClassification: true
        ),
      );

      final faces = await faceDetector.processImage(inputImage);
      await faceDetector.close();
      return faces;
    });
  }

This code splits the function and uses two Isolates to handle the work. I admit I didn't think this through though. To run the detection I would need the InputImage parsed thus making me wait for it and I also didn't consider the operations related to the spawning and killing of an Isolate. I still think the refactor made it easier to read, but it made the 'latency' go from ~700ms to ~1000ms. It got worse!

To decide what to do next, I did some measurements and discovered I was spending too much time instantiating heavy objects and spawning the isolates, so I decided to get rid of them.

Performance: long-lived Isolates!

Here things get interesting. During my first ever usage of Isolates in this project I went through the docs and chose the simpler approach: Isolate.run. This spawns the Isolate in a way you don't need to handle the communication between the main and the spawned Isolate. To solve my problem I just needed a long-lived Isolate (or worker Isolate). This approach has more complexity but it allows me to create the Isolate when my faceDetection layer is mounted and kill it on dispose, saving me spawn time between detections. Also, I could make it in a way that I was able to instantiate the faceDetection object within the isolate and save its instantiation time as well.

These changes granted me an average processing time (in debug mode) of ~340ms (an enhancement of almost 300%)!!! The responsible for it is the face detector worker that hold all the logic to spawn, maintain and use the Isolate to run the steps I described before.

Results

Let's do a recap of the results we have so far:

Enabled the face detection and live streaming of data
Provided visual feedback of the face Euler angles and facial expressions
Measurement of detection performance

Regarding the performance, we have

Initial average performance of 700ms
Worsening of 42% with first refactor (from 700ms to 1000ms average)
Enhancement of 300% with Isolate worker approach (340ms average)

Solving the detection time bottleneck was extremely necessary once, in the end, we will be performing mathematical operations to decide whether the variation of the angles is real or not. For this to be more accurate, we need more points on time during the tilt of the head in any direction and that's exactly where the time for detection is crucial.

So far, the achievements of this project are small, but the journey has been full of new concepts, techniques, and tools. Regarding the UI, the visual choices provided the opportunity to work with 3D figures, object transformation, and graphs which I didn't have the opportunity to build before using Flutter. As for the performance, to note the necessity and to be able to evolve from one approach to another was a fantastic chance to build up knowledge on how to handle the usage of Isolates with problems related to processing time.

What's next

Now, I think we can start to build the liveness check. I have to re-read the 'base article' I'm using to back up this whole idea and find out what steps I need to determine if a face is a live person or a photo. I'll also try some things related to the variation of the data (if not provided in the article as well) since I got the feeling that part of the parameter is related to the variation itself.

Don't be shy, let your comments out if you have any.

A journey to Flutter liveness (pt1)

Jodamco — Tue, 18 Jun 2024 18:38:48 +0000

Here we are again. This time I decided to write the posts as I go with the project, so it may or may not have an end, for sure it'll not have an order!

Google Machine Learning Kit

I was trying to decide on some Flutter side project to exercise some organizations and concepts from the framework and since AI is at hype I did some research and found out about Google Machine Learning kit which is a set of machine learning tools for different tasks such as face detection, text recognition, document digitalization, among other features (you should really check the link above). They're kinda plug and play, one can just install the plugin dependency and use the capabilities and it doesn't depend on API integrations or third-party accounts, so I decided to move on using it.

For the project itself, I decided to go with liveness - oh boy, if I had done some more research before maybe I would've selected something else - because I got curious about how current tools differentiate between photographs and real people. I have to be honest and say that I didn't do a deep research on the matter and I'll follow the path of reproducing the results I found in this great article. In it the author gets to the conclusion that
the usage of the GMLKit for liveness is feasible and my first goal is to reproduce the Euler Angles graphs, but using a Flutter app. I'm not sure what may be a final casual use for liveness, but I'm sure I'll learn through the process, so let's start!

Flutter app

The startup of a project is always a good thing. You know, follow up the docs for the init, run a flutter create my_app or do it using vscode through the command bar. I'll be using FVM to manage the Flutter version and you can check out the full code here.

Camera Layer

First things first, I needed the camera preview set to get the image data (and to see something at least). For that, I added camera and permission_handler as dependencies to get access to the camera widgets. I also tried to split my camera component in a way that it would become agnostic regarding the machine learning layer, so I could reuse it in different contexts. Here's a small part of the camera widget

class CustomCameraPreview extends StatefulWidget {
  final Function(ImageData inputImage)? onImage;
  final CustomPaint? customPaint;
  final VoidCallback? onCameraFeedReady;

  const CustomCameraPreview({
    super.key,
    this.onImage,
    this.onCameraFeedReady,
    this.customPaint,
  });

  @override
  State<CustomCameraPreview> createState() => _CustomCameraPreviewState();
}

class _CustomCameraPreviewState extends State<CustomCameraPreview> {

//... more code

  Future<void> _startLiveFeed() async {
    if (selectedCamera == null) {
      setState(() {
        hasError = true;
      });
      return;
    }

    _controller = CameraController(
      selectedCamera!,
      ResolutionPreset.high,
      enableAudio: false,
      imageFormatGroup: Platform.isAndroid
          ? ImageFormatGroup.nv21
          : ImageFormatGroup.bgra8888,
    );

    await _controller?.initialize();
    _controller?.startImageStream(_onImage);

    if (widget.onCameraFeedReady != null) {
      widget.onCameraFeedReady!();
    }
  }

//... more code

Widget display() {
    if (isLoading) {
      return PreviewPlaceholder.loadingPreview();
    } else if (hasError) {
      return PreviewPlaceholder.previewError(
        onRetry: _initialize,
      );
    } else if (!hasPermissions) {
      return PreviewPlaceholder.noPermission(
        onAskForPermissions: _initialize,
      );
    } else {
      return Stack(
        fit: StackFit.expand,
        children: <Widget>[
          Center(
            child: CameraPreview(
              _controller!,
              child: widget.customPaint,
            ),
          ),
        ],
      );
    }
  }

  @override
  Widget build(BuildContext context) {
    return Scaffold(
      body: display(),
    );
  }
}

I think the most important part is the startup of the camera live feed. When creating the camera controller you must set the image type using the imageFormatGroup property since this is required for the mlkit plugin to work. The ones on the code above are the ones recommended for each platform and you can check it better on the docs of the face detection plugin. This widget was inspired on the example widget from the official example from the docs.

One great thing I was able to test out was the usage of factories on widgets when I wrote the placeholder for the camera. There were other options, I was suggested to use widget extension and enums, but in the end, I was satisfied with the factory and decided to let it be since it simplified the way the parent was calling the placeholder.


enum PreviewType { permission, loading, error }

class PreviewPlaceholder extends StatelessWidget {
  final PreviewType type;
  final VoidCallback? onAction;

  const PreviewPlaceholder._({
    required this.type,
    this.onAction,
  });

  factory PreviewPlaceholder.noPermission({
    required VoidCallback onAskForPermissions,
  }) =>
      PreviewPlaceholder._(
        type: PreviewType.permission,
        onAction: onAskForPermissions,
      );

  factory PreviewPlaceholder.loadingPreview() => const PreviewPlaceholder._(
        type: PreviewType.loading,
      );

  factory PreviewPlaceholder.previewError({required VoidCallback onRetry}) =>
      PreviewPlaceholder._(
        type: PreviewType.error,
        onAction: onRetry,
      );

  @override
  Widget build(BuildContext context) {
    return Column(
      mainAxisAlignment: MainAxisAlignment.center,
      children: [
        if (type == PreviewType.permission)
          ElevatedButton(
            onPressed: onAction,
            child: const Text("Ask for camera permisions"),
          ),
        if (type == PreviewType.error) ...[
          const Text("Couldn't load camera preview"),
          ElevatedButton(
            onPressed: onAction,
            child: const Text("Ask for camera permisions"),
          ),
        ],
        if (type == PreviewType.loading) ...const [
          Text("Loading preview"),
          Center(
            child: LinearProgressIndicator(),
          )
        ],
      ],
    );
  }
}

With the camera layer done, let's dive into face detection.

Face detection

For the face detection, so far, I just needed to add two more dependencies: google_mlkit_commons and google_mlkit_face_detection. The docs of the GMLKit recommend using the specific plugin dependency for release builds instead of the Flutter GMLKit dependency.

If you ~~copy~~ write your first ever approach to face detection, it can be very straightforward to reach data and have the results, unless by one problem: if you're using android and android-camerax plugin, you will not be able use the camera image with face detection. This is because although you must've set ImageFormatGroup.nv21 as the output format, the current version of the flutter android-camerax plugin will only provide images using the yuv_420_888 format (you may find more info here). The good part is that someone provided a solution (community always rocks 🚀).

I set the detection widget as my main "layer" for the detection since it does the heavy job of running the face detection from the GMLKit plugin. It ended up being a very small widget with a core function for face detection

 Future<void> _processImage(ImageData imageData) async {
    if (_isBusy) return;
    _isBusy = true;

    RootIsolateToken rootIsolateToken = RootIsolateToken.instance!;
    final analyticData = await Isolate.run<Map<String, dynamic>>(() async {
      BackgroundIsolateBinaryMessenger.ensureInitialized(rootIsolateToken);

      final inputImage = imageData.inputImageFromCameraImage(imageData);
      if (inputImage == null) return {"faces": null, "image": inputImage};

      final FaceDetector faceDetector = FaceDetector(
        options: FaceDetectorOptions(
          enableContours: true,
          enableLandmarks: true,
        ),
      );

      final faces = await faceDetector.processImage(inputImage);
      await faceDetector.close();

      return {"faces": faces, "image": inputImage};
    });

    _isBusy = false;
  }

A few comments on this function:

It is VERY SIMPLE to get the data from the GMLKit and it can be done without the Isolate.
Although the isolate is not needed you might want to use it since Flutter code should build in 16ms. I was eager to try out Isolates and never had a real good reason, but without it the processing of the image would drop the framerate and the app look would become terrible. By applying the Isolate I can remove all the processing and conversion from the main event loop and guarantee that the frames will be built on time.
I decided to have the face detector instantiated inside the Isolate since I had trouble passing it out from the main isolate to the new one. I also had this specific conversion imageData.inputImageFromCameraImage(imageData) done inside the isolate since it is also time-consuming. This is what allows me to parse the yuv_420_888 format into the one needed for the GMLKit plugin. For this job, I decided that the best approach was to use a class to receive all the data from the camera and smoothly provide the InputImage object for the GMLKit. You can check out the class here and the extension for the conversion here.

Results

So far I still don't have the Euler Angles in a graph as I wanted but I was able to at least get the data from the kit and paint out the bounding box of my face. I also did some tests regarding the execution time of the face detection and could see that the average time to execute the detection for a high-quality image is about 600ms with a debug build and about 380ms with a release build. Since I have the Isolate the framerate of the app is running ok but I would like to enhance this performance later.

My next step will be to get the Euler Angles and paint a graph with them so I can try to reproduce the comparison between photos and real people.

See you there!

Controlling user auth flow with Lambda & Cognito (pt2)

Jodamco — Mon, 03 Jun 2024 23:06:11 +0000

Last post, we wrote the code for our preAuth trigger that would handle the count of attempts to login. The idea now is to reset the counter after the login is successful since we want all the atempts to be available when the user come to login for another session.

The code for the postAuth trigger is way simpler than the one for the preAuth. Let's dive into it

module.exports.postAuthTrigger = async (event) => {
    await this.clearLoginAttempts(event)
    return event
}

exports.clearLoginAttempts = async (event) => {
    const updateParams = {
        UserAttributes: [
            {
                Name: 'custom:login_attempts',
                Value: '0',
            },
        ],
        UserPoolId: event.userPoolId,
        Username: event.userName,
    }

    await cognitoService.adminUpdateUserAttributes(updateParams).promise()
}

To reset the login attempts, we just need to update the same counter we used in the preAuth trigger, which is stored into the prop 'custom:login_attempts'. We can do it by calling the adminUpdateUserAttributes function from the cognito API.

One other important thing to mention is that we need to return the 'event' object that we receive when the lambda is called since cognito will expect to have it to continue with the auth flow. After creating the lambdas, we need to setup the cognito accordingly with the needed property (login_attempts) and the setup for the triggers.

First you may need to create your user pool. To do so you start by signing in to the AWS Management Console and navigating to Amazon Cognito. Click "Manage User Pools" and then "Create a user pool". Along the wizard to create the user pool you will notice an option to create custom attributes. Here you will create your 'login_attempts' attribute. You will notice that even though we set the name as 'login_attempts', cognito will ask you to access it by calling 'custom:login_attempts'. That's the default for custom attributes on cognito

To finish the creation of the user pool, follow the steps and after review your configurations and click "Create pool."

Now, you just simply need to attach the created lambdas to your cognito. Open the created user pool and search for 'User pool properties', there you'll find the lambda trigger setup. Click on 'Add lambda trigger'.

You will notice a few different types of triggers. We will use Authentication triggers. Select the Authentication option, the preAuth trigger and attach the preAuthTrigger lambda you created. Then repeat the process but to attach the postAuthTrigger lambda to postAuth trigger.

And it's done! Now you have a cognito user pool setup to block user after n unsuccessful attempts of login! To test it out, you may integrate cognito with some web or mobile app using the Amplify SDK. You can also use this same triggers to add other features, such as saving last login from the user or triggering other services after the login.

Controlling user auth flow with Lambda & Cognito

Jodamco — Sat, 25 May 2024 20:05:47 +0000

Disclaimer: the hero image of this post was the result of the following prompt AWS lambda and AWS cognito logos into a Renaissance paint. Use full logos and a less known painting. I think I still have much to learn into AI image prompts 😅😅

Authentication is a common topic between many kinds of systems. There are different ways to handle it and my preferred ones make usage of managed services. I found AWS Cognito a really great solution to handle authentication speacially if you are later connecting the authenticated app with other hosted services. Cognito will provide you built in ways to manage and cross validate users against services and recently I've been using it's hooks to build even more complex auth features

Cognito triggers

Cognito user pools have a feature named 'Lambda triggers' which let's you use previously created Lambdas to perform custom actions during four types of flow:

Sign up
Authentication
Custom authentication (such as CAPTCHA or security questions)
Messaging

Each of these flows have different triggers that will execute lambda code in between specific steps of the flow. Sign up for instance has Pre sign-up trigger, Post confirmation trigger and Migrate user trigger that can be attached to a Lambda function.

To test the capacities of Lambda triggers, we will develop a system that prevents login after 5 consecutive failures.

Coding the lambdas

We're gonna need two lambdas to make the flow controll, one of them would take care of updating the user data so we may count how many times the user tryed login. This one will also block the user if the number of attempts exceeds the maximumm. The second one, will be used reset our counter, so in the future the user will still have the maximumm number of attempts left.

The first lambda trigger would be like this

module.exports.preAuthTrigger = async (event) => {
    if (!(await this.isUserEnabled(event))) throw new Error('Usuário Bloqueado')

    const attempts = await this.getLoginAttempts(event)
    if (attempts > 4) {
        await this.disableUser(event)
        throw new Error('Usuário Bloqueado')
    }

    await this.updateLoginAttempts(event, attempts)
    return event
}

Our first step is to check whether the user is already blocked by the amount of attempts. We can do it with a separate fn:

exports.isUserEnabled = async (event) => {
    const getParams = {
        UserPoolId: event.userPoolId,
        Username: event.userName,
    }
    const userData = await cognitoService.adminGetUser(getParams).promise()
    return userData.Enabled
}

With this we are accessing the properties of the user on the cognito user pool and checking out the Enable property that dictates if the user is able to user it's username and password to login. A disabled user can't login into a cognito pool and that's exactly we want here.

For the second step, we need to check if the number of attempts is greater than the max permitted.

exports.getLoginAttempts = async (event) => {
    const getParams = {
        UserPoolId: event.userPoolId,
        Username: event.userName,
    }
    const userData = await cognitoService.adminGetUser(getParams).promise()
    const attribute = userData.UserAttributes.find(
        (att) => att.Name === 'custom:attempts'
    )
    if (attribute !== undefined && attribute !== null)
        return parseInt(attribute.Value)
    else return 0
}

It is a very simmilar process to the previous fn, but now we're looking for a custom attribute named custom:attempts that we will create into our user pool in the next steps. If the user has more than 5 attempts (we start counting at 0), then we should block the user. Piece of cake:

exports.disableUser = async (event) => {
    await cognitoService
        .adminDisableUser({
            UserPoolId: event.userPoolId,
            Username: event.userName,
        })
        .promise()
}

We also have to throw an Error and stop executing the lambda since this will make the login process fail as we want. Now that we are able to block the user, we just need to update the number of attempts if it isn't blocked:

exports.updateLoginAttempts = async (event, attempts) => {
    const updateParams = {
        UserAttributes: [
            {
                Name: 'custom:login_attempts',
                Value: (attempts + 1).toString(),
            },
        ],
        UserPoolId: event.userPoolId,
        Username: event.userName,
    }

    await cognitoService.adminUpdateUserAttributes(updateParams).promise()
}

This last function sets everything for the first lambda trigger. Now we are able to perform all the actions from our main lambda function. The final code with all functions will be like this:

module.exports.preAuthTrigger = async (event) => {
    if (!(await this.isUserEnabled(event))) throw new Error('Usuário Bloqueado')

    const attempts = await this.getLoginAttempts(event)
    if (attempts > 4) {
        await this.disableUser(event)
        throw new Error('Usuário Bloqueado')
    }

    await this.updateLoginAttempts(event, attempts)
    return event
}

exports.isUserEnabled = async (event) => {
    const getParams = {
        UserPoolId: event.userPoolId,
        Username: event.userName,
    }
    const userData = await cognitoService.adminGetUser(getParams).promise()
    return userData.Enabled
}

exports.getLoginAttempts = async (event) => {
    const getParams = {
        UserPoolId: event.userPoolId,
        Username: event.userName,
    }
    const userData = await cognitoService.adminGetUser(getParams).promise()
    const attribute = userData.UserAttributes.find(
        (att) => att.Name === 'custom:login_attempts'
    )
    if (attribute !== undefined && attribute !== null)
        return parseInt(attribute.Value)
    else return 0
}


exports.disableUser = async (event) => {
    await cognitoService
        .adminDisableUser({
            UserPoolId: event.userPoolId,
            Username: event.userName,
        })
        .promise()
}


exports.updateLoginAttempts = async (event, attempts) => {
    const updateParams = {
        UserAttributes: [
            {
                Name: 'custom:login_attempts',
                Value: (attempts + 1).toString(),
            },
        ],
        UserPoolId: event.userPoolId,
        Username: event.userName,
    }

    await cognitoService.adminUpdateUserAttributes(updateParams).promise()
}

In my next post we will write the code for the PostAuth lambda trigger and see how can we setup cognito to use both lambdas!

Angular code obfuscation made easy

Jodamco — Fri, 24 May 2024 16:56:17 +0000

If you ever had to code a real-life project, the concern for security was there—or at least should have been. As technologies advance, we can code amazing, robust, high-performance systems within short time schedules, but that also means that malicious people and techniques become more powerful and tricky to overcome. That's why nowadays securing all common breaches is a must when developing systems.

Angular handles a lot of security out of the box: it has its own variable protection system and sanitization to prevent malicious code from running in your app. Another feature is code minification.

Minification vs. Obfuscation

Code minification is a technique that reduces the size of source code by removing unnecessary characters like whitespace and comments, improving the load performance of the source code. This process is common in web development for JavaScript, CSS, and HTML files and, somehow, adds a layer of security by obfuscating the code. Minified code is extremely hard to read, and that's why it is considered some sort of obfuscation. However, tools can de-minify code, making it readable and then reverse-engineerable. This is where obfuscation is useful.

Complementary to minification, code obfuscation is a technique used to make source code difficult to understand and reverse-engineer. This is often used to protect intellectual property, prevent tampering, and deter reverse engineering by making it challenging for attackers to understand the code's logic and identify potential vulnerabilities. It transforms readable code into a more complex and obscure version without altering its functionality. Code obfuscation tools can also add dead code to mislead attackers and make it even more difficult to understand the software codebase.

Well, if you have use for it, let's obfuscate our Angular app.

Webpack Obfuscator

Angular uses Webpack during its bundle phase and has its own default setup to pack the modules you develop. We are going to take advantage of this and customize the way Webpack will bundle your Angular app. First, install these packages:

npm i javascript-obfuscator webpack-obfuscator --save-dev

The javascript-obfuscator is

a powerful free obfuscator for JavaScript, containing a variety of features which provide protection for your source code.

while webpack-obfuscator makes use of it as a plugin to provide functionality for Webpack. You can find the JavaScript obfuscator code here and Webpack obfuscator plugin here.

After that, create a custom-webpack.config.js file that will contain the custom configurations we want to apply during our bundle process. Here's a simple one:

var JavaScriptObfuscator = require("webpack-obfuscator");

module.exports = {
  module: {},
  plugins: [
    new JavaScriptObfuscator(
      {
        debugProtection: true,
      },
      ["vendor.js"]
    ),
  ],
};

There are many different config options you can provide for the webpack-obfuscator plugin to fine-tune the output of the obfuscation. This is the simplest one that adds debugProtection to the code, making it difficult to use the console to track down variables and functions of the app.

So far, we set up our config of Webpack. Now we need to use it. We will need one more dependency:

npm i @angular-builders/custom-webpack --save-dev

This will help us integrate the custom Webpack builder with Angular so we can still use the Angular build structure. After installing the package, we only need to change the angular.json file. Search for the build property and add the following:

... 
"builder": "@angular-builders/custom-webpack:browser",
"customWebpackConfig": {
    "path": "./custom-webpack.config.js",
    "replaceDuplicatePlugins": true
}

By replacing the builder from @angular-devkit/build-angular:browser to @angular-builders/custom-webpack:browser, we will still be able to build for the browser but now can inject our custom Webpack configurations. The customWebpackConfig property sets the reference for the file so Angular can use it.

If everything is properly set, your build command should run normally and the result will be an obfuscated Angular app!

Drawbacks

Be aware, though, that this approach has a drawback on the bundle size. Code obfuscation makes it much more difficult to reverse-engineer the code, but the way it declares the variables uses more characters, leading to an increase in the size of the bundle—almost going in the opposite direction of code minification.

That's it. Be sure to use it with purpose and understand how to tackle the drawbacks of the technique!

Coding the design: Angular & user decision making flow

Jodamco — Mon, 20 May 2024 20:23:55 +0000

I really like design, both visual and usefull. I am a human being driven by beauty and I think that combining it with our daily lives and tasks is what differentiates greatness and uniqueness from pure raw purpose.

There's also beauty in seamless

The effort to make something so usefull and integrated to one's life that it goes unnoticed even when life can't be imagined without it. To achieve this outcome, countless designers spend their time studying both product and user to simplify decision making processes and create balance into complex-simple user flows.

That said, if you are not a designer that's mostly not for you to care, but often we face situations where we are the ones deciding the user flow on front end applications and taking care of the decision making process. That's not easy, but there's theory and that's what I want to tell you about.

Seven Stages of Action

Donald Norman showed us in the past decades that the ones decision making process has 7 steps he called 'Seven stages of action'. Those are:

Forming the goal (What do I want?)
Forming the intention (How can I get it?)
Specifying an action (If I do 'x' then I can get it)
Executing the action (Do 'x')
Perceiving the state of the world (What happened after my actions?)
Interpreting the state of the world (What's the meaning of what happened?)
Evaluating the outcome (Is it what I wanted?)

These steps also come with the cycle of action:

Making it symple, whenever the user wants to do something it'll follow the same cycle of

evaluate the world
decide what is wanted and
act on it to achieve what is wanted

The best applications we use on our daily basis have mastered techniques to diminish the cognitive load on identifying the current state and deciding what to do to achieve an outcome. This is a job itself and requires experience and research, but I came out with a simple rule of thumb to achieve great results even in small systems

Let your user know what is going on

Quite dumb right? I'm gonna demonstrate what I mean, so let's write a few lines of code

Coding user interaction

Regardless of the techonology, we all run some sort of state management to controll user interaction, display of data and feature workflow. State is about what's going on now, the present of the application, the data it has and what's being displayed on the screen. If we want to let the user know what is going on then we need to be clear about the component state. Usually, a simple component with data will have something between 3 to 5 states:

idle or initial
empty
loading
error
data

Let's code each one of them. Consider a simple Angular component

@Component({
    selector: 'app-dummy-component',
    standalone: true,
    template: `<ul> 
                       <li *ngFor="let item of list">
                           {{item.name}}
                       </li>
                   </ul>

                   <button (click)="loadList()" >
                       Load Items
                   </button>`,
})
export class DummyComponent {
    public list: any[] = []
    constructor() {}
    public loadList(){ ... }
}

This would be the idle/initial state and this also happens to be the data state since whenever data is available it will appear. To display a list of items can be as simple as this but at the same time it is a good practice to consider

what happens if the list is empty?
what happens if I am not able to load the list?
how can the user know if the load is still going on?

These questions try to fill the gaps in the decision making process of the user since whenever the cycle of action starts the component has to follow up to new states giving feedback on the actions taken by the user. Let's add some states to our component

@Component({
    selector: 'app-dummy-component',
    standalone: true,
    template: `<ul *ngIf="list.length > 0; else listPlaceholder> 
                       <li *ngFor="let item of list">
                           {{item.name}}
                       </li>
                   </ul>

                   <ngTemplate #listPlaceholder>
                       <h4> You have no items to display </h4>
                   </ngTemplate>

                   <button (click)="loadList()" >
                       <p *ngIf="isLoading; else loadPlaceholder>
                           Load Items
                       </p>
                       <ngTemplate #loadPlaceholder>
                           ... Loading items ...
                       </ngTemplate>
                   </button>`,
})
export class DummyComponent {
    public list: any[] = []
    public isLoading: boolean = false
    constructor() {}
    public loadList(){ 
        if(this.isLoading) return
        this.isLoading = true
        ... // load the list
        this.isLoading = false
    }
}

Our updated component is now capable of giving feedback on two new occasions:

Empty state: now we are able to let the user know if the list is empty. This brings clearence to what is being displayed since now we're able to differentiate an empty list from a failure of the system
Loading state: displaying any kind of loader gives instant feedback for the users regarding their actions. Whenever the user acts on a screen, something must change and by having the loading feedback the user will be able perceive a change to trigger a new cycle of action.

Let's add some error handling to our simple component list:

@Component({
    selector: 'app-dummy-component',
    standalone: true,
    template: `<ul *ngIf="!error && list.length > 0; else listPlaceholder> 
                       <li *ngFor="let item of list">
                           {{item.name}}
                       </li>
                   </ul>

                   <ngTemplate #listPlaceholder>
                       <h4> You have no items to display </h4>
                   </ngTemplate>

                   <h4 *ngIf="error">{{error}}</h4>

                   <button (click)="loadList()" >
                       <p *ngIf="isLoading; else loadPlaceholder>
                           Load Items
                       </p>
                       <ngTemplate #loadPlaceholder>
                           ... Loading items ...
                       </ngTemplate>
                   </button>`,
})
export class DummyComponent {
    public list: any[] = []
    public isLoading: boolean = false
    public error: String|undefined = undefined
    constructor() {}
    public loadList(){
        try{
            if(this.isLoading) return
            this.isLoading = true
            ... // load the list
            this.isLoading = false
        }catch(error){
            this.error = 'Error while loading the list'
        } 
    }
}

Now we are also able to handle the error state on our component.

By caring for a component state and cycle of action we end up increasing it's complexity since we will need new variables and conditions to handle all the things properly, but by doing it we also achieve a more stable and robust component that's able to handle different situations accordingly. Also, the component will be able to fully respond to the user's interactions providing feedback in a fluid cycle of action.

That's mostly it. Keep in mind that not all components will have all states though. Don't go for the number of states but for the clarity and always try to let your user know what is going on. The more you care with it in your smaller components the more you whole app will be able to cover the gaps on decisory process and become seamless into your user life.

Validating the right way: API gateway with JsonSchema

Jodamco — Sun, 19 May 2024 19:04:34 +0000

We all know input validation, right? Tons of ways on doing it and it saves us a lot of time preventing troubles of many types. There are different types of validation and for different use cases we may use different approachs. We may validade inputs with some requirements, custom functions and, whenever using #Angular we are able to validate things through some high level Validation function from Reactive Forms module.

Well, before start with Backend i always got myself thinking about how to validate the body of a REST requisition. Don't get me wrong, I can for sure guarantee that things will be sent fine, but always wandered if the backend function receiving the data would have to parse it back and validate the same way I do when the front end receive data. Some time passed, I started to do backend and had to come up with a solution and since we used API Gateways, things were right next.

When I was getting things done and understanding stuff propperly with serverless and API Gateway, I used to validate inputs like this:



module.exports.myAPIPostFn = async (event) => {
    const { prop1, prop2, prop3 } = event.body
    if(notNull(prop1) || notNull(prop2) || notNull(prop3))
        throw new Error('Error [400]: invalid body')

    ... // code continues...
}

It doesn't sit right, you see? By doing that, I feel like i'm making the function responsible for it's input integrity, while it should receive things right in first place. Of course that if I want to be sure no side effects would occour I could try and use some classes and models for it also, but it also didn't fit with the way things were being done.

If the caller was the responsible, then I should try and find something to validate things at it's side. That's when I found the input validation with JSON Schema. As stated in its official website

JSON Schema is the vocabulary that enables JSON data consistency, validity, and interoperability at scale

See? Just what I was looking for. It has a set of rules that one can use in a very declarative way to define what a JSON should look like. A JSON schema would look like this



{
    "$schema": "http://json-schema.org/draft-04/schema#",
    "type": "object",
    "required": [
        "name",
        "surName",
        "age",
        "email",
        "phone",
        "isMarried"
    ],
    "properties": {
        "name": { "type": "string" },
        "surName": { "type": "string" },
        "age": { "type": "integer" },
        "email": { "type": "string" },
        "phone": { "type": "string" },
        "isMarried": { "type": "boolean" }
    },
    "additionalProperties": false
}

With this schema I am able to define the object's properties with name, type and also prevent additional props from comming with it. The best part of it, when using serverless with API Gateway, is that the integration is as symple as including a line of code (actually, 3 if you count linke breaks):



newClient:
    handler: src/functions/client/handler.newClient
    events:
        - http:
              path: client
              method: put
              integration: LAMBDA
              authorizer: ${self:custom.authorizerConfig}
              **request:
                  schemas:
                      application/json: ${file(schemas/new-client.json)}**

When using serverless you can provide several configs for your event type, and when it comes to using HTTP event type you are also able to define a schema for the request. When the schema is defined, the serveless will setup the API in a way that it uses the provided schema to validate the body of the request being received. This will ensure that when the lambda function executes, the event.body will have all the desired properties and, if they don't, the API Gateway will gracefully respond with Invalid request body without ever call the lambda.

In the end I got quite happy with this approach since it brought me some advantages:

Cleaner code (we all love it)
Better controll over contract between front and backend without changing the way the backend works (AKA not implementing models)
Close one more security breach, since now I am able to prevent some malicious inputs
Cost saving: I no longer have to execute lambda code to notice that inputs are wrong.

That's it for now. Let me know in the comment section if you would have done differently. I am also looking for some simmilar tool to validate data comming from path and query parameters, if you happen to know I would be glad to hear!