DEV Community: Matija Kovaček

Importance of Code reviews

Matija Kovaček — Wed, 12 Jan 2022 19:00:00 +0000

Inspired by some of the last few projects, I have noticed that still a lot of people don't consider Code Review seriously.

From extreme cases where code review doesn't exist at all or where pull/merge requests are approved in a few seconds after being opened. Over the cases where the minority of team members care and perform code reviews, and the rest of the team members ignore them. The funny thing is when they are explicitly asked to check it, they consider them as "punishment".

Luckily, one of my first projects had a really good code review process where all team members were very active and performed strict but fair code reviews. On that project, I learned a lot, learned why this process is so important and gained really good habits on how to perform code reviews.

That's probably one of the reasons why I'm writing this today.

Recently I was preparing guidelines and best practices about code reviews so I decided to share them with you as well.

The things you will see here are not only my statements, they are inspired by other people, teams, and companies. But with those statements, I totally agree and I stand behind them.

1. So what is code review?

A code review is a process where someone other than the author(s) of a piece of code examines that code. Code review should be used to maintain the quality of our code and products.

2. Process Flow

Once you're satisfied and proud with your implementation and the tests, you create a pull request. With a pull request, you are requesting the merge of your local branch into the main source branch.

All dev team members (including Leads / Architects) should be default reviewers of the pull request. The reviewer's job is to check if all requested functionalities are properly implemented and tested. She/He can suggest code improvements like using some library instead of reinventing the wheel, better naming for components and variables, and any other improvement that she/he can suggest.

It is important that you shouldn't be disappointed if you receive a lot of comments on your pull requests. You will learn from the feedback and the solution will be better.

When the minimum required number of reviewers have approved the pull request you can merge it. Don’t forget to delete your source (feature/bugfix) branch and move the ticket that is ready for QA (after it's deployed).

3. Author of the pull request

Goal : Learn, improve your code quality, morale, and your working relationships.

3.1. REVIEW your own code first

Before sending code to your teammate, read it yourself and check the things that you would check as a code reviewer

3.2. Be UP TO DATE with the develop branch state

The recommendation is always to be up to date with the latest state from develop branch. This means that during the day you should pull changes several times and merge it to your local feature branch

Before creating a pull request make sure that your feature/bugfix branch is up to date

3.3. Pull request should be DESCRIPTIVE

Branch Name
- Source branch name should consist of branch type (feature, bugfix, hotfix), Jira ticket number, and Jira ticket title
- e.g feature/ticket123-some-ticket-title
Commit messages
- Commit early, commit often with a descriptive message what you did
Pull request description
- Add an additional summary of what is being done if it’s not clear from the pull request title
- Point out not so obvious/logical decisions and agreements
Comment your decisions
- Comment yourself (inline in code) for not so obvious/logical decisions, so that reviewers can easily understand it

3.4 Prefer SMALLER pull requests

Try to make your pull request small as possible.

Smaller pull requests:

easier to review and understand
faster to review
less chance to miss something
less chance to create a bug

3.5. Respond graciously to feedback and LEARN from it

Don’t take personally a code review feedback. Treat your reviewer’s notes as an objective discussion about the code.

3.6. Be PATIENT when your reviewer is wrong

From time to time, reviewers are wrong. Just as you can accidentally write buggy code, your reviewer can misunderstand the correct code.

3.7. COMMUNICATE your responses explicitly

For every comment that requires action, respond explicitly to confirm that you’ve addressed it. Some code review tools allow you to mark comments as resolved. Otherwise, follow a simple convention, like, "Done", for each note. If you disagree with the note, politely explain why you declined to take action.

Don’t directly ask/chat with the reviewer about every comment. Commenting and discussing directly in the pull request is useful for other reviewers as well.

3.8. MINIMIZE lag between rounds of review

Once you create a pull request, driving the review to completion should be your highest priority. Delays on your end, will waste time for your reviewer, and increase complexity for your whole team.

3.9. RESOLVE Conflicts

In case of:

disagreements
totally wrongly implemented features
too many mistakes
doubts, unclear things, or comments

schedule 1 on 1 call with the reviewer to resolve all doubts and unclear things. Make sure that you comment back agreements directly in a pull request for other reviewers.

4. Code reviewers

Goal: Mentor, educate and improve your working relationships.

4.1. What do Code Reviewers look for

Design
- Is the code well-designed and appropriate for your system?
Functionality
- Does the code behave as the author likely intended?
- Is the way the code behaves good for its users (end-users and developers)?
Complexity
- Could the code be made simpler?
- Would another developer be able to easily understand and use this code when they come across it in the future?
Tests
- Does the code have correct and well-designed tests?
- Do the tests cover requirements?
Naming
- Did the developer choose clear names for variables, classes, methods?
Comments
- Are the comments clear and useful?
Style
- Does the code follow the company style guide?
Documentation
- Did the developer update relevant documentation?

4.2. Don’t criticize, MENTOR and EDUCATE

Code review is a chance for developers to mentor and teach others.

When you comment on someone's pull request:

be kind
instead of criticizing, ask why decisions were made like that
suggest improvements
explain your reasoning
balance giving explicit directions with just pointing out problems and letting the developer decide

4.3. PRAISE good code

Remember to praise good code and decisions. Don't only point out problems and suggestions.

4.4. Don’t ignore pull requests

Don’t ignore pull requests and think someone instead of you will do the review.

Make a habit of doing code reviews like in the morning before daily, around lunchtime, before the end of the working day.

If you are in the middle of a focused task, such as writing code, don’t interrupt yourself to do a code review. Instead, wait for a breakpoint in your work before you respond to a request for review.

4.5. Less experienced team members don’t be shy

Your opinion is important and expected here as well. More experienced team members don't know everything and they also make mistakes.

5. Speed of Code Reviews

The speed of individual development is important, but not as important as the speed of the entire team.

The speed of the team as a whole will be decreased if the individuals don’t respond to the pull requests, do the review, and push other work (task) in the done direction.

If you are not in the middle of a focused task, you should do a code review shortly after it comes in.

6. Code Review Improvements Over Time

If you follow these guidelines and you are strict with your code reviews, you should find that the entire code review process tends to go faster and faster over time.

Developers will learn what is required for healthy code, and when you create a pull request that is great from the start, it will require less and less review time.

Reviewers will learn to respond quickly and not add unnecessary latency into the review process.

But don’t compromise on the code review standards or quality for an imagined improvement in velocity—it’s not actually going to make anything happen more quickly, in the long run.

References:

REST API Integration with Feign HTTP client

Matija Kovaček — Sat, 26 Jun 2021 08:00:00 +0000

1. Introduction

In projects we often have some integrations with API layer of 3rd party systems. This means that on our side we need to prepare HTTP client to be able consume those endpoints. Based on API specification we need to prepare Resource representation classes. Then, HTTP client with some configurations, like headers, timeouts, error handling. Parse request and response payloads into objects and many other things. All off this on the end creates a lot of boilerplate code.

Inspired with my latest project I want to introduce Feign HTTP client. Feign is HTTP client which goal is to simplify writing HTTP clients. Goal is to reduce complexity of REST API integrations.

With Feign we need to declare and annotate an interface based on API specification. And in background Feign will process annotations into a templated requests. Feign offers you to configure different HTTP clients, JSON/XML processors, metric providers, loggers and some other features.

2. Example

Throughout this article, we will integrate Faker APIs in Adobe Experience Manager (AEM) project. Goal is to get some fake generated data about users and addresses.

3. Setup

First, in your AEM project add needed dependencies.

<!-- Feign -->
<dependency>
  <groupId>io.github.openfeign</groupId>
  <artifactId>feign-core</artifactId>
  <version>11.0</version>
</dependency>
<dependency>
  <groupId>io.github.openfeign</groupId>
  <artifactId>feign-jackson</artifactId>
  <version>11.0</version>
</dependency>
<!--Jackson -->
<dependency>
  <groupId>com.fasterxml.jackson.core</groupId>
  <artifactId>jackson-core</artifactId>
  <version>2.12.0</version>
</dependency>
<dependency>
  <groupId>com.fasterxml.jackson.core</groupId>
  <artifactId>jackson-annotations</artifactId>
  <version>2.12.0</version>
</dependency>
<dependency>
  <groupId>com.fasterxml.jackson.core</groupId>
  <artifactId>jackson-databind</artifactId>
  <version>2.12.0</version>
</dependency>

Besides the feign-core , additionaly we'll use feign-jackson for JSON processing. _ Feign-jackson _ requires jackson-core , jackson-annotations and _ jackson-databind _ dependencies.

To prevent unresolved import packages, we need to install those dependencies in Apache Felix Web console or embed them into our project bundle.

Embed dependencies in _ filevault-package-maven-plugin _ configuration.

<embedded>
  <groupId>io.github.openfeign</groupId>
  <artifactId>feign-core</artifactId>
  <target>/apps/aem-feign-vendor-packages/application/install</target>
</embedded>
<embedded>
  <groupId>io.github.openfeign</groupId>
  <artifactId>feign-jackson</artifactId>
  <target>/apps/aem-feign-vendor-packages/application/install</target>
</embedded>
<embedded>
  <groupId>com.fasterxml.jackson.core</groupId>
  <artifactId>jackson-core</artifactId>
  <target>/apps/aem-feign-vendor-packages/application/install</target>
</embedded>
<embedded>
  <groupId>com.fasterxml.jackson.core</groupId>
  <artifactId>jackson-annotations</artifactId>
  <target>/apps/aem-feign-vendor-packages/application/install</target>
</embedded>
<embedded>
  <groupId>com.fasterxml.jackson.core</groupId>
  <artifactId>jackson-databind</artifactId>
  <target>/apps/aem-feign-vendor-packages/application/install</target>
</embedded>

4. Feign HTTP client

First of all we need to prepare Resource representation classes.

@Data
public class Users {

    private String uuid;
    private String firstname;
    private String lastname;
    private String username;
    private String password;
    private String email;
    private String ip;
    private String macAddress;
    private String website;
    private String image;

}

@Data
public class Address {

    private String street;
    private String streetName;
    private String buildingNumber;
    private String city;
    private String zipcode;
    private String country;

    @JsonProperty("county_code")
    private String countyCode;

    private Double latitude;
    private Double longitude;

}

@Data
public class Response<T> {

    private String status;
    private int code;
    private long total;
    private List<T> data;

}

Next step is to define FakerApi client which follows Faker endpoints specification. The @RequestLine annotation defines HTTP method and UriTemplate for request. Expressions wrapped in curly-braces are resolved using their corresponding @param annotated parameters.

public interface FakerApi {

    @RequestLine("GET /users?_quantity={quantity}&_gender={gender}")
    Response<Users> users(@Param("quantity") long quantity, @Param("gender") String gender);

    @RequestLine("GET /addresses?_quantity={quantity}")
    Response<Address> addresses(@Param("quantity") long quantity);

}

Last thing is to create FakerApi client and that's all.

FakerApi fakerApi = Feign.builder()
                     .decoder(new JacksonDecoder())
                     .target(FakerApi.class, "https://fakerapi.it/api/v1");

Response<Address> address = fakerApi.addresses(2);
Response<Users> femaleUsers = fakerApi.users(2, "female");

5. Conclusion

In this article, we've explained how to simplify REST API Integrations in AEM with Feign HTTP client.

All code samples are available on Github.

For more details about Feign check official documentation.

Speed up the AEM Build Time

Matija Kovaček — Wed, 30 Dec 2020 12:13:00 +0000

How to Speed up the AEM Build Time

Recently I found out tool to speed up build time of your Maven based projects.

Maven Daemon is let's say maven wrapper tool which provides faster build time.

Some facts about Maven Daemon:

build executes in long living background process
native executable
use multiple CPU cores to build modules in parallel

More information about Maven Daemon you can find in github repository.

Maven Daemon Installation

Use sdk or brew to install Maven Daemon

$ sdk install mvnd

$ brew install mvndaemon/homebrew-mvnd/mvnd

You can verify installation with

➜ ~ mvnd --version
mvnd native client 0.2.0-darwin-amd64 (0cd0b3f04692b7970fda06c206c1fbaac68fe9ce)
Terminal: org.jline.terminal.impl.PosixSysTerminal with pty org.jline.terminal.impl.jansi.osx.OsXNativePty
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /usr/local/Cellar/mvnd/HEAD-1c610c3/libexec/mvn
Java version: 15.0.1, vendor: N/A, runtime: /usr/local/Cellar/openjdk/15.0.1/libexec/openjdk.jdk/Contents/Home
Default locale: en_GB, platform encoding: UTF-8
OS name: "mac os x", version: "10.14.6", arch: "x86_64", family: "mac"

Maven vs Maven Daemon build time comparison

For testing purposes I will compare build times for AEM project. Project was build 2x times before measurement.

This measurement is not perfect, since a lot of factors can affect final build time (+- few seconds), but is fair enough to see a difference.

For testing purposes I have used AEM Project Archtype version 24.

mvn clean install vs mvnd clean install
- mvn: 26.586 s
- mvnd: 12.654 s

mvn clean install

[INFO] Reactor Summary for mysite 1.0.0-SNAPSHOT:
[INFO]
[INFO] mysite ............................................. SUCCESS [0.242 s]
[INFO] My Site - Core ..................................... SUCCESS [6.994 s]
[INFO] My Site - UI Frontend .............................. SUCCESS [11.099 s]
[INFO] My Site - Repository Structure Package ............. SUCCESS [0.795 s]
[INFO] My Site - UI apps .................................. SUCCESS [3.890 s]
[INFO] My Site - UI content ............................... SUCCESS [1.672 s]
[INFO] My Site - UI config ................................ SUCCESS [0.105 s]
[INFO] My Site - All ...................................... SUCCESS [0.138 s]
[INFO] My Site - Dispatcher ............................... SUCCESS [0.080 s]
[INFO] My Site - UI Tests ................................. SUCCESS [0.274 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 26.586 s
[INFO] Finished at: 2020-12-30T11:59:11+01:00
[INFO] ------------------------------------------------------------------------

mvnd clean install

[INFO] Reactor Summary for mysite 1.0.0-SNAPSHOT:
[INFO]
[INFO] mysite ............................................. SUCCESS [0.016 s]
[INFO] My Site - Core ..................................... SUCCESS [4.300 s]
[INFO] My Site - UI Frontend .............................. SUCCESS [10.442 s]
[INFO] My Site - Repository Structure Package ............. SUCCESS [0.178 s]
[INFO] My Site - UI apps .................................. SUCCESS [1.126 s]
[INFO] My Site - UI content ............................... SUCCESS [0.911 s]
[INFO] My Site - UI config ................................ SUCCESS [0.156 s]
[INFO] My Site - All ...................................... SUCCESS [0.107 s]
[INFO] My Site - Dispatcher ............................... SUCCESS [0.078 s]
[INFO] My Site - UI Tests ................................. SUCCESS [0.063 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 12.654 s (Wall Clock)
[INFO] Finished at: 2020-12-30T11:59:58+01:00
[INFO] ------------------------------------------------------------------------

mvn clean install -PautoInstallPackage vs mvnd clean install -PautoInstallPackage
- mvn: 29.582 s
- mvnd: 12.995 s

mvn clean install -PautoInstallPackage

[INFO] Reactor Summary for mysite 1.0.0-SNAPSHOT:
[INFO]
[INFO] mysite ............................................. SUCCESS [0.223 s]
[INFO] My Site - Core ..................................... SUCCESS [7.354 s]
[INFO] My Site - UI Frontend .............................. SUCCESS [11.338 s]
[INFO] My Site - Repository Structure Package ............. SUCCESS [0.826 s]
[INFO] My Site - UI apps .................................. SUCCESS [4.343 s]
[INFO] My Site - UI content ............................... SUCCESS [2.885 s]
[INFO] My Site - UI config ................................ SUCCESS [0.454 s]
[INFO] My Site - All ...................................... SUCCESS [0.346 s]
[INFO] My Site - Dispatcher ............................... SUCCESS [0.061 s]
[INFO] My Site - UI Tests ................................. SUCCESS [0.374 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 29.582 s
[INFO] Finished at: 2020-12-30T12:06:29+01:00
[INFO] ------------------------------------------------------------------------

mvnd clean install -PautoInstallPackage

[INFO] Reactor Summary for mysite 1.0.0-SNAPSHOT:
[INFO]
[INFO] mysite ............................................. SUCCESS [0.005 s]
[INFO] My Site - Core ..................................... SUCCESS [4.004 s]
[INFO] My Site - UI Frontend .............................. SUCCESS [10.288 s]
[INFO] My Site - Repository Structure Package ............. SUCCESS [0.177 s]
[INFO] My Site - UI apps .................................. SUCCESS [1.433 s]
[INFO] My Site - UI content ............................... SUCCESS [1.010 s]
[INFO] My Site - UI config ................................ SUCCESS [0.292 s]
[INFO] My Site - All ...................................... SUCCESS [0.227 s]
[INFO] My Site - Dispatcher ............................... SUCCESS [0.079 s]
[INFO] My Site - UI Tests ................................. SUCCESS [0.053 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 12.995 s (Wall Clock)
[INFO] Finished at: 2020-12-30T12:08:30+01:00
[INFO] ------------------------------------------------------------------------

Note:

Your AEM project build will fail with Maven Daemon if you use bnd-maven-plugin version 5.1.0

[ERROR] Failed to execute goal biz.aQute.bnd:bnd-maven-plugin:5.0.0:bnd-process (bnd-process) on project lot-polish-airlines.core: bnd error: null: ConcurrentModificationException -> [Help 1]

As we can see by results Maven Daemon build time is approximately 2x faster then with normal maven. Installation was pretty much straightforward and so far didn't notice any downsides, so why not use it and save some time.

Speed up the Maven Build Time

Matija Kovaček — Tue, 29 Dec 2020 23:00:00 +0000

How to Speed up the Maven Build Time

Recently I found out tool to speed up build time of your Maven based projects.

Maven Daemon is let's say maven wrapper which provides faster build time.

Some facts about Maven Daemon:

build executes in long living background process
native executable
use multiple CPU cores to build modules in parallel

More information about Maven Daemon you can find in github repository.

Maven Daemon Installation

Use sdk or brew to install Maven Daemon

$ sdk install mvnd

$ brew install mvndaemon/homebrew-mvnd/mvnd

You can verify installation with

➜ ~ mvnd --version
mvnd native client 0.2.0-darwin-amd64 (0cd0b3f04692b7970fda06c206c1fbaac68fe9ce)
Terminal: org.jline.terminal.impl.PosixSysTerminal with pty org.jline.terminal.impl.jansi.osx.OsXNativePty
Apache Maven 3.6.3 (cecedd343002696d0abb50b32b541b8a6ba2883f)
Maven home: /usr/local/Cellar/mvnd/HEAD-1c610c3/libexec/mvn
Java version: 15.0.1, vendor: N/A, runtime: /usr/local/Cellar/openjdk/15.0.1/libexec/openjdk.jdk/Contents/Home
Default locale: en_GB, platform encoding: UTF-8
OS name: "mac os x", version: "10.14.6", arch: "x86_64", family: "mac"

Maven vs Maven Daemon build time comparison

For testing purposes I will compare build times for AEM project. Project was build 2x times before measurement.

This measurement is not perfect, since a lot of factors can affect final build time (+- few seconds), but it's fair enough to see a difference.

For testing purposes I have used AEM Project Archtype version 24.

mvn clean install vs mvnd clean install
- mvn: 26.586 s
- mvnd: 12.654 s

mvn clean install

[INFO] Reactor Summary for mysite 1.0.0-SNAPSHOT:
[INFO]
[INFO] mysite ............................................. SUCCESS [0.242 s]
[INFO] My Site - Core ..................................... SUCCESS [6.994 s]
[INFO] My Site - UI Frontend .............................. SUCCESS [11.099 s]
[INFO] My Site - Repository Structure Package ............. SUCCESS [0.795 s]
[INFO] My Site - UI apps .................................. SUCCESS [3.890 s]
[INFO] My Site - UI content ............................... SUCCESS [1.672 s]
[INFO] My Site - UI config ................................ SUCCESS [0.105 s]
[INFO] My Site - All ...................................... SUCCESS [0.138 s]
[INFO] My Site - Dispatcher ............................... SUCCESS [0.080 s]
[INFO] My Site - UI Tests ................................. SUCCESS [0.274 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 26.586 s
[INFO] Finished at: 2020-12-30T11:59:11+01:00
[INFO] ------------------------------------------------------------------------

mvnd clean install

[INFO] Reactor Summary for mysite 1.0.0-SNAPSHOT:
[INFO]
[INFO] mysite ............................................. SUCCESS [0.016 s]
[INFO] My Site - Core ..................................... SUCCESS [4.300 s]
[INFO] My Site - UI Frontend .............................. SUCCESS [10.442 s]
[INFO] My Site - Repository Structure Package ............. SUCCESS [0.178 s]
[INFO] My Site - UI apps .................................. SUCCESS [1.126 s]
[INFO] My Site - UI content ............................... SUCCESS [0.911 s]
[INFO] My Site - UI config ................................ SUCCESS [0.156 s]
[INFO] My Site - All ...................................... SUCCESS [0.107 s]
[INFO] My Site - Dispatcher ............................... SUCCESS [0.078 s]
[INFO] My Site - UI Tests ................................. SUCCESS [0.063 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 12.654 s (Wall Clock)
[INFO] Finished at: 2020-12-30T11:59:58+01:00
[INFO] ------------------------------------------------------------------------

mvn clean install -PautoInstallPackage vs mvnd clean install -PautoInstallPackage
- mvn: 29.582 s
- mvnd: 12.995 s

mvn clean install -PautoInstallPackage

[INFO] Reactor Summary for mysite 1.0.0-SNAPSHOT:
[INFO]
[INFO] mysite ............................................. SUCCESS [0.223 s]
[INFO] My Site - Core ..................................... SUCCESS [7.354 s]
[INFO] My Site - UI Frontend .............................. SUCCESS [11.338 s]
[INFO] My Site - Repository Structure Package ............. SUCCESS [0.826 s]
[INFO] My Site - UI apps .................................. SUCCESS [4.343 s]
[INFO] My Site - UI content ............................... SUCCESS [2.885 s]
[INFO] My Site - UI config ................................ SUCCESS [0.454 s]
[INFO] My Site - All ...................................... SUCCESS [0.346 s]
[INFO] My Site - Dispatcher ............................... SUCCESS [0.061 s]
[INFO] My Site - UI Tests ................................. SUCCESS [0.374 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 29.582 s
[INFO] Finished at: 2020-12-30T12:06:29+01:00
[INFO] ------------------------------------------------------------------------

mvnd clean install -PautoInstallPackage

[INFO] Reactor Summary for mysite 1.0.0-SNAPSHOT:
[INFO]
[INFO] mysite ............................................. SUCCESS [0.005 s]
[INFO] My Site - Core ..................................... SUCCESS [4.004 s]
[INFO] My Site - UI Frontend .............................. SUCCESS [10.288 s]
[INFO] My Site - Repository Structure Package ............. SUCCESS [0.177 s]
[INFO] My Site - UI apps .................................. SUCCESS [1.433 s]
[INFO] My Site - UI content ............................... SUCCESS [1.010 s]
[INFO] My Site - UI config ................................ SUCCESS [0.292 s]
[INFO] My Site - All ...................................... SUCCESS [0.227 s]
[INFO] My Site - Dispatcher ............................... SUCCESS [0.079 s]
[INFO] My Site - UI Tests ................................. SUCCESS [0.053 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time: 12.995 s (Wall Clock)
[INFO] Finished at: 2020-12-30T12:08:30+01:00
[INFO] ------------------------------------------------------------------------

Note:

Your AEM project build will fail with Maven Daemon if you use bnd-maven-plugin version older than 5.1.0

[ERROR] Failed to execute goal biz.aQute.bnd:bnd-maven-plugin:5.0.0:bnd-process (bnd-process) on project my-site.core: bnd error: null: ConcurrentModificationException -> [Help 1]

Sum up

As test results shows, Maven Daemon build time is approximately 2x faster then normal maven. Installation is pretty much straightforward and so far didn't notice any downsides, so why not use it and save some time.

Update :

Not 100% sure, but I think installation of Maven Daemon has override my maven configuration (settings.xml).

Test behaviour, not implementation

Matija Kovaček — Sat, 20 Jun 2020 09:30:00 +0000

Introduction

Last year I spent a lot of time with writing and also practicing how to write good unit/integration tests in AEM (Adobe Experience Manager). Now I would like to share with you what I have learned so far. What I have learned is not only AEM related, you can apply it to any programming language or framework.

Before that time I had some "experience" with unit testing. I wrote several unit tests so that I can say "I have experience with it". But to be honest I didn't like to write it.

Even though I knew what benefits tests brings to the product what we are building, what brings to the team members and to me, I didn't care. Typical excuses were:

"I don't have time or we don't have time for it"
"I can't test or mock that"
"You can't write test, pick up some new feature or fix bug"

and on the end nobody pushed me to write it. Sadly, writing tests wasn't part of development process.

Now when I'm thinking a little bit, I didn't know how to write tests. Let's face it, writing tests is not easy, like many other things when you don't have experience with it.

Luckily that kind of things has changed at some point, and I would like to try convince all of you who are still thinking like "old" me.

I would like that we all start thinking more about Quality and not Quantity.

Writing tests as part of development process

Most of us works in "Agile" way (whatever that means) and use DDD (Deadline-driven development) methodology (I could write about that in separate post). This usually means that there is no time for writing tests. This need's to be changed, developers and all other technical team members should convince all other team members that writing tests should be part of development process. Writing tests should be part of any estimation. Period. Why?

There are a lot of benefits, but I will point out most important of them:

Bugs prevention
Better code quality
Provides some kind of documentation
Time saving
Money saving
Feeling "safe"

Now let's see typical "disadvantages":

Time consuming
Money consuming
Tests are slow to write
Tests are slow to run
Changing implementation require changing tests

Probably you have notice that I have mention time and money as advantage and disadvantage. Depending if you are thinking in short term, then yes it's waist of time and money, but If you think in long term then it's not true. Actually on the end it saves your time and money.

A lot of people thinks that they are waist of time and money. I think because they don't see some visual outcome of them, like how they see it when you build some feature. Try to think like this, with tests you can prevent a lot of bugs and a lot of ping pongs between Developers and QAs. Very often we have change requests during development and it happens that we implemented something in wrong way. New request just doesn't fit anymore to existing implementation. That means we need to refactor our old code or reimplement it from scratch. Here tests provides you some safe feel because you know if you broke behaviour or not. Another example could be big, never ending projects where several different teams have worked before you. Probably that project has poorly written documentation, you need to deal with legacy code and implement new features on top of it. Having tests is gold here. Also, a lot of projects starts like MVP which turns out to some core / base project with several subprojects. Not having test coverage here is total nonsense.

Last 3 disadvantages are also not true.

Tests are slow to write
- yes, If you don't know how to write it and if you don't have experience
- practice
Tests are slow to run
- again yes, if you don't know how to write it
Changing implementation require changing tests
- yes, because you are testing wrong things
- test behaviour not implementation

You don't believe me?

Take 1h of your time and watch the talk "TDD, Where Did It All Go Wrong" from Ian Cooper. For me this was eye opener. Before this talk I read few books about testing and I was not so convinced. In my opinion this is definitely the best talk about it.

tl; dr;

Test behaviour / requirements, not implementation
- with this kind of approach you will eliminate previously mentioned disadvantages
Test the public API of a module, not classes, methods or technical details
Unit test shouldn't be focused on classes, methods it should be focused on module, user stories
Test gives you promise what should be expected result / behaviour, so when you are refactoring an implementation, use tests to make sure that the implementation still yields the expected results
Write tests to cover the use cases or stories
Use the "Given When Then" model
Avoid mocks

This testing approach helps you to build right product. But negative point could be that it doesn't help you to build product right. Other downside is that you don't see exactly what is wrong when test is failing.

So classic unit testing approach push you to write more clean and quality code than "behaviour testing". In my opinion strict code reviews and static code analysis tools are better approach to achieve the same result. Second downside for me is really minor thing, since with debugging you can quickly find out what is happening.

I hope that you are still follow me and that I'm start changing a little bit your thinking about testing.

Now let's stop with theory and let's see how it works in practice.

Testing in AEM

Because last few years I'm working with AEM, I will show you how to test behaviours in your AEM projects. The same things you can apply in any other programming languages or frameworks. Depending on testing library support, this can be easier or harder to achieve.

As an example let say we need to implement Product Details API which is consumed by client side. To build Product Details API lets say in Spring you will probably create several classes like Product Controller, Service, Repository, DTO and so on. In AEM world this means you need to create Sling Servlet, OSGi Service, Sling Model and some DTO classes.

Product Details acceptance criteria:

show product details information (id, name, description, category id, images and variants)
product variants need's to be available for specific country
product variants are available from specific date
product variants need's to be sorted by sort order
name and description of product need's to be localized (depending on market), fallback is English

Implementation what you will see here is not perfect, it's simplified and hardcoded. In real world this is more complex. But here implementation is not important, instead we should focus how to test requirements of this API.

I will add here just 3 most important classes, other implementations you can see on Github

ProductDetails Sling Servlet

for handling request
it does some request validation
it use ProductDetailsService to get all information about requested product

package com.mkovacek.aem.core.servlets.products;

import com.mkovacek.aem.core.models.products.ProductDetailsModel;
import com.mkovacek.aem.core.records.response.Response;
import com.mkovacek.aem.core.services.products.ProductDetailsService;
import com.mkovacek.aem.core.services.response.ResponseService;

import lombok.extern.slf4j.Slf4j;

import org.apache.commons.lang3.StringUtils;
import org.apache.sling.api.SlingHttpServletRequest;
import org.apache.sling.api.SlingHttpServletResponse;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.servlets.HttpConstants;
import org.apache.sling.api.servlets.SlingSafeMethodsServlet;
import org.apache.sling.servlets.annotations.SlingServletResourceTypes;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;

import javax.servlet.Servlet;
import javax.servlet.ServletException;

import java.io.IOException;

@Slf4j
@Component(service = Servlet.class)
@SlingServletResourceTypes(
    resourceTypes = ProductDetailsServlet.RESOURCE_TYPE,
    selectors = ProductDetailsServlet.ALLOWED_SELECTOR,
    extensions = ProductDetailsServlet.JSON,
    methods = HttpConstants.METHOD_GET)
public class ProductDetailsServlet extends SlingSafeMethodsServlet {

    public static final String ALLOWED_SELECTOR = "productdetails";
    static final String RESOURCE_TYPE = "demo/components/productdetails";
    static final String JSON = "json";

    @Reference
    private transient ResponseService responseService;

    @Reference
    private transient ProductDetailsService productDetailsService;

    @Override
    public void doGet(final SlingHttpServletRequest request, final SlingHttpServletResponse response) throws ServletException, IOException {
        try {
            this.responseService.setJsonContentType(response);
            final String selector = request.getRequestPathInfo().getSelectorString();
            final String productId = this.responseService.getSuffix(request);

            if (this.responseService.areSelectorsValid(selector, ALLOWED_SELECTOR) && StringUtils.isNotBlank(productId)) {
                final Resource resource = request.getResource();
                final Response<ProductDetailsModel> data = this.productDetailsService.getProductDetails(productId, resource);
                this.responseService.sendOk(response, data);
            } else {
                this.responseService.sendBadRequest(response);
            }
        } catch (final Exception e) {
            log.error("Exception during handling request", e);
            this.responseService.sendInternalServerError(response);
        }
    }

}

ProductDetails OSGi Service

it's searching for requested product in repository / database
it's doing some product validation
maps product resource to ProductDetails model
returns product details

package com.mkovacek.aem.core.services.products.impl;

import com.day.cq.wcm.api.PageManager;
import com.mkovacek.aem.core.models.products.ProductDetailsModel;
import com.mkovacek.aem.core.records.response.Response;
import com.mkovacek.aem.core.records.response.Status;
import com.mkovacek.aem.core.services.products.ProductDetailsService;
import com.mkovacek.aem.core.services.resourceresolver.ResourceResolverService;

import lombok.extern.slf4j.Slf4j;

import org.apache.commons.lang3.StringUtils;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ResourceResolver;
import org.osgi.service.component.annotations.Component;
import org.osgi.service.component.annotations.Reference;

import java.util.Locale;
import java.util.Optional;

@Slf4j
@Component(service = ProductDetailsService.class, immediate = true)
public class ProductDetailsServiceImpl implements ProductDetailsService {

    private static final String PIM_READER = "pimReader";
    private static final Response<ProductDetailsModel> notFoundResponse = new Response<>(new Status(true, "Product Details not found"), null);
    private static final Response<ProductDetailsModel> errorResponse = new Response<>(new Status(false, "Error during fetching product details"), null);

    @Reference
    private ResourceResolverService resourceResolverService;

    @Override
    public Response<ProductDetailsModel> getProductDetails(final String id, final Resource resource) {
        try (final ResourceResolver resourceResolver = this.resourceResolverService.getResourceResolver(PIM_READER)) {
            final Locale locale = resourceResolver.adaptTo(PageManager.class).getContainingPage(resource).getLanguage(false);
            //usually this would be implemented with query
            final String productPath = StringUtils.join("/var/commerce/products/demo/", id);
            return Optional.ofNullable(resourceResolver.getResource(productPath))
                       .map(productResource -> productResource.adaptTo(ProductDetailsModel.class))
                       .map(productDetailsModel -> productDetailsModel.setLocale(locale))
                       .filter(ProductDetailsModel::isValid)
                       .map(productDetailsModel -> new Response<>(new Status(true), productDetailsModel))
                       .orElse(notFoundResponse);
        } catch (final Exception e) {
            log.error("Exception during fetching product details", e);
        }
        return errorResponse;
    }

}

ProductDetails Sling Model

representation of product resource in repository / database
used as response in JSON format

package com.mkovacek.aem.core.models.products;

import com.fasterxml.jackson.annotation.JsonIgnore;
import com.fasterxml.jackson.annotation.JsonProperty;
import com.mkovacek.aem.core.services.products.ProductLocalizationService;
import com.mkovacek.aem.core.services.products.ProductValidatorService;

import lombok.Getter;
import lombok.extern.slf4j.Slf4j;

import org.apache.commons.lang3.StringUtils;
import org.apache.sling.api.resource.Resource;
import org.apache.sling.api.resource.ValueMap;
import org.apache.sling.models.annotations.Default;
import org.apache.sling.models.annotations.DefaultInjectionStrategy;
import org.apache.sling.models.annotations.Model;
import org.apache.sling.models.annotations.injectorspecific.ChildResource;
import org.apache.sling.models.annotations.injectorspecific.OSGiService;
import org.apache.sling.models.annotations.injectorspecific.Self;
import org.apache.sling.models.annotations.injectorspecific.ValueMapValue;

import java.util.ArrayList;
import java.util.Comparator;
import java.util.List;
import java.util.Locale;

@Slf4j
@Model(adaptables = {Resource.class}, defaultInjectionStrategy = DefaultInjectionStrategy.OPTIONAL)
public class ProductDetailsModel {

    @ValueMapValue
    @Default(values = StringUtils.EMPTY)
    @Getter
    private String id;

    @ValueMapValue
    @Default(values = StringUtils.EMPTY)
    @Getter
    private String categoryId;

    @ChildResource
    @Getter
    private List<ImageModel> images;

    @ChildResource
    private List<VariantsModel> variants;

    @Self
    private ValueMap valueMap;

    @OSGiService
    private ProductLocalizationService productLocalizationService;

    @OSGiService
    private ProductValidatorService productValidatorService;

    @Getter
    @JsonProperty("variants")
    private List<VariantsModel> validVariants = new ArrayList<>();

    @Getter
    private String name = StringUtils.EMPTY;

    @Getter
    private String description = StringUtils.EMPTY;

    @JsonIgnore
    public boolean isValid() {
        return !this.validVariants.isEmpty();
    }

    @JsonIgnore
    public ProductDetailsModel setLocale(final Locale locale) {
        this.setLocalizedValues(locale);
        this.validateAndSortVariants(locale);
        return this;
    }

    private void setLocalizedValues(final Locale locale) {
        this.name = this.productLocalizationService.getLocalizedProductDetail(this.valueMap, "name.", locale);
        this.description = this.productLocalizationService.getLocalizedProductDetail(this.valueMap, "description.", locale);
    }

    private void validateAndSortVariants(final Locale locale) {
        this.validVariants = this.productValidatorService.getValidVariants(this.variants, locale);
        this.validVariants.sort(Comparator.comparing(VariantsModel::getSortOrder));
    }

}

Except those 3 classes I need to create several more:

ImageModel, VariantsModel
BlobStorageService, ProductValidatorService, ProductLocalizationService, ResourceResolverService, ResponseService
Response and Status records

You saw that we have a lot of classes to build this user story. Usually what would developer test here are OSGi services. I'm not saying this is a bad approach, but for that you will need more time, and every time when you will refactor your code or add some new stuff, it's very likely that you will need to change your tests as well.

Instead of that let's test only Servlet because this is public API of this user story. So what we need to test in Servlet? First of all, we need to cover all requirments from acceptance criteria, additionaly we can cover some technical details of servlet implementation.

Test libraries in AEM

At the moment in my opinion the best library what you can use are AEM Mocks. AEM Mocks supports most common mock implementations of AEM APIs + contains Apache Sling and OSGi mock implementations. For other not implemented mocks you will need to implement it by yourself or use Mockito. Besides those two I will use Junit 5.

Some tips before we start:

Try to have Test classes clean as possible, it should contains just tests.
Move mocks in separate classes
Create some Util classes with common helper methods, if you repeat yourself in multiple places
Use @BeforeAll / AfterAll, @BeforeEach / AfterEach, Junit 5 annotations to not repeat yourself in every test method and to speed up your tests
Create common AEM context in separate class if you repeat yourself in several test classes
Don't programmatically create complex resources in AEM context, instead export it from real AEM instance as JSON resource and load it into AEM context.
Use ResourceResolverMock type whenever is possible to speed up your tests

ProductDetailsServletTest

You will see that this test class is more or less clean and it focused only on tests. There is no mocking here, separated mock example you can see here. I'm using @BeforeAll and @BeforeEach to do some common setup, like setting up market pages/resources and common request informations. Also I needed some helper class to easier register all necessary classes into AEM context. All resources are exported as JSON from real AEM instance and imported into AEM context so that we test on real data.

In this test class I'm testing technical details and requirements

technical details
- request validation
requirements
- response for non existing product id
- product details in different markets to cover localization
- product variants validation for specific markets
- product variants availability from specific date
- product varinats sorting

package com.mkovacek.aem.core.servlets.products;

import com.day.cq.wcm.api.Page;
import com.mkovacek.aem.core.context.AppAemContextBuilder;
import com.mkovacek.aem.core.context.constants.TestConstants;
import com.mkovacek.aem.core.context.utils.ResourceUtil;
import com.mkovacek.aem.core.services.blobstorage.impl.BlobStorageServiceImpl;
import com.mkovacek.aem.core.services.products.impl.ProductDetailsServiceImpl;
import com.mkovacek.aem.core.services.products.impl.ProductLocalizationServiceImpl;
import com.mkovacek.aem.core.services.products.impl.ProductValidatorServiceImpl;
import com.mkovacek.aem.core.services.resourceresolver.impl.ResourceResolverServiceImpl;
import com.mkovacek.aem.core.services.response.impl.ResponseServiceImpl;

import io.wcm.testing.mock.aem.junit5.AemContext;
import io.wcm.testing.mock.aem.junit5.AemContextExtension;

import org.apache.commons.lang3.StringUtils;
import org.apache.sling.api.resource.ResourceResolverFactory;
import org.apache.sling.testing.mock.sling.servlet.MockRequestPathInfo;
import org.apache.sling.testing.resourceresolver.MockResourceResolverFactory;
import org.junit.jupiter.api.*;
import org.junit.jupiter.api.extension.ExtendWith;

import javax.servlet.ServletException;
import javax.servlet.http.HttpServletResponse;

import java.io.IOException;
import java.util.Collections;

import static org.junit.jupiter.api.Assertions.assertAll;
import static org.junit.jupiter.api.Assertions.assertEquals;

@ExtendWith(AemContextExtension.class)
class ProductDetailsServletTest {

    private static final AemContext context = new AppAemContextBuilder()
                                                  .loadResource(TestConstants.HR_HR_LANDING_PAGE_JSON, TestConstants.HR_HR_LANDING_PAGE_PATH)
                                                  .loadResource(TestConstants.DE_AT_LANDING_PAGE_JSON, TestConstants.DE_AT_LANDING_PAGE_PATH)
                                                  .loadResource(TestConstants.FR_FR_LANDING_PAGE_JSON, TestConstants.FR_FR_LANDING_PAGE_PATH)
                                                  .loadResource(TestConstants.PRODUCTS_JSON, TestConstants.PRODUCTS_PATH)
                                                  .registerService(ResourceResolverFactory.class, new MockResourceResolverFactory())
                                                  .registerInjectActivateService(new ResourceResolverServiceImpl())
                                                  .registerInjectActivateService(new ResponseServiceImpl())
                                                  .registerInjectActivateService(new BlobStorageServiceImpl(), Collections.singletonMap("productImagesFolderPath", "https://dummyurl.com/images/products/"))
                                                  .registerInjectActivateService(new ProductValidatorServiceImpl())
                                                  .registerInjectActivateService(new ProductLocalizationServiceImpl())
                                                  .registerInjectActivateService(new ResponseServiceImpl())
                                                  .registerInjectActivateService(new ProductDetailsServiceImpl())
                                                  .build();

    private static final MockRequestPathInfo requestPathInfo = context.requestPathInfo();
    private final ProductDetailsServlet servlet = context.registerInjectActivateService(new ProductDetailsServlet());
    private static final String CONTENT_RESOURCE_PATH = "root/productdetails";
    private static String NOT_FOUND_RESPONSE;
    private static String BAD_REQUEST_RESPONSE;

    @BeforeAll
    static void setUpBeforeAllTests() throws IOException {
        context.addModelsForPackage(TestConstants.SLING_MODELS_PACKAGES);
        requestPathInfo.setExtension("json");
        NOT_FOUND_RESPONSE = ResourceUtil.getExpectedResult(ProductDetailsServlet.class, "responses/not-found-response.json");
        BAD_REQUEST_RESPONSE = ResourceUtil.getExpectedResult(ProductDetailsServlet.class, "responses/bad-request-response.json");
    }

    @BeforeEach
    void setupBeforeEachTest() {
        context.response().resetBuffer();
        requestPathInfo.setSelectorString(ProductDetailsServlet.ALLOWED_SELECTOR);
        requestPathInfo.setSuffix("123456789");
        final Page page = context.pageManager().getPage(TestConstants.HR_HR_LANDING_PAGE_PATH);
        context.request().setResource(page.getContentResource(CONTENT_RESOURCE_PATH));
    }

    @Test
    @DisplayName("GIVEN landing page (en-HR) WHEN servlet is called with not valid selector THEN it returns bad request response in JSON format")
    void testNotValidSelector() throws ServletException, IOException {
        requestPathInfo.setSelectorString(ProductDetailsServlet.ALLOWED_SELECTOR + ".test");
        this.servlet.doGet(context.request(), context.response());

        assertAll(
            () -> assertEquals(HttpServletResponse.SC_BAD_REQUEST, context.response().getStatus()),
            () -> assertEquals(BAD_REQUEST_RESPONSE, context.response().getOutputAsString())
        );
    }

    @Test
    @DisplayName("GIVEN landing page (en-HR) WHEN servlet is called without productId suffix THEN it returns bad request response in JSON format")
    void testNoProductId() throws ServletException, IOException {
        requestPathInfo.setSuffix(StringUtils.EMPTY);
        this.servlet.doGet(context.request(), context.response());

        assertAll(
            () -> assertEquals(HttpServletResponse.SC_BAD_REQUEST, context.response().getStatus()),
            () -> assertEquals(BAD_REQUEST_RESPONSE, context.response().getOutputAsString())
        );
    }

    @Test
    @DisplayName("GIVEN landing page (en-HR) WHEN servlet is called with not existing productId THEN it returns not found response in JSON format")
    void testNotExistingProductId() throws ServletException, IOException {
        requestPathInfo.setSuffix("123abc");
        this.servlet.doGet(context.request(), context.response());

        assertAll(
            () -> assertEquals(HttpServletResponse.SC_OK, context.response().getStatus()),
            () -> assertEquals(NOT_FOUND_RESPONSE, context.response().getOutputAsString())
        );
    }

    @Test
    @DisplayName("GIVEN landing page (en-HR) WHEN servlet is called with existing productId THEN it returns an expected localized (fallback) product details response in JSON format")
    void testProductDetailsInCroatianMarket() throws ServletException, IOException {
        this.servlet.doGet(context.request(), context.response());
        final String expectedProductDetails = ResourceUtil.getExpectedResult(this.getClass(), "responses/product-123456789-hr-HR.json");

        assertAll(
            () -> assertEquals(HttpServletResponse.SC_OK, context.response().getStatus()),
            () -> assertEquals(expectedProductDetails, context.response().getOutputAsString())
        );
    }

    @Test
    @DisplayName("GIVEN landing page (de-AT) WHEN servlet is called with existing productId THEN it returns an expected localized product details response in JSON format")
    void testProductDetailsInAustrianMarket() throws ServletException, IOException {
        this.setPageResource(TestConstants.DE_AT_LANDING_PAGE_PATH);
        this.servlet.doGet(context.request(), context.response());
        final String expectedProductDetails = ResourceUtil.getExpectedResult(this.getClass(), "responses/product-123456789-at-DE.json");

        assertAll(
            () -> assertEquals(HttpServletResponse.SC_OK, context.response().getStatus()),
            () -> assertEquals(expectedProductDetails, context.response().getOutputAsString())
        );
    }

    @Test
    @DisplayName("GIVEN landing page (fr-FR) WHEN servlet is called with existing productId which is not valid for French market THEN it returns not found response in JSON format")
    void testProductDetailsInFrenchMarket() throws ServletException, IOException {
        this.setPageResource(TestConstants.FR_FR_LANDING_PAGE_PATH);
        this.servlet.doGet(context.request(), context.response());

        assertAll(
            () -> assertEquals(HttpServletResponse.SC_OK, context.response().getStatus()),
            () -> assertEquals(NOT_FOUND_RESPONSE, context.response().getOutputAsString())
        );
    }

    private void setPageResource(final String path) {
        final Page page = context.pageManager().getPage(path);
        context.request().setResource(page.getContentResource(CONTENT_RESOURCE_PATH));
    }
}

With this testing approach I have covered 87% of lines of code. Other 13% what is not covered is catching exceptions.

Other good examples for testing in AEM would be components. For every component you have requirements. To achieve those requirements you will probaly create several classes like OSGi service, some Utils, Records and those requirements you will publicly exposed through Sling model to view layer. Ideal candidats for testing.

Sum up

If you don't write tests, start writing it
Test requirements not implementation
Developers should have time for writing tests

Really? Preselected checkbox not working, common AEM?

Matija Kovaček — Mon, 01 Jun 2020 07:00:00 +0000

Why one simple preselected checkbox doesn't work in page properties?

Today I needed to add one, simple checbox in page properties and it should be preselected by default. Sounds simple, 5 min task, yea sure....

Since I'm not doing this stuff so often I needed to check granite documentation to see how to achive it. Looks simple, I found properties what I need ( value , uncheckedValue , checked ).

<showSomething
    jcr:primaryType="nt:unstructured"
    sling:resourceType="granite/ui/components/coral/foundation/form/checkbox"
    cq:showOnCreate="{Boolean}true"
    text="Please be preselected by default"
    name="./showSomething"
    checked="{Boolean}true" // tried also with string "true"
    value="{Boolean}true" // "true"
    uncheckedValue="{Boolean}false" /> //"false"

And of course it doesn't work in page properties just in component dialog...

Since I'm tired of doing some hacks or writing Javascript to get this stuff working, I just switched dialog to hideSomething so that not preselected checkbox have sense.

I don't understand how that kind of simple things doesn't work as expected OOTB.

Share your ridiculous cases which doesn't work? :D

Updating AEM content with Sling Pipes

Matija Kovaček — Fri, 22 May 2020 13:00:00 +0000

You are still updating content manually? Try out Sling pipes.

Several weeks ago I saw some tweet how Adobe guys are using Sling pipes for updating their AEM content, so I was curious and I try it out.

Let's see what I have find out...

Sling pipes is simple tool for executing CRUD operations over resources in AEM repository. But is it powerful enough to replace your groovy scripts? Let's discover it in following examples.

Introduction

Just to get your attention, let's say in past you have made some bad decision during development of some of your component and now you need to update all content with that component. With Sling Pipes that is very easy.

Example

plumber.newPipe(resourceResolver)
  .echo("/content/yoursite")
  .$("nt:unstructured[sling:resourceType=projectName/components/content/imageSlider]")
  .write("autoplay", true)
  .run()

5 lines? Yes, you can set autoplay property to true in all your content with imageSlider component in 5 lines, and I'm pretty much sure you can't do that so simply with your groovy code, or you can do it manually in crx/de for whole day...

So what the hell is happening in this code, what are this methods?!

I will explain it later in more details, but in tl;dr;

plumber is OSGi service and to start of all this magic you need to pass resourceResolver object to newPipe method. After that you need to call pipes (methods) to achieve what you want.
echo("path") is used to receive root resource on specific path.
$(expression) is wrapper of Sling query expressions and I'm using it here to filter out specific resources with imageSlider component
write(conf) is pipe to write/update resources/properties, in this case I'm creating or updating autoplay property with value true
run() is used to execute code

Still interested? Let me explain you Sling pipes in more details.

About Sling Pipes

Sling pipes is a tool set for doing extract - transform - load operations by chaining proven code blocks. A sling pipe is essentially a sling resource stream, encapsulating a well-known sling operations.

There are 3 types of pipes:

reader pipes - used to get resources
writer pipes - for writing/updating resources in repository, depending on configuration and input
logical pipes - they refer to other pipes, chaining them or using their results in a general way

Reader pipes

Used to get resources.

Base pipe: echo(path)

Used to receive resource on given path

Let's say you want to get "/content/we-retail" resource.

plumber.newPipe(resourceResolver)
  .echo("/content/we-retail")
  .run()

XPathPipe: xpath(expr)

Used to receive resource with given xpath query

Let's say you want to get all nt:unstructured resources with heroimage component under "/content/we-retail".

plumber.newPipe(resourceResolver)
  .xpath("/jcr:root/content/we-retail//element(*, nt:unstructured)[(@sling:resourceType = 'weretail/components/content/heroimage')]")
  .run()

Sling Query Pipes

Wrapper for executing Sling Query methods in sling pipe way.

Find Pipe: $(expr)
Children Pipe: children(expr)
Siblings Pipe: siblings(expr)
Parent Pipe: parent()
Closest Pipe: closest(expr)

Here is more details about Sling Query

Same example as previous one, let's say you want to get all resources with heroimage component under "/content/we-retail".

plumber.newPipe(resourceResolver)
  .echo("/content/we-retail")
  .$("nt:unstructured[sling:resourceType=weretail/components/content/heroimage]")
  .run()

Writer pipes

Used to modify resources.

Write Pipe: write(conf)

Used to write given nodes and properties to current input resource

Let's say you want to create/update property "foo" with value "bar" and property "foo2" with value "bar2".

Please let me know is it possible to write different values except strings, like long, date, boolean?

plumber.newPipe(resourceResolver)
  .echo("/content/we-retail")
  .write('foo','bar','foo2','bar2')
  .run()

PathPipe: mkdir(expr)

Used to create resource on given location.

Let's you want to create resource "master" on "/content/we-retail/master" location. This will create "sling:Folder" node.

Please let me know if is it possible to specify different jcr:primaryType?

plumber.newPipe(resourceResolver)
  .mkdir("/content/we-retail/master")
  .run()

MovePipe: mv(expr)

Moves resource or property to target path

Following example will move master resource from "/content/we-retail/master" to "/content/we-retail/language-master".

plumber.newPipe(resourceResolver)
  .echo("/content/we-retail/master")
  .mv("/content/we-retail/language-master")
  .run()

RemovePipe: rm()

Removes resource or property on specific location.

Following example will remove master resource on given path "/content/we-retail/master".

plumber.newPipe(resourceResolver)
  .echo("/content/we-retail/master")
  .rm()
  .run()

PackagePipe: pkg(expr)

Creates a package and add current resource as a filter.

Following example will package all we retail content

plumber.newPipe(resourceResolver)
  .echo("/content/we-retail")
  .pkg("/etc/packages/we-retail-content.zip")
  .run()

How to use it in your AEM project

Okey, how to run all this in your AEM project? You can configure and execute a pipes with java, groovy console, http, or jmx

Dependencies

You need to add "org.apache.sling.query" & "org.apache.sling.pipes" as dependencies and you need to embedded them as part of your bundle or install it as separate bundles (my recommended way).

<dependency>
    <groupId>org.apache.sling</groupId>
    <artifactId>org.apache.sling.query</artifactId>
    <version>4.0.2</version>
    <scope>provided</scope>
</dependency>
<dependency>
    <groupId>org.apache.sling</groupId>
    <artifactId>org.apache.sling.pipes</artifactId>
    <version>3.1.0</version>
    <scope>provided</scope>
</dependency>

<plugin>
  <groupId>org.apache.jackrabbit</groupId>
  <artifactId>filevault-package-maven-plugin</artifactId>
  <extensions>true</extensions>
  <configuration>
      <allowIndexDefinitions>true</allowIndexDefinitions>
      <packageType>mixed</packageType>
      <group>pipes</group>
      <embeddeds>
          <!--sling pipes related-->
          <embedded>
              <groupId>org.apache.sling</groupId>
              <artifactId>org.apache.sling.query</artifactId>
              <target>/apps/pipes-vendor-packages/application/install</target>
          </embedded>
          <embedded>
              <groupId>org.apache.sling</groupId>
              <artifactId>org.apache.sling.pipes</artifactId>
              <target>/apps/pipes-vendor-packages/application/install</target>
          </embedded>
      </embeddeds>
  </configuration>
</plugin>

Code execution

You can write in groovy console.

def plumber = getService("org.apache.sling.pipes.Plumber")

plumber.newPipe(resourceResolver)
  .echo("/content/yoursite")
  .$("nt:unstructured[sling:resourceType=projectName/components/content/imageSlider]")
  .write("autoplay", true)
  .run()

You can write in you Java class.

@Component(service = SlingPipesExamples.class, immediate = true)
public class SlingPipesExamples {

    private static final Logger logger = LoggerFactory.getLogger(SlingPipesExamples.class);

    private static final String SERVICE_USER = "sling-pipes-service-user";

    @Reference
    private ResourceResolverFactory resourceResolverFactory;

    @Reference
    private Plumber plumber;

    public void example() {
        try (final ResourceResolver resourceResolver = this.resourceResolverFactory.getServiceResourceResolver(this.getAuthenticationInfo())) {
            PipeBuilder pipeBuilder = this.plumber.newPipe(resourceResolver);
            // your logic with pipes
        } catch (final LoginException e) {
            logger.error("Exception during creating service resource resolver ", e);
        }
    }

    private Map<String, Object> getAuthenticationInfo() {
        return Collections.singletonMap(ResourceResolverFactory.SUBSERVICE, SERVICE_USER);
    }

}

You can create resource on specific location e.g "/etc/sling-pipes/example" and execute it with HTTP request

<?xml version="1.0" encoding="UTF-8"?>
<jcr:root xmlns:sling="http://sling.apache.org/jcr/sling/1.0" xmlns:jcr="http://www.jcp.org/jcr/1.0"
  jcr:description="Create directory pipe"
  jcr:primaryType="sling:OrderedFolder"
  sling:resourceType="slingPipes/mkdir"
  expr="/content/sling-pipes/foo"/>

Sum up

Basicly that's it what I discover so far. In general I like Sling Pipes, specialy how with few lines of code you can do a lot of stuff. Didn't try with some more complex examples but I guess it would be almost the same.

For more details take a look into offical documentation.

Let me know if for mkdir pipe is possible to specify different jcr:primaryType? If it's only "sling:Folder" then 👎

Also for write pipe, is it possible to write in property some value different then String?

I hope that you learn something and that you will give a try Sling Pipes. Let me know what you think and what are your experiences with Sling Pipes :)

Getting started with Scala

Matija Kovaček — Mon, 17 Oct 2016 22:00:00 +0000

So you want to write your first Scala application? Before that, you need to install Scala on your PC and know some basic syntax.

First, you need to install Java JDK and Scala binaries. To check if you have successfully installed Scala just write command “scala” in command prompt or terminal and if everything is OK you will see a welcome message.

Scala has REPL (Read-Eval-Print-Loop) which allows you to write Scala code and immediately see results of your code in command prompt/terminal. REPL allows you to experiment with all kind of Scala expressions and immediately see results.

If you don’t like cmd/terminal you can use your favorite IDE or text editor. IntelliJ, Eclipse and NetBeans have good support for writing Scala code. Also if you want REPL in your IDE you have to create Scala worksheet and in them you can experiment with your code.

Scala syntax

Variable declarations

In Scala, you have two types of variables, immutable and mutable. Immutable variables are read-only and that means when you declare a variable, later you can’t assign new values/objects to that variable. An immutable variable is declared with keyword val.

val name: String = "Scala"

//shorter
val newName = "Scala"

// you can't assign new value
newName = "Java" // Error: reassignment to val name = "Java"

Mutable variables are declared with keyword var and later you can assign new values/objects to that variable.

var age = 23

//you can assign new values
age = 24
age = 25

So keywords val (immutable) and var (mutable) specify whether the reference can be changed to a different object (var) or not (val).

To declare variable you need to choose if you want immutable (val) or mutable (var) variable. Than you write name, after name you optionally write type of variable and on right side of equality you assign some value, object or expression. Defining type is optional because compiler can figure out it from right side of equality.

val nameOfImmutableVariable: type = value
var nameOfMutableVariable: type = value

Method declarations

Method declarations start with keyword def , followed by optional argument list, optional return type and after equals sign body of the method.

def methodName (argument: argumentType): returnType = {
  //body of the method
}

Methods that don’t return some value for return type have a Unit type.

def printHelloWorld: Unit = {
  println("Hello world")
}

Also, you don’t have to write return keyword to return some value from the method, it’s enough to write value or some object that you want to return.

def sum(a: Int, b: Int): Int = {
  a + b
}

Scala methods can be written even more a concisely. Return type doesn’t have to be specified and parentheses around the body of the short method aren’t required.

def sum(a: Int, b: Int) = a + b

Scala is flexible when you working with methods. You can call object methods in two ways, the first way is to call a method on the object with dot character and the second way is like you write a sentence.

someobject.methodName()
//or
someobject methodName()

Also it’s important to mention when you call method which don’t take argument and has no side effects (pure functions) like mutating the state of objects or I/O operations than when you call method you don’t have to put rounded brackets after the method name.

List(1,2,3,4,5).size // 5
List(1,2,3,4,5).size() // error

Classes

To create Scala class you need to write keyword “class” and name of the class. A primary constructor of Scala class makes a combination of constructor parameters which are defined after class name and expressions and methods that are called in the body of the class. Depends on constructor parameters and fields types Scala automatically generates getter and setter methods. So if a field is mutable than are generated both getter and setter methods and if a field is an immutable than is generated the only getter method.

class Person(val firstName: String, val lastName: String){
  println("beginning of the constructor")
  var age = 25
  def printInfo = println("Firstname: " + firstName + ", lastname: " + lastName + ", age: " + age)
  printInfo
  println("end of the constructor")
}

val person = new Person("Matija","Kovacek")
/*
beginning of the constructor
Firstname: Matija, lastname: Kovacek, age: 25
end of the constructor
*/
// getter 
person.firstName
//setter
person.age = 30

No semicolons

You may have noticed that in previous examples I didn’t use semicolons. Scala is treating end of the line as the end of the statement or an expression so you don’t have to put semicolons. You only have to use semicolons when you want to put multiple statements or expressions on the same line.

I started learning Scala programming language and I don’t regret

Matija Kovaček — Sun, 18 Sep 2016 08:30:00 +0000

Couple months ago I start learning Scala programming language and I want to share what I have learned so far.

I will try in several weeks maybe months share everything I know about Scala.

So let (scala) games begin.

Scala is a programming language created by professor Martin Odersky. It’s object-oriented and also a functional programming language. So you can write your code using OOP, FP or both. In Scala everything is an object, primitive types don’t exist and operators are methods. Also, interesting OOP stuff are trait interfaces, case and object classes. Functions are first-class citizens in FP, so you can assign them to variables and pass to other functions like other objects. Other interesting FP concepts in Scala are immutable variables, immutable data structures, and pattern matching.

Scala is statically typed language but it feels like it’s dynamically typed. I call Scala like a modern version of Java with some mix of Ruby and Python. Scala minimizes unnecessary syntax (such as declarations of variable type) and with the flexible syntax, you have feeling that you write code in the dynamically typed language.

It’s a JVM language, but there is a .NET version of Scala, also exists Scala.js and Scala-native is in a developing mode.

Who uses Scala? Twitter, LinkedIn, Netflix, Tumbler, Foursquare …

And for the end, I want to share few interesting quotes.

Why Twitter choose Scala:

„ … The other big reason we looked at Scala was that, although we’ve run into problems with Ruby, we like the flexibility of the language. We like that it’s such a full featured language , that it’s fun to code in. It’s the same reason so many Java people end up writing Ruby after they leave some big enterprise company. They want to have fun day to day. We didn’t want to leave that behind and go to a language with a very dry, businesslike community, like C++, for example. We know that people write super high-performance code in C++, and engineers like Steve and Robey have had experience with that. But we wanted to be using a language that we’re really passionate about, and it seemed worth taking a gamble on Scala.“ — Alex Payne

What creators of Java, Groovy, and jRuby thinks about Scala:

“If I were to pick a language to use today other than Java, it would be Scala.” — James Gosling, creator of Java

“Scala, it must be stated, is the current heir apparent to the Java throne. No other language on the JVM seems as capable of being a “replacement for Java” as Scala, and the momentum behind Scala is now unquestionable. While Scala is not a dynamic language, it has many of the characteristics of popular dynamic languages, through it’s rich and flexible type system, it’s sparse and clean syntax, and its marriage of functional and object paradigms.” — Charles Nutter, creator of JRuby

“I can honestly say if someone had shown me the Programming in Scala book by Martin Odersky, Lex Spoon & Bill Venners back in 2003 I’d probably have never created Groovy.” — James Strachan, creator of Groovy.

Check how to get started with Scala.