DEV Community: pyltsin

How to write your own language plugin for IDEA (part 2)

pyltsin — Tue, 26 Apr 2022 18:22:23 +0000

Disclaimer: I don't work for JetBrains, so there may be and most likely there will be inaccuracies and errors in the article and code.

In the previous article, I showed the process of creating a framework for a language plugin. Well-known plugins for Java, go, Frege were used as examples. There are also examples from the plugin for the Monkey language, which I developed while I was figuring out how everything works. Since my goal was not to cover everything, the plugin covers a limited subset of the language. The interpreter for Monkey can be found here.

Formatting

Documentation, examples from go-plugin, monkey.
As always, everything starts from an extension point.

<lang.formatter language="Monkey"
implementationClass="com.github.pyltsin.monkeyplugin.formatter.MonkeyFormattingModelBuilder"/>

A class must implement the next interface

public interface FormattingModelBuilder {
@NotNull 
FormattingModel createModel(@NotNull FormattingContext formattingContext);

@Nullable 
TextRange getRangeAffectingIndent(PsiFile file, int offset,
                                  ASTNode elementAtOffset) ;
}

The most important method is the first one. It returns a formatting model, which is built on formatting blocks. To simplify it, you can use FormattingModelProvider.createFormattingModelForPsiFile method

Let's take a look at what a formatting block is. In IDEA, a formatting block is represented as an interface com.intellij.formatting.Block. It is some range of text (often corresponding to some PSI element) which is formatted according to some rules. The formatting blocks are nested into each other and create a tree.

public interface Block {

  @NotNull
  TextRange getTextRange();

  @NotNull
  List<Block> getSubBlocks();

  @Nullable
  Wrap getWrap();

  @Nullable
  Indent getIndent();

  @Nullable
  Alignment getAlignment();

  @Nullable
  Spacing getSpacing(@Nullable Block child1, @NotNull Block child2);

  @NotNull
  ChildAttributes getChildAttributes(final int newChildIndex);

/// some methods are skipped
}

If you want to visualize this tree you can use PSI-Viewer
(Tools->View PSI Structure, tab "Block Structure")

To simplify the implementation, you can use the AbstractBlock class.

getSpacing - determines the number of spaces or line breaks between the specified child elements. To simplify the implementation of this logic, you can use the com.intellij.formatting.SpacingBuilder class, which provides a convenient API to describe rules.

//an extraction from plugin for Golang
public Spacing getSpacing(@Nullable Block child1, @NotNull Block child2) {
return new SpacingBuilder(settings, GoLanguage.INSTANCE)
      .before(COMMA).spaceIf(false)
      .after(COMMA).spaceIf(true)
      .betweenInside(SEMICOLON, SEMICOLON, FOR_CLAUSE).spaces(1)
      .before(SEMICOLON).spaceIf(false)
      .after(SEMICOLON).spaceIf(true)
      .beforeInside(DOT, IMPORT_SPEC).none()
      .afterInside(DOT, IMPORT_SPEC).spaces(1)
      //and more rules
      .getSpacing(this, child1, child2);
}

Structure view

Documentation. Examples of implementation in plugin-go, Monkey.

In this section we will talk about filling this panel:

The next extension point is responsible for that

<lang.psiStructureViewFactory language="Monkey"
implementationClass="com.github.pyltsin.monkeyplugin.editor.MonkeyStructureViewFactory"/>

Your class must implement the next interface:

@FunctionalInterface
public interface PsiStructureViewFactory {
  @Nullable
  StructureViewBuilder getStructureViewBuilder(@NotNull PsiFile psiFile);
}

For implementation StructureViewBuilder you can also use prepared classes: com.intellij.ide.structureView.TreeBasedStructureViewBuilder, com.intellij.ide.structureView.StructureViewModelBase and com.intellij.ide.structureView.impl.common.PsiTreeElementBase.

Working with PsiTreeElementBase is similar to working with formatting blocks.

Caches, indexes, stub and goto

Caches

Let's start with the caches. IDEA asks extensions several times. If the extension is doing some time-consuming work, then the best solution is to cache the result of this work. IDEA provides many wrappers for this. For example, this is how the type of Go expression is calculated in the plugin-go:

  @Nullable
  public static GoType getGoType(@NotNull GoExpression o, @Nullable ResolveState context) {
    return RecursionManager.doPreventingRecursion(o, true, () -> {
      if (context != null) return unwrapParType(o, context);
      return CachedValuesManager.getCachedValue(o, () -> CachedValueProvider.Result
        .create(unwrapParType(o, createContextOnElement(o)), PsiModificationTracker.MODIFICATION_COUNT));
    });
  }

2 different managers are used here. The first is the CachedValuesManager, which caches the result for the psi element, and the second is the Recursion Manager, which helps to prevent infinite recursion and StackOverflowError. There is also a specialized cache com.intellij.psi.impl.source.resolve.ResolveCache, which is used for resolving elements (this will be specified below).

Indexes

Documentation.

We all know how IDEA likes to index everything. Let's see what it is and how it can be used.

Indexes in IDEA provide the opportunity to quickly find a necessary file or a psi element (for example, by the name of the annotation, you can find all the places where it is used).

You can view existing indexes using the Index viewer plugin.

IDEA supports two types of indexes - File-based and Stub. File-based indexes work with the contents of a file, Stub indexes work with Stub-tree, which is built on the basis of PSI-tree.

File-based indexes

An example of use can be found in the hackforce plugin

We, as usual, start from the extension point

<fileBasedIndex implementation="com.haskforce.index.HaskellModuleIndex"/>

Class must extend FileBasedIndexExtension. To get result of indexing you can use FileBasedIndex.getInstance()

The hack force plugin, for example, uses this type of index to get all the files within the module:

@NotNull
public static Collection<VirtualFile> getVirtualFilesByModuleName(@NotNull String moduleName, @NotNull GlobalSearchScope searchScope) {
  return FileBasedIndex.getInstance()
    .getContainingFiles(HASKELL_MODULE_INDEX, moduleName, searchScope);
}

Stub indexes

It seems to me that this type of indexes is used more often because it allows you to search through PSI elements (or rather through stub, which represents the required part of the psi tree). Stubs are used only for named psi elements (which implement the PsiNamedElement interface). They will be described in more detail in the Reference section

To declare a new index, the following extension point is used:

<stubIndex implementation=
"com.github.pyltsin.monkeyplugin.stubs.MonkeyVarNameIndex"/>

Your class must implement StubIndexExtension.
Example from the plugin for Monkey:

class MonkeyVarNameIndex : StringStubIndexExtension<MonkeyNamedElement>() {
    override fun getVersion(): Int {
        return super.getVersion() + VERSION
    }

    override fun getKey(): StubIndexKey<String, MonkeyNamedElement> {
        return KEY
    }

    companion object {
        val KEY: StubIndexKey<String, MonkeyNamedElement> =
            StubIndexKey.createIndexKey("monkey.var.name")
        const val VERSION = 0
    }
}

Examples from go-plugin, frege.

Now we need to teach IDEA how to create a tree of Stubs and save the necessary elements under the desired index.

For each element type that we want to save as a Stub, we create a Stub definition. In this case, the root of all Stubs should be FileStub.

Example from the plugin for Monkey:

class MonkeyFileStub(file: MonkeyFile?) : PsiFileStubImpl<MonkeyFile>(file)

class MonkeyVarDefinitionStub : NamedStubBase<MonkeyVarDefinition> {
    constructor(parent: StubElement<*>?, elementType: IStubElementType<*, *>, name: StringRef?) : super(
        parent,
        elementType,
        name
    )

    constructor(parent: StubElement<*>?, elementType: IStubElementType<*, *>, name: String?) : super(
        parent,
        elementType,
        name
    )
}

Examples from go-plugin (file, element), Frege (file, element)

The next step is to create a description of the element type for each Stub. (For automatically generated PSI elements with Grammar-Kit plugin, descriptions of each type of element are created automatically following elementTypeHolderClass and elementTypeClass parameters). The ElementType for the file must extend IStubFileElementType, for the element - IStubElementType.

IStubElementType requires the implementation of the following methods:

@NotNull
String getExternalId();

void serialize(@NotNull T stub, @NotNull StubOutputStream dataStream) throws IOException;

@NotNull
T deserialize(@NotNull StubInputStream dataStream, P parentStub) throws IOException;

void indexStub(@NotNull T stub, @NotNull IndexSink sink);

PsiT createPsi(@NotNull StubT stub);

@NotNull StubT createStub(@NotNull PsiT psi, StubElement<?> parentStub);

shouldCreateStub(ASTNode node)

How to index Stubs is specified in the index Sub method. For example, in Monkey I used this implementation:

    override fun indexStub(stub: S, sink: IndexSink) {
        val name = stub.name
        if (name != null) {
            sink.occurrence(MonkeyVarNameIndex.KEY, name)
        }
    }

The implementation of other methods can be found in the examples - plugin for Monkey, go-plugin, Frege

Now our stubs need to be connected to the parser. This can be done in 2 steps.

Step 1: define your element type factory for which we have made IStubElementType, for example as

object MonkeyElementTypeFactory {
    @JvmStatic
    fun factory(name: String): IElementType {
        if (name == "VAR_DEFINITION") return MonkeyVarDefinitionStubElementType(name)
        throw RuntimeException("Unknown element type: $name")
    }
}

and specify in bnf file that it should be used for some PSI elements:

elementTypeFactory("var_definition")=
"com.github.pyltsin.monkeyplugin.psi.impl.MonkeyElementTypeFactory.factory"

Step 2. Specify that these PSI elements should extend StubBasedPsiElementBase

Let's consider everything together. During indexing, StubBasedPsiElementBase is created, then a Stub is created using IStubElementType.createStub. This stub can be serialized and a reference to it is saved in the index(indexStub).

Client code which works with Stubs should call only those methods that have enough stored information to execute. Therefore, it is necessary to include in a stub all the information that may be needed later in the analysis. To get the PSI element, you can call the getNode() method, but it is expensive because it requires file parsing.

An example of saving information can be found in com.intellij.psi.impl.java.stubbs.impl.PsiAnnotationStubImpl#getPsiElement, which uses text from ASTNode.

Using stub indexes

Indexes are widely used for go-to functions. For example, they are used for this panel:

Simplified go-to implementation for symbols based on stub indexes:

plugin.xml

<gotoSymbolContributor implementation=
"com.github.pyltsin.monkeyplugin.usages.MonkeySymbolContributor"/>

MonkeySymbolContributor:

class MonkeySymbolContributor : ChooseByNameContributorEx {
    private val myIndexKeys = listOf(MonkeyVarNameIndex.KEY)
    override fun processNames(
        processor: Processor<in String>,
        scope: GlobalSearchScope,
        filter: IdFilter?
    ) {
        for (key in myIndexKeys) {
            ProgressManager.checkCanceled()
            StubIndex.getInstance().processAllKeys(
                key,
                processor,
                scope,
                filter
            )
        }
    }

    override fun processElementsWithName(
        name: String,
        processor: Processor<in NavigationItem>,
        parameters: FindSymbolParameters
    ) {
        for (key in myIndexKeys) {
            ProgressManager.checkCanceled()
            StubIndex.getInstance().processElements(
                key,
                name,
                parameters.project,
                parameters.searchScope,
                parameters.idFilter,
                MonkeyNamedElement::class.java,
                processor
            )
        }
    }
}

Many custom plugins also use indexes widely. For example, Request mapper (currently not supported, since the same functionality appeared in IDEA), which helps to search for REST method declaration points

Request Mapper uses this code under the hood:

//JavaAnnotationIndex - the usual Subindexbindex for Java for annotations
JavaAnnotationIndex
            .getInstance()
            .get(annotationName, project, GlobalSearchScope.projectScope(project))
            .asSequence()

References

Documentation

At the moment, the API is being changed, so there may be inaccuracies.

References create links between elements. When you press Ctrl+B, you will go to the element to which this link refers. By pressing Ctrl+B again, you will see all the elements that link to this element.

Only the element that defines a name should implement PsiNamedElement (or better PsiNameIdentifierOwner, you can see their use in the Rename section)

@JvmStatic
fun setName(expr: MonkeySimpleRefExpr, name: String): PsiElement {
    val e: PsiElement = MonkeyElementTextFactory.createStatementFromText(expr.project, "$name + 1")
    //newLetExpr must implement PsiNamedElement
  val newLetExpr = PsiTreeUtil.findChildOfType(e, MonkeySimpleRefExpr::class.java)
    if (newLetExpr != null) {
        expr.replace(newLetExpr) 
    }
    return expr
}

In order for the PSI element to provide a link, you need to implement the methods

PsiReference getReference();

PsiReference @NotNull [] getReferences();

@Experimental
default @NotNull Iterable<? extends @NotNull PsiSymbolReference> getOwnReferences() {
    return Collections.emptyList();
}

To simplify the implementation of the PsiReference interface, you can use the PsiReferenceBase. You have to implement the method PsiElement resolve() or ResolveResult [] multiResolve(boolean incompleteCode), which return the referenced elements. When implementing this method, it makes sense to use a specialized cache:

override fun multiResolve(incompleteCode: Boolean): Array<ResolveResult> {
    return ResolveCache.getInstance(psiElement.project).resolveWithCaching(
        this, { referenceBase, _ ->
            referenceBase.resolveInner(false)
                .map { PsiElementResolveResult(it) }
                .toTypedArray()
        },
        true, false
    )
}

After the implementation of this part, IDEA will already be able to provide navigation to the referenced element and back.

Rename

Documentation

Renaming is one of the most popular refactorings (Shift+F6)

2 extension points are responsible for that:

<lang.refactoringSupport language="Monkey"
                         implementationClass="com.github.pyltsin.monkeyplugin.refactor.MonkeyRefactoringSupportProvider"/>
<renameInputValidator implementation="com.github.pyltsin.monkeyplugin.refactor.MonkeyRenameInputValidator"/>

renameInputValidator must implement RenameInputValidator.

refactoringSupport must extend RefactoringSupportProvider.

This class also contains methods to support other types of refactorings. At the moment we are interested in a method that indicates whether in-place editing is supported.

  public boolean isMemberInplaceRenameAvailable(@NotNull PsiElement element, @Nullable PsiElement context) {
    return false;
  }

Now it is required to implement renaming methods. As mentioned before, the PSI element that defines the name should implement the PsiNamedElement interface.

public interface PsiNamedElement extends PsiElement {
  String getName();
  PsiElement setName(@NlsSafe @NotNull String name)
  throws IncorrectOperationException;
}

We are interested in setName method here. One of the easiest ways to implement this method is to create a new PSI element from text, like this

private fun createFileFromText(project: Project, text: String): MonkeyFile {
    return PsiFileFactory.getInstance(project)
        .createFileFromText("A.monkey", MonkeyLanguage.INSTANCE, text) as MonkeyFile
}

And replace the element

expr.replace(newLetExpr)

It remains to implement the renaming of elements that refer to our named PSI element. To do this, you need to implement the method from PsiReference.

PsiElement handleElementRename(@NotNull String newElementName) 
throws IncorrectOperationException;

or you can use com.intellij.psi.PsiReferenceBase.

Markers

IDEA widely uses hints in the form of markers

Examples from go-plugin, Frege, monkey

The required extension point:

<codeInsight.lineMarkerProvider language="Monkey"
implementationClass="com.github.pyltsin.monkeyplugin.editor.MonkeyFunctionLineMarkerProvider"/>

The file must implement LineMarkerProvider interface, whose methods return LineMarkerInfo object. Please note that markers should be linked only to the leaves of the PSI-tree.

Autocompletion

I think everyone who uses IDEA likes how it works with hints. It is very difficult to write a good autocompletion mechanism. But at the same time, you can implement some autocompletion relatively quickly, for example, these:

This completion is implemented with PsiReference.getVariants method, which returns all visible suitable values. Filtering by characters is performed by IDEA itself.

For more complex cases, you can use the extension point:

<completion.contributor language="Monkey"                         implementationClass="com.github.pyltsin.monkeyplugin.completion.MonkeyKeywordCompletionContributor"/>

Your class must implement CompletionContributor abstract class.

Documentation. Examples of implementation from go-plugin, frege, monkey

Testing

Documentation

In IDEA, tests are often several files with the source code of the language that show the state BEFORE and AFTER the action. BasePlatformTestCase class is usually used for it. Usually tests look like this:

myFixture.configureByFiles("RenameTestData.monkey")
myFixture.renameElementAtCaret("test")
myFixture.checkResultByFile("RenameTestData.monkey", "RenameTestDataAfter.monkey", false)

where RenameTestData.monkey and RenameTestDataAfter.monkey are source code files before and after renaming.

The conclusion

This is the end of the story about creating a language plugin for IDEA. Good luck in setting up your IDE for yourself!

How to write your own language plugin for IDEA (part 1)

pyltsin — Tue, 26 Apr 2022 18:22:12 +0000

Disclaimer: I don't work for JetBrains, so there will be and most likely there may be inaccuracies and errors in the article and code.

Introduction

I am working as a backend developer, and sometimes I am tired of the work routine. Going deep into some opensource helps me, besides I am keen on Jetbrains' product and IDEA. IDEA can be customized with plugins, which you can write yourself.
The biggest part of IDEA's code is open-source. Developers from Jetbrains always help you to make pull requests and improve your main tool.
I have already written an article about how to write a small plugin with one inspection. But if you want to understand how IDEA works, you should try to write a language plugin.
Let's do this.
As a "guinea pig" we will take a simple language - Monkey, the interpreter and compiler for which are described in the books on Golang. Since my goal was not to cover everything, the plugin covers a limited subset of this language. The interpreter can be found here.

Example of calculating the Fibonacci number on Monkey:

let fibonacci = fn(x){
                    if (x == 0){
                        0;
                    }
                    else{
                        if (x == 1){
                            return 1;
                        }
                        else{
                            fibonacci(x - 1) + fibonacci(x - 2);
                        };
                    };
};

We will write this plugin with Java and Kotlin (JVM)

Prerequisites

The easiest way to create a new plugin is to use the template. It is being actively developed and had already included the most needed features.

Also, we need DevKit plugin, which you can enable in IDEA

Grammar-Kit plugin will be used to generate a lexer and parser

Examples of other plugins:

Java (in the code of IDEA)
go-plugin (now this plugin is known as Goland)
Haskell
Erlang
Frege
Monkey plugin (in this article)

Creating the basis of a language plugin

The first stages of creating a language plugin are well described in the documentation.

It is required to declare a new language (examples in go-plugin, frege, monkey)

Example for Monkey:

import com.intellij.lang.Language

class MonkeyLanguage : Language("Monkey") {
    companion object {
        @JvmStatic
        val INSTANCE = MonkeyLanguage()
    }
}

Declare an icon (examples in go-plugin, frege, monkey)
Declare a new file type and combine everything together (examples in go-plugin, frege, monkey)

Example for Monkey

import com.intellij.openapi.fileTypes.LanguageFileType
import javax.swing.Icon

class MonkeyFileType : LanguageFileType(MonkeyLanguage.INSTANCE) {
    override fun getName(): String {
        return "Monkey File"
    }

    override fun getDescription(): String {
        return "Monkey language file"
    }

    override fun getDefaultExtension(): String {
        return "monkey"
    }

    override fun getIcon(): Icon {
        return MonkeyIcons.FILE
    }

    companion object {
        @JvmStatic
        val INSTANCE = MonkeyFileType()
    }
}

After that, you need to enable a new file type via the extension point. All the features that plugins provide are enabled through one or more extension points. They must be declared in the file plugin.xml (example for go-plugin, frege). Other examples of using extension points will be given below or can be checked in the documentation.

Example for Monkey (resources/META-INF/plugin.xml)

<extensions defaultExtensionNs="com.intellij">
  <fileType name="Monkey File"
            implementationClass="com.github.pyltsin.monkeyplugin.MonkeyFileType"
            fieldName="INSTANCE"
            language="Monkey"
            extensions="monkey"/>
</extensions>

Creation PSI-tree

It is impossible to describe the whole theory that is required to parse the code in one article. Fundamental knowledge for this topic can be found in the "Dragon Book"

According to this book, the process of the compiler working with the code consists of the following steps:

Lexical Analyzer
Syntax Analyzer
Semantic Analyzer
Intermediate Code Generator
Machine-Independent Code Optimizer
Code Generator
Machine-Dependent Code Optimizer

It is required to implement the first 3 analyzers for any IDE to work successfully:

Lexical analyzer (to read a stream of characters and to group them into tokens)
Parser (to receive a stream of tokens and to build a syntax tree from them - AST)
Semantic analyzer (to use a tree to verify the source code according to some rules).

In IDEA, an analog called PSI tree (Program structure Interface) is used instead of an AST tree.

The process of creating a PSI tree is well shown in the illustration from the documentation:

In order to see PSI-tree, you can use the PSI Viewer (Tools->View PSI structure)

In IDEA, the abstract TreeElement class is mainly used to implement the PSI tree

public abstract class TreeElement extends ElementBase implements ASTNode, Cloneable {
  private TreeElement myNextSibling;
  private TreeElement myPrevSibling;
  private CompositeElement myParent;
...
}

In IDEA, you can use the GrammarKit plugin to create a lexer and a parser.

Lexer

Perhaps, the easiest way to create a lexer for IDEA is to use JFlex. The GrammarKit plugin already contains implementation and allows you to generate a lexer from .bnf file or from .flex file (with more customization options). An example for the Monkey language can be viewed here, a more complex one for Frege - here.

To generate Lexer, you need to configure the Gradle plugin or use the context menu in .flex file - "Run JFlex Generator".

After that, you need to declare a class that implements com.intellij.lexer.Lexer. There is already an adapter for the generated JFlex lexer - com.intellij.lexer.FlexAdapter

Parser

In IDEA, code generation by the GrammarKit plugin is mainly used to create a parser. Unfortunately, there is not much documentation about it and it is only presented in Tutorial and HOWTO.

The grammar of the language is described in BNF. The only difference that is used is ::= as "is".

Example grammar for expressions can be found here

As you can see, the bnf file consists of 2 parts:

the first part describes the meta-information (and the description of tokens, if flex files are not used)
the second part describes the grammar itself.

Let's consider some parts of the meta-information:

parserClass - name and location of the generated parser class

parserUtilClass - a reference to a class containing a set of additional methods for the parser (usually the com.intellij.lang.parser.GeneratedParserUtilBase class or its inheritors)

extends = <some class> - a reference to the base class from which all PSI elements (tree nodes) will be inherited. Usually com.intellij.extapi.psi.ASTWrapperPsiElement or its inheritors.

extends(<regexp for tree nodes>) = <psi-element> (for example: extends(".*expr")=expr) - all psi elements will be inherited from the specified psi element.

psiClassPrefix, psiImplClassSuffix - respectively the prefix for classes and interfaces (usually by the name of the language) and the suffix for implementing interfaces (usually - Impl)

psiPackage and psiImplPackage are respectively a package for interfaces and their implementations.

implements - similar to extends, but for interfaces

elementTypeHolderClass - a generated storage for all types of elements

elementTypeClass - a class for types of elements (not generated, inheritors to com.intellij.psi.tree.IElementType)

elementTypeFactory - creating a factory for generating element types (used for Stub - about them below)

psiImplUtilClass - a class with a set of static methods that are used as an implementation of the required methods for psi elements.
Let's suppose we have such lines (from go-plugin)

ImportSpec ::= [ '.' | identifier ] ImportString {
  stubClass="com.goide.stubs.GoImportSpecStub"
  methods=[getAlias getLocalPackageName shouldGoDeeper isForSideEffects isDot getPath getName isCImport]
}

For that, we need to declare and implement a static method with the same name in psiImplUtilClass:

public static String getAlias(@NotNull GoImportSpec importSpec)

After that getAlias will be generated as:

  public String getAlias() {
    return GoPsiImplUtil.getAlias(this);
  }

Now let's move on to the bnf rules themselves. Modifiers can be used for each rule (for example, private, fake, and so on). Their description is given here. So, for example private in

private boolean_group ::= xor_expr | between_expr | is_not_expr

indicates that the PSI element for boolean_group will not be generated.

If it is not possible to describe the grammar in a bnf file correctly, then it will be useful to describe it in code using external rules.

One of the important parts of grammar is the rules of dealing with errors. Two keywords are used for this: pin and recoverWhile.

pin - indicates the token number. As soon as we get to it, the parser starts waiting only for the current declaration. For example, declaring a structure in Golang

Struct Type ::= struct '{' Fields? '}' {pin=1}

recoverWhile - specifies which tokens can be consumed only after matching with all rules is completed. Recommendations for the use of this attribute are described here.

You should also pay attention to the recommendations for parsing expressions based on priority.

It seems to me that creating correct and convenient grammar rules for future work is one of the most difficult parts of implementing a plugin for a language. To get started, you can use examples: go-plugin, Frege, Monkey (for Monkey, only a subset of this language is implemented for simplification).

After creating a bnf file and generating a parser from it, you need to define the class for File (inheritor from com.intellij.extapi.psi.PsiFileBase) (examples from go-plugin, Frege, Monkey) and the definition class for your parser (heir from com.intellij.lang.ParserDefinition) (example from go-plugin, Frege, Monkey), and then enable it via the extension point.

<lang.parserDefinition language="Monkey"
implementationClass="com.github.pyltsin.monkeyplugin.parser.MonkeyParserDefinition"/>

Annotators

In the previous parts, we looked at how a lexer and a parser are created. Now let's move on to the third part - semantic analysis. Studying the code of IDEA and its plugins, I found two ways to implement it (excluding inspections).

The first way is used in a plugin for Java language. Consider the following invalid code:

IDEA highlighted it and said Operator'-'cannot be applied to 'java.lang.String', 'java.lang.String'. It works thanks to the following extension point:

<highlightVisitor implementation=
"com.intellij.codeInsight.daemon.impl.analysis.HighlightVisitorImpl"/>

The class must implement com.intellij.codeInsight.daemon.impl.HighlightVisitor

public interface HighlightVisitor {

  boolean suitableForFile(@NotNull PsiFile file);

  void visit(@NotNull PsiElement element);

  boolean analyze(@NotNull PsiFile file, 
                  boolean updateWholeFile, 
                  @NotNull HighlightInfoHolder holder, 
                  @NotNull Runnable action);
}

We use the method analyze to configure, launch (action.run()) and close resources

//From HighlightVisitorImpl
    @Override
  public void visit(@NotNull PsiElement element) {
  // some code
      element.accept(this);
  // some code
  }

//Example for ClsJavaModuleImpl
  @Override
  public void accept(@NotNull PsiElementVisitor visitor) {
    if (visitor instanceof JavaElementVisitor) {
      ((JavaElementVisitor)visitor).visitModule(this);
    }
    else {
      visitor.visitElement(this);
    }
  }

The visitor pattern is used there.
HighlightVisitorImpl extends JavaElementVisitor

public abstract class JavaElementVisitor extends PsiElementVisitor {
  public void visitAnonymousClass(PsiAnonymousClass aClass) {
    visitClass(aClass);
  }

  public void visitArrayAccessExpression(PsiArrayAccessExpression expression) {
    visitExpression(expression);
  }

  public void visitArrayInitializerExpression(PsiArrayInitializerExpression expression) {
    visitExpression(expression);
  }
...

The second way is applied in go-plugin and Frege.
I used it too in the Monkey plugin. It uses an annotator extension point.

<annotator language="Monkey"
implementationClass="com.github.pyltsin.monkeyplugin.annotator.MonkeyWarningAnnotator"/>

Your class must implement the next interface:

public interface Annotator {
  void annotate(@NotNull PsiElement element, 
                @NotNull AnnotationHolder holder);
}

An error message can be registered with next way:

holder.newAnnotation(HighlightSeverity.ERROR, errorMsg)
.range(element)
.create()

Examples for Frege, go-plugin, Monkey.

For Monkey language, I have currently implemented 2 rules - the inability to resolve links (resolve references - about them below) and the simple type check (via DSL).

Highlighting brackets

In this part, we'll look at a couple more extension points.

The first point of extension: lang.braceMatcher.

<lang.braceMatcher language="Monkey"
implementationClass="com.github.pyltsin.monkeyplugin.editor.MonkeyBraceMatcher"/>

This extension point enables highlighting a pair of brackets and adding a closing bracket

The class must implement com.intellij.lang interface.PairedBraceMatcher

The implementation that I made for the Monkey language can be viewed here, for the go-plugin here, for Java - here and here.

The second point of extension: highlightVisitor. I have already mentioned it when we were creating a semantic analyzer. I didn't use it in my plugin, but it is used in the popular Rainbow Brackets plugin, which colors pairs of brackets in unique colors.

If you look in his plugin.xml, then you can find this line

<highlightVisitor implementation="com.github.izhangzhihao.rainbow.brackets.visitor.DefaultRainbowVisitor"/>

The class implements the interface - com.intellij.CodeInsight.daemon.impl.HighlightVisitor. The implementation can be viewed here. The coloring takes place in the com.github.izhangzhihao.rainbow.brackets method.visitor.RainbowHighlightVisitor#setHighlightInfo

holder.add(HighlightInfo
    .newHighlightInfo(rainbowElement)
    .textAttributes(attr)
    .range(element)
    .create())

Continuation here.

How to write your own language plugin for IDEA (part 1)

pyltsin — Sun, 17 Apr 2022 17:27:32 +0000

Introduction

I am working as a backend-developer, and sometimes I am tired of work routine. Deeping into some opensource helps me and I really keen on Jetbrains' product and IDEA. IDEA can be customized with plugins, which you can write yourself.
The most part of IDEA's code is open-source. Developers from Jetbrains always help you to make pull request and improve your main tool.
I already have the article about how to write a small plugin with one inspection. But if you want to understand how IDEA works, you should try to write a language plugin.
Let's do this.
As a "guinea pig" we will take the simple language - Monkey, the interpreter and compiler for it is described in the books on Golang. Since my goal was not to cover everything, the plugin covers some limited subset of this language. The interpreter can be found here.

Example of calculating the Fibonacci number on Monkey:

let fibonacci = fn(x){
                    if (x == 0){
                        0;
                    }
                    else{
                        if (x == 1){
                            return 1;
                        }
                        else{
                            fibonacci(x - 1) + fibonacci(x - 2);
                        };
                    };
};

We will write this plugin with Java and Kotlin (JVM)

Prerequisites

The easiest way to create a new plugin is using the template. It is being actively developing and had already included most needed features.

Also we need DevKit plugin, which you can enable in IDEA

Grammar-Kit plugin will be used to generate a lexer and parser

Examples of other plugins:

Java (in the code of IDEA)
go-plugin (now this plugin is known as Goland)
Haskell
Erlang
Frege
Monkey plugin (in this article)

Creating the basis of a language plugin

The first stages of creating a language plugin are well described in the documentation.

It is required to declare a new language (examples in go-plugin, frege, monkey)

Example for Monkey:

import com.intellij.lang.Language

class MonkeyLanguage : Language("Monkey") {
    companion object {
        @JvmStatic
        val INSTANCE = MonkeyLanguage()
    }
}

Declare an icon (examples in go-plugin, frege, monkey)
Declare a new file type and combine everything together (examples in go-plugin, frege, monkey)

Example for Monkey

import com.intellij.openapi.fileTypes.LanguageFileType
import javax.swing.Icon

class MonkeyFileType : LanguageFileType(MonkeyLanguage.INSTANCE) {
    override fun getName(): String {
        return "Monkey File"
    }

    override fun getDescription(): String {
        return "Monkey language file"
    }

    override fun getDefaultExtension(): String {
        return "monkey"
    }

    override fun getIcon(): Icon {
        return MonkeyIcons.FILE
    }

    companion object {
        @JvmStatic
        val INSTANCE = MonkeyFileType()
    }
}

Example for Monkey (resources/META-INF/plugin.xml)

<extensions defaultExtensionNs="com.intellij">
  <fileType name="Monkey File"
            implementationClass="com.github.pyltsin.monkeyplugin.MonkeyFileType"
            fieldName="INSTANCE"
            language="Monkey"
            extensions="monkey"/>
</extensions>

Creation PSI-tree

Unfortunately, it is impossible to describe the whole theory that is required to parse the code in one article. Fundamental knowledge for this topic can be found in the "Dragon Book"

According to this book, the process of the compiler working with the code consists of the following steps:

Lexical Analyzer
Syntax Analyzer
Semantic Analyzer
Intermediate Code Generator
Machine-Independent Code Optimizer
Code Generator
Machine-Dependent Code Optimizer

For any IDE to work successfully, it is required to implement the first 3 analyzers:

Lexical analyzer (to read a stream of characters and to group them into tokens)
Parser (to receive a stream of tokens and to build a syntax tree from them - AST)
Semantic analyzer (to use a tree to verify the source code according to some rules).

In IDEA, instead of an AST tree, an analog is used - a PSI tree (Program structure Interface).

The process of creating a PSI tree is well shown in the illustration from the documentation:

In order to see PSI-tree, you can use the PSI Viewer (Tools->View PSI structure)

In IDEA, the abstract TreeElement class is mainly used to implement the PSI tree

public abstract class TreeElement extends ElementBase implements ASTNode, Cloneable {
  private TreeElement myNextSibling;
  private TreeElement myPrevSibling;
  private CompositeElement myParent;
...
}

In IDEA, you can use the GrammarKit plugin to create a lexer and a parser.

Lexer

Perhaps, the easiest way to create a lexer for IDEA is to use JFlex. The GrammarKit plugin already contains an implementation and allows you to generate a lexer from .bnf file or from .flex file (with more customization options). An example for the Monkey language can be viewed here, a more complex one for Frege - here.

To generate Lexer, you need to configure the Gradle plugin or use the context menu in .flex file - "Run JFlex Generator".

After that, you need to declare a class that implements com.intellij.lexer.Lexer. There is already an adapter for the generated JFlex lexer - com.intellij.lexer.FlexAdapter

Parser

In IDEA, code generation by the GrammarKit plugin is mainly used to create a parser. Unfortunately, there is not much documentation about it and it is only presented in Tutorial and HOWTO.

The grammar of the language is described in BNF. The only difference that is used is ::= as "is".

Example grammar for expressions you can find here

As you can see, the bnf file consists of 2 parts:

the first part describes the meta information (and the description of tokens, if flex files are not used)
the second part describes the grammar itself.

Let's consider some part of the meta-information:

parserClass - name and location of the generated parser class

parserUtilClass - a reference to a class containing a set of additional methods for the parser (usually the com.intellij.lang.parser.GeneratedParserUtilBase class or its inheritors)

extends = <some class> - a reference to the base class from which all PSI elements (tree nodes) will be inherited. Usually com.intellij.extapi.psi.ASTWrapperPsiElement or its inheritors.

extends(<regexp for tree nodes>) = <psi-element> (for example: extends(".*expr")=expr) - all psi elements will be inherited from the specified psi element.

psiClassPrefix, psiImplClassSuffix - respectively the prefix for classes and interfaces (usually by the name of the language) and the suffix for implementing interfaces (usually - Impl)

psiPackage and psiImplPackage are respectively a package for interfaces and their implementations.

implements - similar to extends, but for interfaces

elementTypeHolderClass - a generated storage for all types of elements

elementTypeClass - a class for types of elemets (not generated, inheritors to com.intellij.psi.tree.IElementType)

elementTypeFactory - creating a factory for generating element types (used for Stub - about them below)

psiImplUtilClass - a class with a set of static methods that are used as an implementation of the required methods for psi elements.
Let's suppose we have such lines (from go-plugin)

ImportSpec ::= [ '.' | identifier ] ImportString {
  stubClass="com.goide.stubs.GoImportSpecStub"
  methods=[getAlias getLocalPackageName shouldGoDeeper isForSideEffects isDot getPath getName isCImport]
}

For that, we need declare and implement static method with the same name in psiImplUtilClass:

public static String getAlias(@NotNull GoImportSpec importSpec)

After that getAlias will be generated as:

  public String getAlias() {
    return GoPsiImplUtil.getAlias(this);
  }

Now let's move on to the bnf rules themselves. Modifiers can be used for each rule (for example, private, fake, and so on). Their description is given here. So for example private in

private boolean_group ::= xor_expr | between_expr | is_not_expr

indicates that the PSI element for boolean_group will not be generated.

If it is not possible to correctly describe the grammar in a bnf file, then it will be useful to describe it in code using external rules.

One of the important parts of grammar is the rules for dealing with errors. Two keywords are used for this: pin and recoverWhile.

pin - indicates the token number, as soon as we get to which, the parser starts waiting only for the current declaration. For example, declaring a structure in Golang

Struct Type ::= struct '{' Fields? '}' {pin=1}

recoverWhile - specifies which tokens can be consumed after matching with all rules is completed. Recommendations for the use of this attribute are described here.

You should also pay attention to the recommendations for parsing expressions based on priority.

It seems to me that creating a correct and convenient grammar rules for future work is one of the most difficult parts of implementing a plugin for a language. To get started, you can use examples: go-plugin, Frege, Monkey (for Monkey, only a subset of this language is implemented for simplification).

After creating a bnf and generating a parser from it, you need to define the class for File (heir from com.intellij.extapi.psi.PsiFileBase) (examples from go-plugin, Frege, Monkey) and the definition class for your parser (heir from com.intellij.lang.ParserDefinition) (example from go-plugin, Frege, Monkey), and then enable it via the extension point.

<lang.parserDefinition language="Monkey"
implementationClass="com.github.pyltsin.monkeyplugin.parser.MonkeyParserDefinition"/>

Annotators

The first way is used in a plugin for the Java language. Consider the following invalid code:

IDEA, of course, highlighted it and said Operator'-'cannot be applied to 'java.lang.String', 'java.lang.String'. It works thanks to the following extension point:

<highlightVisitor implementation=
"com.intellij.codeInsight.daemon.impl.analysis.HighlightVisitorImpl"/>

The class must implement com.intellij.codeInsight.daemon.impl.HighlightVisitor

public interface HighlightVisitor {

  boolean suitableForFile(@NotNull PsiFile file);

  void visit(@NotNull PsiElement element);

  boolean analyze(@NotNull PsiFile file, 
                  boolean updateWholeFile, 
                  @NotNull HighlightInfoHolder holder, 
                  @NotNull Runnable action);
}

We use the method analyze to configure, launch (action.run()) and close resources

//From HighlightVisitorImpl
    @Override
  public void visit(@NotNull PsiElement element) {
  // some code
      element.accept(this);
  // some code
  }

//Example for ClsJavaModuleImpl
  @Override
  public void accept(@NotNull PsiElementVisitor visitor) {
    if (visitor instanceof JavaElementVisitor) {
      ((JavaElementVisitor)visitor).visitModule(this);
    }
    else {
      visitor.visitElement(this);
    }
  }

There is used the visitor pattern.
HighlightVisitorImpl extends JavaElementVisitor

public abstract class JavaElementVisitor extends PsiElementVisitor {
  public void visitAnonymousClass(PsiAnonymousClass aClass) {
    visitClass(aClass);
  }

  public void visitArrayAccessExpression(PsiArrayAccessExpression expression) {
    visitExpression(expression);
  }

  public void visitArrayInitializerExpression(PsiArrayInitializerExpression expression) {
    visitExpression(expression);
  }
...

The second way is applied in go-plugin and Frege.
In the Monkey plugin, I used it too. It uses annotator extension point.

<annotator language="Monkey"
implementationClass="com.github.pyltsin.monkeyplugin.annotator.MonkeyWarningAnnotator"/>

Your class must implement the next interface:

public interface Annotator {
  void annotate(@NotNull PsiElement element, 
                @NotNull AnnotationHolder holder);
}

An error message can be registered with next way:

holder.newAnnotation(HighlightSeverity.ERROR, errorMsg)
.range(element)
.create()

Examples for Frege, go-plugin, Monkey.

For the Monkey language, I have currently implemented 2 rules - the inability to resolve links (resolve references - about them below) and a simple type check (via DSL).

Highlighting brackets

In this part, we'll look at a couple more extension points.

The first point of extension: lang.braceMatcher.

<lang.braceMatcher language="Monkey"
implementationClass="com.github.pyltsin.monkeyplugin.editor.MonkeyBraceMatcher"/>

This extension point enable highlighting a pair of brackets and adding a closing bracket

The class must implement com.intellij.lang interface.PairedBraceMatcher

The implementation that I made for the Monkey language can be viewed here, for the go-plugin here, for Java - here and here.

If you look in his plugin.xml, then you can find this line

<highlightVisitor implementation="com.github.izhangzhihao.rainbow.brackets.visitor.DefaultRainbowVisitor"/>

holder.add(HighlightInfo
    .newHighlightInfo(rainbowElement)
    .textAttributes(attr)
    .range(element)
    .create())

Continuation here.

From Java to Kotlin. There and back again

pyltsin — Fri, 01 Apr 2022 20:08:38 +0000

In this article, I would like to consider the problems and their solutions, which we encountered during the migration of our small microservice from Java to Kotlin.

Stack

Java 11
Spring Web MVC (в рамках Spring Boot)
Spring Data JPA
Map Struct
Lombok
Maven

Beginning

Firstly, I would recommend anyone, who wants to put Kotlin in your project to start from tests. During this process, we configure almost all you need. You go through these steps:

Configure the project
Add necessary libraries
Catch many errors
And, finally, migrate a part of your test.

At the same time, the main code responsible for the business logic is not affected.

Add kotlin

Maven

As usual, we start from https://start.spring.io/

Let's compare 2 pom.xml - one for Java and one for Kotlin.

Add Kotlin version

<properties>
 ....
  <kotlin.version>1.5.31</kotlin.version>
</properties>

Add the standard library for Kotlin and Jackson

<dependency>
  <groupId>com.fasterxml.jackson.module</groupId>
  <artifactId>jackson-module-kotlin</artifactId>
</dependency>
<dependency>
  <groupId>org.jetbrains.kotlin</groupId>
  <artifactId>kotlin-reflect</artifactId>
</dependency>
<dependency>
  <groupId>org.jetbrains.kotlin</groupId>
  <artifactId>kotlin-stdlib-jdk8</artifactId>
</dependency>

Add build section

    <build>
        <sourceDirectory>${project.basedir}/src/main/kotlin</sourceDirectory>
        <testSourceDirectory>${project.basedir}/src/test/kotlin</testSourceDirectory>
        <plugins>
          .....
            <plugin>
                <groupId>org.jetbrains.kotlin</groupId>
                <artifactId>kotlin-maven-plugin</artifactId>
                <configuration>
                    <args>
                        <arg>-Xjsr305=strict</arg>
                    </args>
                    <compilerPlugins>
                        <plugin>spring</plugin>
                        <plugin>jpa</plugin>
                    </compilerPlugins>
                </configuration>
                <dependencies>
                    <dependency>
                        <groupId>org.jetbrains.kotlin</groupId>
                        <artifactId>kotlin-maven-allopen</artifactId>
                        <version>${kotlin.version}</version>
                    </dependency>
                    <dependency>
                        <groupId>org.jetbrains.kotlin</groupId>
                        <artifactId>kotlin-maven-noarg</artifactId>
                        <version>${kotlin.version}</version>
                    </dependency>
                </dependencies>
            </plugin>
        </plugins>
    </build>

More information about kotlin-maven-plugin you can find here

But this configuration is not appropriate if you want to use Java with Kotlin. In this case you need to configure maven-compiler-plugin (documentation)

<build>
  <plugins>
    .....
    <plugin>
      <groupId>org.jetbrains.kotlin</groupId>
      <artifactId>kotlin-maven-plugin</artifactId>
      <executions>
        <execution>
          <id>compile</id>
          <goals>
            <goal>compile</goal>
          </goals>
          <configuration>
            <sourceDirs>
              <sourceDir>${project.basedir}/src/main/kotlin</sourceDir>
              <sourceDir>${project.basedir}/src/main/java</sourceDir>
            </sourceDirs>
          </configuration>
        </execution>
        <execution>
          <id>test-compile</id>
          <goals> <goal>test-compile</goal> </goals>
          <configuration>
            <sourceDirs>
              <sourceDir>${project.basedir}/src/test/kotlin</sourceDir>
              <sourceDir>${project.basedir}/src/test/java</sourceDir>
            </sourceDirs>
          </configuration>
        </execution>
      </executions>
      <configuration>
        <args>
          <arg>-Xjsr305=strict</arg>
        </args>
        <compilerPlugins>
          <plugin>spring</plugin>
          <plugin>jpa</plugin>
        </compilerPlugins>
      </configuration>
      <dependencies>
        <dependency>
          <groupId>org.jetbrains.kotlin</groupId>
          <artifactId>kotlin-maven-allopen</artifactId>
          <version>${kotlin.version}</version>
        </dependency>
        <dependency>
          <groupId>org.jetbrains.kotlin</groupId>
          <artifactId>kotlin-maven-noarg</artifactId>
          <version>${kotlin.version}</version>
        </dependency>
      </dependencies>
    </plugin>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-compiler-plugin</artifactId>
      <executions>
        <!-- Replacing default-compile as it is treated specially by maven -->
        <execution>
          <id>default-compile</id>
          <phase>none</phase>
        </execution>
        <!-- Replacing default-testCompile as it is treated specially by maven -->
        <execution>
          <id>default-testCompile</id>
          <phase>none</phase>
        </execution>
        <execution>
          <id>java-compile</id>
          <phase>compile</phase>
          <goals>
            <goal>compile</goal>
          </goals>
        </execution>
        <execution>
          <id>java-test-compile</id>
          <phase>test-compile</phase>
          <goals>
            <goal>testCompile</goal>
          </goals>
        </execution>
      </executions>
    </plugin>
  </plugins>
</build>

Some information about plugins All-open and No-arg you can find below.

Kotlin plugins

Kotlin uses compiler plugins to change ast-tree during compilation. By default, spring-initializer adds 2 plugins: all-open and no-arg. Also, kapt and the plugin for Lombok are popular.

all-open

By default, all classes in Kotlin are final and they can not be overridden, so Spring can't use them to create proxy-class. All-open plugin adds open to classes with specified annotations.

For spring there is a pre-configured kotlin-spring plugin.
It works only with these annotations:

@Component
@Async
@Transactional
@Cacheable
@SpringBootTest

If you want to work with your custom annotations, you need to configure kotlin-allopen.

Also, I would like to highlight, that there are no JPA annotations, and you have to add them if you use JPA repositories.

No-arg

This plugin adds an empty constructor to each class. For JPA there is kotlin-jpa. It works with annotations:

@Entity
@Embeddable
@MappedSuperclass

But it doesn't add open to these classes.

Kapt

It is an adapter for annotation processors (documentation). It is used, for example, by mapstruct to generate code for mappers with Kotlin.

Example, how you can add kapt plugin:

<plugin>
  <groupId>org.jetbrains.kotlin</groupId>
  <artifactId>kotlin-maven-plugin</artifactId>
  <version>${kotlin.version}</version>
  <executions>
    <execution>
        <id>kapt</id>
        <goals>
            <goal>kapt</goal>
        </goals>
        <configuration>
            <sourceDirs>
                <sourceDir>src/main/kotlin</sourceDir>
                <sourceDir>src/main/java</sourceDir>
            </sourceDirs>
            <annotationProcessorPaths>
                <annotationProcessorPath>
                    <groupId>org.mapstruct</groupId>
                    <artifactId>mapstruct-processor</artifactId>
                    <version>${mapstruct.version}</version>
                </annotationProcessorPath>
            </annotationProcessorPaths>
        </configuration>
    </execution>
  ...
  </executions>
  ....
</plugin>

In my experience, this plugin can break your build, if you use Lombok because the plugin can't parse java-file where Lombok methods are used.

Lombok plugin

This plugin is used to cope with this problem, but it still has only beta versions and doesn't support @Builder

Cook Spring

Spring is an awesome framework, and, of course, it supports
Kotlin. But there are some interesting features.

Some presentations:

Features:

the main entry point with Kotlin:

@SpringBootApplication
class DemoKotlinApplication

fun main(args: Array<String>) {
    runApplication<DemoKotlinApplication>(*args)
}

some new extension-methods for Kotlin were added, like RestOperationsExtensions.kt
it is recommended to use val arguments with constructor

@Component
class YourBean(
  private val mongoTemplate: MongoTemplate, 
  private val solrClient: SolrClient
)

but there are some other options, for example, you can use latenin:

@Component
class YourBean {

    @Autowired
    lateinit var mongoTemplate: MongoTemplate

    @Autowired
    lateinit var solrClient: SolrClient
}

It is similar to

@Component
public class YourBean {
   @Autowired
   public MongoTemplate mongoTemplate;
   @Autowired
   public SolrClient solrClient;
}

Also, you can use injection with set-methods:

    var hello: HelloService? = null
        @Autowired
        set(value) {
            field = value
            println("test")
        }

You can create properties classes with @ConstructorBinding

@ConfigurationProperties("test")
@ConstructorBinding
class TestConfig(
    val name:String
)

or with lateinit

@ConfigurationProperties("test")
class TestConfig {
    lateinit var name: String
}

if you want to generate meta-information, you should configure spring-boot-configuration-processor with kapt

Kotlin with Hibernate

The main errors and problems are described in the Haulmont article.

Once again, I will draw your attention that you need to configure no-args and all-open plugins and implement hashCode and equals methods.

Kotlin with Jackson

You should add Jackson Module Kotlin in your project. After that, you can't specify a type of object explicitly.

import com.fasterxml.jackson.module.kotlin.jacksonObjectMapper
import com.fasterxml.jackson.module.kotlin.readValue

data class MyStateObject(val name: String, val age: Int)

...
val mapper = jacksonObjectMapper()

val state = mapper.readValue<MyStateObject>(json)
// or
val state: MyStateObject = mapper.readValue(json)
// or
myMemberWithType = mapper.readValue(json)

Kotlin with MapStruct

MapStruct works through the annotation processor. Therefore, it is necessary to configure kapt correctly. At the same time, if models use Lombok annotations, then there could be problems with mappers.

<plugin>
  <groupId>org.jetbrains.kotlin</groupId>
  <artifactId>kotlin-maven-plugin</artifactId>
  <version>${kotlin.version}</version>
  <executions>
    <execution>
        <id>kapt</id>
        <goals>
            <goal>kapt</goal>
        </goals>
        <configuration>
            <sourceDirs>
                <sourceDir>src/main/kotlin</sourceDir>
                <sourceDir>src/main/java</sourceDir>
            </sourceDirs>
            <annotationProcessorPaths>
                <annotationProcessorPath>
                    <groupId>org.mapstruct</groupId>
                    <artifactId>mapstruct-processor</artifactId>
                    <version>${mapstruct.version}</version>
                </annotationProcessorPath>
            </annotationProcessorPaths>
        </configuration>
    </execution>
  ...
  </executions>
  ....
</plugin>

Kotlin with Lombok

Honestly, it is a pain.
Of course, you can try to use Lombok compiler plugin, but often if you use Kapt and Lombok at the same time, you may encounter many problems.
By default, kapt uses all annotation processors and disables their work with javac, but for lombok it doesn't work correctly and you need to disable this behavior.

<plugin>
    <groupId>org.apache.maven.plugins</groupId>
    <artifactId>maven-compiler-plugin</artifactId>
    <version>3.5.1</version>
    <configuration>
        <source>1.8</source>
        <target>1.8</target>
        <annotationProcessorPaths>
            <annotationProcessorPath>
                <groupId>org.projectlombok</groupId>
                <artifactId>lombok</artifactId>
                <version>${lombok.version}</version>
            </annotationProcessorPath>
        </annotationProcessorPaths>
    </configuration>
</plugin>

At the same time, Lombok works correctly with kapt only if the annotation processor, which is used by kapt, does not depend on Lombok. In our case, it was not true. It was the reason, why we had to translate the entire domain model to Kotlin in the first step. Also one of the options is to use Delombok.

Kotlin with Mockito

Mockito does not work correctly with Kotlin types out of the box. Spring recommends using Mockk. There is also a special module for Mockito, which adds support for Kotlin - Mockito-Kotlin.

In our project we used Mockito-Kotlin. We found only one problem: you need to be careful, because many methods are duplicated in different modules, for example, any() will now be in 2 places - org.mockito.kotlin and org.mockito.Mockito.

Kotlin and logging

We chose kotlin-logging. It is a really convenient library and you can use it like this:

import mu.KotlinLogging
private val logger = KotlinLogging.logger {} 
class FooWithLogging {
    val message = "world"
    fun bar() {
        logger.debug { "hello $message" }
    }
}

Conclusions

I would like to finish the article with brief conclusions. Using Java and Kotlin together in one project requires additional settings, but almost everything is solved and we get the opportunity to use 2 languages in one project. The biggest problem for us was incompatibility Lombok and Kotlin.

References

How to create a new inspecion for IDEA

pyltsin — Tue, 15 Mar 2022 07:48:53 +0000

Disclaimer: I don't work for JetBrains, so my code may have some errors and it is only an example.

Introduction

Every large company has its code style. And it is necessary to make people use it.

That's why all checks should be automated (ideally, with CI). For me, the most useful tool is your IDE. We are going to write a simple inspection for IDEA (might be the most popular IDE for Java)

Every IDEA is a base part and a set of plugins for it. Almost every feature can be considered as a plugin, so adding a new one is easy;

Technical specification

One of the most popular questions in a technical interview is about HashMap and equals and hashCode (links for hashCode, HashMap)

So in our company, we have the rule, that every class which we use as a key in HashMap (HashCode) must override equalsand hashCode. Unfortunately, IDEA doesn't contain an inspection for that (issue), but we can fix it!

We are going to support the next expressions:

new HashMap<>();
new HashSet<>();
.collect(Collectors.toSet());
.collect(Collectors.toMap());

Where do we begin?

The start points are:

Now we have a great template , which contains a useful todo list

Template ToDo list

[x] Create a new [IntelliJ Platform Plugin Template][template] project.

[ ] Verify the pluginGroup, plugin ID and sources package.

[ ] Review the Legal Agreements.

[ ] Publish a plugin manually for the first time.

[ ] Set the Plugin ID in the above README badges.

[ ] Set the Deployment Token.

[ ] Click the Watch button on the top of the [IntelliJ Platform Plugin Template][template] to be notified about releases containing new features and fixes.

Also, there are several simple examples, which help us to figure out how all is working.

Implementation

The source code of the plugin is here.

Every plugin for IDEA must contain a file resources/META-INF/plugin.xml, which describes used extension points. if we want to create a new inspection, we have to use localInspection and implement LocalInspectionTool or AbstractBaseJavaLocalInspectionTool for Java)

<extensions defaultExtensionNs="com.intellij">
  <localInspection language="JAVA"
                   displayName="Sniffer: Using HashMap with default hashcode"
                   groupPath="Java"
                   groupBundle="messages.SnifferInspectionsBundle"
                   groupKey="group.names.sniffer.probable.bugs"
                   enabledByDefault="true"
                   level="WEAK WARNING"
                   implementationClass="com.github.pyltsin.sniffer.EqualsHashCodeOverrideInspection"/>
</extensions>

The necessary method is buildVisitor

public PsiElementVisitor buildVisitor(@NotNull final ProblemsHolder holder,
                                        final boolean isOnTheFly, 
                                        @NotNull LocalInspectionToolSession session)

It is an example of Visitor pattern

To work with your code, IDEA creates PSI Tree (PSI - Program Structure Interface). You can see this tree, using PSI Viewer (Tools->View PSI Structure)

If you want to read more about PSI you have to use the documentation

Our visitor will go through all leaves of the tree and decide if it contains something wrong or not.
Let's start with an easy case:

The leaf new HashMap<>() matches PsiNewExpression, so we have to override this method:

    override fun buildVisitor(
        holder: ProblemsHolder,
        isOnTheFly: Boolean,
        session: LocalInspectionToolSession
    ): PsiElementVisitor {
        return object : JavaElementVisitor() {
            override fun visitNewExpression(expression: PsiNewExpression?) {
                super.visitNewExpression(expression)
            }
        }

IDEA contains a lot of useful utils classes, methods, constants, for example, PsiReferenceUtil, PsiTypesUtil.

Firstly, I need to check if this leaf implements HashMapor HashSet

expression.classOrAnonymousClassReference?.qualifiedName 
in (JAVA_UTIL_HASH_MAP, JAVA_UTIL_HASH_SET)

JAVA_UTIL_HASH_MAP, JAVA_UTIL_HASH_SET- String constants from com.intellij.psi.CommonClassNames. This class contains almost all important names.

Secondly, I need to get a class of the key. Thank IDEA, it has already been done for us.

val keyType: PsiType = 
expression.classOrAnonymousClassReference?.parameterList?.typeArguments[0]

Then I need to check this class if it overrides hashCode and equals method:

private fun hasOverrideHashCode(psiType: PsiType): Boolean {
  // get PsiClass
  val psiClass = PsiTypesUtil.getPsiClass(psiType) 
   // get similar methods by name
  val methods: Array<PsiMethod> =
  psiClass?.findMethodsByName(HardcodedMethodConstants.HASH_CODE, false) ?: arrayOf()
  // check, if it is hashCode
  return methods.any { MethodUtils.isHashCode(it) }
}

Finally, I have to register the problem:

holder.registerProblem(
        expression,
        "hashCode is not overriden"
)

Now let's consider an example with Stream:

Map<Clazz2, Clazz2> collect1 = Stream.of(new Clazz2(), new Clazz2())
.collect(Collectors.toMap(t -> t, t -> t));

In this case we work with PsiMethodCallExpression

override fun visitMethodCallExpression(expression: PsiMethodCallExpression?)

For the first check CallMatcher is useful:

val matcher = CallMatcher.instanceCall(JAVA_UTIL_STREAM_STREAM, "collect")
val isCollect = matcher.matches(expression)

After that, Collectors.toMap() must be checked:

val collectorExpression = expression.argumentList.expressions[0]
val isToMap = CallMatcher.staticCall(JAVA_UTIL_STREAM_COLLECTORS, "toMap")
  .matches(collectorExpression)

To get class the next method can be applied:

val psiType = expression.methodExpression.type.parameters[0]

After that, we can reuse our previous code to check overriden equals and hashCode.

For tests, Idea has LightJavaInspectionTestCaseclass. (examples)

Finally, our result:

Of course, you can simply enhance these ideas for other cases.

It is really important to get to know your tools!