Commit Teller - Problems during the development

🚨 Problems

After the start I knew that I'll be integrating OpenAI's API and the GitHub API into the application, and I wanted to make sure I can go native with Quarkus. Going native requires GraalVM, but it is not enough, your code must be compatible with the native generation, and at after the first dependencies I pulled in I was not checking it frequently if my code is still able to be compiled into native.

🔨 Dependencies

I only needed two important dependencies that can fulfil my requirements:

• jcabi-github - The lib to access the GitHub API easily
• openai-java - The lib to access the OpenAI's API easily

🚀 Why did I want to use 3rd party libs?

I wanted to move quickly, I checked the GitHub's API, it looked fine, but the request and response objects can be really big, so if there is any lib that can help me to skip writing a lot of boilerplate code, I try to utilize them.

OpenAI integration is not different, I wanted to skip writing the boilerplate and move fast.

🧐 Problems and solution with the OpenAI API

The openai-java has 3 modules that the developers can use, and I needed all of them to fulfil the requirements.

• api
• client
• server

The usage of the lib was easy, multiple features are implemented but for me the prompt completion was the needed one. It has small request and response objects, with a client class that can invoke the API.

The problems arrived when I tried to compile it to native, and in the background the lib uses the Retrofit library, which in my opinion created the problem. I tried to dig deeper, but a few hours just passed, and I said that is enough, I know Quarkus has an extension to invoke REST APIs, I can do it super fast by just writing the request and response classes.

The client interface:

@RegisterRestClient
public interface OpenAIClient {

    @POST
    @Path("/v1/completions")
    @ClientHeaderParam(
      name = "Authorization",
      value = "Bearer {token}"
    )
    CompletionResponse postCompletion(
      CompletionRequest request,
      @NotBody String token);
}

The request class:

public record CompletionRequest(
    String prompt,
    String model,
    double temperature,
    @JsonProperty("max_tokens")
    int maxTokens
) {
}

And the response class:

public record CompletionResponse(List<Completion> choices) {
    public record Completion(String text) {
    }
}

And the configuration for the client:

quarkus.rest-client."org.rodnansol
.committeller.core.language
.openai.OpenAIClient"
.url=https://api.openai.com

That's it, and poor me was looking for libs that can help me with that :) It took less than 10 minutes to write it and wire it into the existing code.

I used records, because these classes are just data classes, I'm not going to manipulate them later, just use the results.

The point of execution:

@ApplicationScoped
public class OpenAILanguageProcessor
    implements LanguageProcessor {
    private static final Logger LOGGER = LoggerFactory.getLogger(OpenAILanguageProcessor.class);
    private final OpenAIProperties openAIProperties;
    private final OpenAIClient openAIClient;

    public OpenAILanguageProcessor(
        OpenAIProperties openAIProperties,
        @RestClient OpenAIClient openAIClient) {
        this.openAIProperties = openAIProperties;
        this.openAIClient = openAIClient;
    }

    @Override
    public ProcessResult processPrompt(
      ProcessPromptCommand command
) {
        CompletionResponse completionResponse
            = openAIClient.postCompletion(
            new CompletionRequest(
                command.prompt(),
                openAIProperties.model(),
                openAIProperties.temperature(),
                openAIProperties.maxToken()),
            openAIProperties.apiKey()
        );

        List<CompletionResponse.Completion> choices
            = completionResponse.choices();
        if (choices == null || choices.isEmpty()) {
            throw new LanguageProcessorException(
                "No choices are returned," +
                    " unable to return result"
            );
        }
        var completionChoice = choices.get(0);
        return new ProcessResult(completionChoice.text());
    }
}

That's it, native was working fine. 🔥

🙈 Problems and (partial) solution with the GitHub API

The jcabi-github dependency looked nice in the beginning but it was having issues with the event feed management, the endpoints were not properly setup in the code, but it was only a minor issue, I was able to overcome it by using two different operations to fetch the required data.

After trying to go native, the problems just came out, the lib is written in a way that the native compiler was unable to overcome and I was looking for quick fixes. In Quarkus you can setup hints for native compiler, but it was not clear at first glance, what should be setup.

The solution was to move to another solution and use a Quarkus extension that I did not discover at the first place.

Quarkus extensions are basically plugins for the framework itself, you need/should/would write an extension when you want to utilize the framework benefits like build-time optimizations or the native binary generation. I was happy that I found it because if you find an official Quarkus extension than you can "hope" that it will compile to native.

Not in this case 🚧

The extension is backed by a different 3rd party lib that can interact with the GitHub API, it is called Hub4j GitHub API, the change from the jcabi dependency was easy and everything was working as before, BUT not the native mode.

There are some problems with that dependency too, I was digging deeper but I was unable to figure out the real reason behind that.

Outcome

Even I planned to go native but I can not, this does not mean the tool can not be used, it definitely can be, but users would need Java on their machine to run it, but the GitHub action will contain a Java 17 so there should be no problem with the execution. 🎉