DEV Community: Roberto Conte Rosito

Spring Boot and MongoDB Aggregation Issues - What a pain!

Roberto Conte Rosito — Sat, 18 May 2024 00:00:00 +0000

Welcome back to my dev notes!

I guys, too much time is passed since my last post, but I'm back with a new one. Today I want to talk about a problem I faced with Spring Boot and MongoDB aggregations.

This problem is really annoying, and I spent a lot of time trying to solve it. The problem is that sometimes the aggregation doesn't work as expected, and you don't know why. The real problem is that I've created a suite of tests that cover all the possible cases, but sometimes the aggregation still doesn't work as expected.

I was in trouble but after hours of debugging I remembered that I faced the same problem in the past and it took me a lot of time to solve it too (which is way I've turned on my blog to write about it).
But let's stop talking, and get to the point!

Suppose you have something like that in your spring boot application:


var aggregation = newAggregation(
  match(Criteria.where("field").is("value")),
  group("field").count().as("count"),
  project("count").and("field").previousOperation()
);
var result = mongoTemplate.aggregate(aggregation, "collection", YourClass.class).getMappedResults();

Sometimes, and I still don't know why, this aggregation doesn't work as expected. The problem is that the result list is empty. You can try to change the aggregation in many ways, and I can guarantee that I've tried every kind of operation, but the problem is still there, waiting for you to solve it.

You can try to get the result of aggregation in debug mode, copy it in to your MongoDB Compass instance, try to execute it and guess what? It works! But in your application no, it doesn't work.

The solution is quite simple, but it took me a lot of time to find it. The problem is that the mongoTemplate is not able to work with the name of the collection as a string (I suppose it's a bug but I have to investigate, for now it's my reponsability to share what I know with you).

This line of code is wrong (or to be more precise, sometimes is wrong):

var result = mongoTemplate.aggregate(aggregation, "collection", YourClass.class).getMappedResults();

If you want to be sure that everything is executed as mongodb do, you have to pass your class instance instead of the collection name, is my habit to do thinks like that:

var collectionName = mongoTemaplte.getCollectionName(YourClass.class);

Wrong!

Do it like that:

var result = mongoTemplate.aggregate(aggregation, YourClass.class, YourClass.class).getMappedResults();

This signature is similar to the previous one, but the difference is that you pass the class instance instead of the collection name. This way the mongoTemplate is able to work with the aggregation as expected.

Believe me, I had nightmares with this problem, but now I'm happy to share the solution with you.
If you know more about this problem, please let me know, I'm really interested in understanding why this happens.

For now, as always, happy coding!

Setup Google Cloud Storage Service Account

Roberto Conte Rosito — Tue, 08 Aug 2023 00:00:00 +0000

I was in trouble with this for a while so I decided to write down this guide to help me in the future and maybe it will help you too.

The goal is quite simple: I’ve to create a google service account needed by my pipeline to publish docker images to the Google Cloud Container Registry.

This account must have the following permissions:

App Engine Admin
Cloud API Gateway Service Agent
Cloud Deploy Admin
Cloud Infrastructure Manager Admin
Container Registry Service Agent
Environment and Storage Object Administrator

Sorry guys but without these permissions I was not able to make it work! Do you know which one is not needed? Please let me know!

Setup Spring Boot with JPA and MySQL

Roberto Conte Rosito — Mon, 20 Mar 2023 00:00:00 +0000

Welcome back to my dev notes!

Few things that I’ve to remember next time I need to setup a Spring Boot project with JPA and MySQL:

Add the following dependencies to your pom.xml:

<dependency>
 <groupId>org.springframework.boot</groupId>
 <artifactId>spring-boot-starter-data-jpa</artifactId>
</dependency>
<dependency>
 <groupId>mysql</groupId>
 <artifactId>mysql-connector-java</artifactId>
</dependency>

Add the following properties to your application.properties:

spring.datasource.url=jdbc:mysql://localhost:3306/db_name
spring.datasource.username=root
spring.datasource.password=root
spring.jpa.hibernate.ddl-auto=update
spring.jpa.show-sql=true

Create your model using @Entity and @Id annotations:

@Entity
public class User {
 @Id
 @GeneratedValue(strategy = GenerationType.IDENTITY)
 private Long id;
 private String name;
 private String email;
 private String password;
 // getters and setters
}

Create your repository extending JpaRepository adding @Repository annotation:

@Repository
public interface UserRepository extends JpaRepository<User, Long> {}

Now you are ready to @Autowired your repository and use it in your services/controllers.

That’s all for today, see you next time!

Setup HTTPS with Nginx and Docker

Roberto Conte Rosito — Tue, 28 Feb 2023 00:00:00 +0000

Welcome back to my dev notes!

Today I was in trouble for a while because I wanted to setup HTTPS with Nginx and Docker and I completely forgot how to do it. So I decided to write down this guide to help me in the future and maybe it will help you too.

I have a Docker container running a PHP application served by Nginx and I wanted to setup HTTPS with a self-signed certificate because I need to test some OAuth flows.

Follow these steps to setup HTTPS with Nginx and Docker:

Create a self-signed certificate

openssl x509 -outform pem -in localhost.crt -out localhost.pem

Edit nginx configuration

Please take attention to the ssl_certificate and ssl_certificate_key files paths.

server {
 listen 443 ssl;
 server_name localhost;
 ssl_certificate /etc/nginx/ssl/localhost.crt;
 ssl_certificate_key /etc/nginx/ssl/localhost.key;
 ...
}

Restart docker

If you are a perfectionist like me you can also add the certificate to your system keychain to avoid browser warnings. Using Mac OS you can do it like this:

sudo security add-trusted-cert -d -r trustRoot -k /Library/Keychains/System.keychain localhost.crt

Enjoy!

How to use useRef with React Hook Form

Roberto Conte Rosito — Mon, 30 Jan 2023 00:00:00 +0000

Welcome back to my dev notes!

Today I was in trouble for a while because I wanted to use a ref inside a React-Hook-Form form with an input of type file and I couldn’t do it.

The problem was quite simple, suppose you have a form like this:

const { register } = useFormContext();
const fileInputRef = useRef(null);
const handleClick = useCallback(() => {
 fileInputRef.current.click();
}, [fileInputRef]);
return (
 <>
   <input type="file" ref={fileInputRef} {...register("file")} />
   <button onClick={handleClick}>Upload</button>
 </>
);

And guess what? It didn’t work. The fileInputRef was null inside the handleClick function and I couldn’t understand why. After googling for a while I found the solution: you have to use the register method in a different way to make it work:

const { register } = useFormContext();
const { ref, ...inputProps } = register("file");
const fileInputRef = useRef(null);
const handleClick = useCallback(() => {
 fileInputRef.current.click();
}, [fileInputRef]);

return (
 <>
   <input
     type="file"
     ref={(e) => {
       fileInputRef.current = e;
       ref(e);
     }}
     {...inputProps}
   />
   <button onClick={handleClick}>Upload</button>
 </>
);

I hope this will help you if you are in trouble with refs and React Hook Form.

Notes : this is a dev note, not a tutorial. If you want to learn more about React Hook Form, check out the official documentation.

How to use setValue with React Hook Form

Roberto Conte Rosito — Wed, 18 Jan 2023 00:00:00 +0000

My dev notes about how to set value with React Hook Form. Following the official documentationyou can set a value with React Hook Form using the setValue method to change it programmatically like this:

const { setValue } = useForm();
...
setValue("firstName", "bill");

But this is wrong!

You have to use the setValue method to change the value of a field programmatically, but only using the useFormContext hook instead of the useForm hook:

const { setValue } = useFormContext();
...
// Somewhere inside the form
setValue("firstName", "bill");

Remember: the first one is used during initialization of the form, the second one is used inside the form.

Publish superset dashboard on React/NextJS apps

Roberto Conte Rosito — Mon, 02 Jan 2023 00:00:00 +0000

Superset is a great tool to create and publish different kind of dashboards. In this post I will show you how to publish and embed a superset dashboard on a React/NextJS app.

Configure Superset

Superset expose everything required to publish data but you need to configure few things before start (this guide assumes you have already installed superset).

Open config.py and add these changes:

search ENABLE_CORS and change it to True;
replace CORS_OPTIONS (should be located after ENABLE_CORS) with this:

CORS_OPTIONS = {
 'supports_credentials': True,
 'allow_headers': ['*'],
 'resources':['*'],
 'origins': ['*']
}

The origins property should be configured based on your requirements, in development mode you can use * to allow all origins but in production you should specify the domain of your app.

search EMBEDDED_SUPERSET and change it to True;
search PUBLIC_ROLE_LIKE and change it to Public;
search GUEST_TOKEN_JWT_SECRET and change it to random string to enforce security.

Restart your superset instance to activate the changes.

Configure public role

Public role needs to be updated with the following permissions:

can read on Chart
can read on Dataset
can read on Dashboard
can read on Database
can write on DashboardFilterStateRestApi
can read on DashboardfilterStateRestApi
can dashboard on Superset
can explore json on Superset
can grant guest token on SecurityRestApi
all database access on all_database_access

Without these permissions you will not be able to publish your dashboard ( you will get a 403 error when you will try to access the dashboard).

Configure guest account

Now you are ready to configure guest account that will be used to access dashboard without username and password. Go in Settings/List Users and create a new user, the most important part is the Roles field, you have to select Public and Gamma roles.

Setup your React/NextJS app

Superset comes out with a default NPM plugin, @superset-ui/embedded-sdk, that helps in the process of publication. On top of this I’ve created another wrapper that uses React component to display data and take care of DOM initialization.

Install the package:

npm i -S superset-dashboard-sdk

To publish a dashboard you need to create a new data provider, this is a class that will be used to fetch data from superset. The data provider is a class that extends DefaultDataProvider and it’s used to fetch data from superset. The DefaultDataProvider class is a wrapper of SupersetClient class that comes out with @superset-ui/embedded-sdk package.

// dataProvider.js
import { DefaultDataProvider } from 'superset-dashboard-sdk';
const dp = new DefaultDataProvider("http://localhost:8088", {
 username: "guest"
 password: "guest"
});
export default dp;

Then you can use the data provider to publish your dashboard:

// index.js
import { Dashboard } from "superset-dashboard-sdk";
import dataProvider from "./dataProvider";
const App = () => <Dashboard dataProvider={dp} id="<id of dashboard>" />;

To obtain a valid dashboard id you can open superset, navigate to dashboard and click on “…” (three dots) on the top right corner. Then click on Enable Embedding to generate and see the uuid associated with selected dashboard. Also remember to add Guest account to the list of owners of the dashboard to avoid problems with permissions.

Hope this helps you too, feel free to write a comment if you have any question or suggestion.

Deploy a .net app on a local kind cluster

Roberto Conte Rosito — Tue, 20 Dec 2022 00:00:00 +0000

This guide will help you throught the process of deploying a .Net app on a local kind cluster. I was just curious to see how it works and I decided to write an article about it (after studying the correct way to do it).

Prerequisites

This guide assumes you have alreay installed:

You can follow the guide Deploy a local kind cluster to install all the prerequisites or you can just install kind and ingress-nginx fastes way possible by running the following commands:

kind create cluster --config kind/config.yaml
kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/kind/deploy.yaml

And you are ready to go!

Create a .Net app

I’ve created a github repo with a simple .Net app that will be deployed on the kind cluster. You can find it here. The app is a simple web api that returns a string or interact with files (to test my understanding of volumes).

You can try it locally by running the following commands:

git clone https://github.com/RoBYCoNTe/hello-world-k8s-net.git
cd hello-world-k8s-net
dotnet restore
dotnet run

The app will be available on port 5000 and you can test it by running the following commands:

curl http://localhost:5000/v1
curl http://localhost:5000/v1/files/append?line=Hello

Create a Dockerfile

Using kubernetes we can deploy a containerized application. To do that we need to create a Dockerfile that will be used to build the image that will be deployed on the cluster.

Inside the repo you can find a Dockerfile already prepared. You can find it here.

Before building the image you have to login into your docker hub account. To do that run the following command:

docker login

Then build the image by running the following command:

docker build -t <your-docker-hub-username>/hello-world-k8s-net:latest .

Push the image to Docker Hub

To push the image to Docker Hub run the following command:

docker push <your-docker-hub-username>/hello-world-k8s-net:latest

Deployment

Now that we have the image ready we can create a deployment that will deploy the image on the cluster. Before creating the deployment you have to instruct your cluster to pull the image from Docker Hub. To do that run the following command:

kubectl create secret docker-registry dockerhub --docker-server=https://index.docker.io/v1/ --docker-username=<your-docker-hub-username> --docker-password=<your-docker-hub-password> --docker-email=<your-docker-hub-email>

With previous command we have created a secret that will be used by the deployment to pull the image from Docker Hub (you cannot pull an image from Docker Hub without a secret). For more information about how to configure a secret to pull an image from Docker Hub you can read this article.

The deployment is defined in the file deployment.yaml. You can find it in the repo.

Before creating the deployment we have to create PVCs that will be used by the deployment to store data. To do that run the following commands:

kubectl apply -f k8s/pv.yaml
kubectl apply -f k8s/pvc.yaml

Without the PVCs the deployment will fail because it will not be able to store data.

Now we can create the deployment by running the following command:

kubectl apply -f k8s/deployment.yaml

Remember to change the image name in the deployment file with your own image name.

Now we can create service and ingress to expose the app to the outside world. To do that run the following commands:

kubectl apply -f k8s/service.yaml
kubectl apply -f k8s/ingress.yaml

Remember to change the host in the ingress file with your own host and to add the host to your /etc/hosts file (if you are using linux) or to your hosts file (if you are using windows).

Test the app

To test the app run the following commands:

curl -H "Host: hello-world-k8s-net.local" http://localhost/v1

You should see the classic “Hello World!” message.

I was just curious to see how volumes work on a local kind cluster. To test it run the following commands:

curl -H "Host: hello-world-k8s-net.local" http://localhost/v1/files/append?line=Hello

If you execute the previous command multiple time you will see that the file is updated with the new line (it is returned by the api). Furthermore if you delete the pod or restart the cluster the file will still be there.

For example, if we redeploy the app by running the following command:

kubectl rollout restart deployment hello-world-k8s-net

Executing the curl command again will return the same result!

Conclusion

This guide was just a way to test how volumes work on a local kind cluster. I’ve learned that volumes are not deleted when you delete a pod or when you restart the cluster. Furthermore I’ve learned that you can use a secret to pull an image from Docker Hub. I’ve to says that first tries were not successful because I was using a wrong secret, missing to install ingress-nginx, missing to create PVC and so on. I’ve learned that you have to read the error messages and try to understand what is going on. Most of the time the error messages are pretty clear and you can understand what is going on, if not you can always ask for help on the internet.

I hope you enjoyed this guide and I hope you learned something new. If you have any question or suggestion please leave a comment below.

Deploy a local kind cluster

Roberto Conte Rosito — Mon, 19 Dec 2022 00:00:00 +0000

This guide will help you throught the process of deploying a local kind cluster.

Kind is a tool for running local Kubernetes clusters using Docker container “nodes”. Kind was primarily designed for testing Kubernetes itself, but may be used for local development or CI.

Prerequisites

This guide assumes you have alreay installed:

Create a cluster

Create a kind config file called kind-config.yaml with the following content:

kind: Cluster
apiVersion: kind.x-k8s.io/v1alpha4
nodes:
 - role: control-plane
 kubeadmConfigPatches:
 - |
 kind: InitConfiguration
 nodeRegistration:
 kubeletExtraArgs:
 node-labels: "ingress-ready=true"
 extraPortMappings:
 - containerPort: 80
 hostPort: 80
 protocol: TCP
 - containerPort: 443
 hostPort: 443
 protocol: TCP

Then create the cluster using the following command:

kind create cluster --config kind-config.yaml

The configured cluster is ready to accept requests on port 80 and 443.

Install ingress controller

By default k8s does not have an ingress controller installed. To install the ingress controller run the following command:

kubectl apply -f https://raw.githubusercontent.com/kubernetes/ingress-nginx/master/deploy/static/provider/kind/deploy.yaml

This will install the nginx ingress controller in the cluster (the most used ingress controller right now, but you can install many other, the only thing you have to do is to change the ingressClassName in your ingress resources).

Test the cluster

To test the cluster you can create a simple deployment and a service:

kubectl run hello \
 --expose \
 --image nginxdemos/hello:plain-text \
 --port 80

Then create a file called ingress.yaml with the following content:

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
 name: hello
spec:
 rules:
 - host: hello.robertoconterosito.it
 http:
 paths:
 - pathType: ImplementationSpecific
 backend:
 service:
 name: hello
 port:
 number: 80

Then apply the ingress resource:

kubectl apply -f ingress.yaml

Now you can test the ingress resource using the following command:

curl -H "Host: hello.robertoconterosito.it" localhost

Bootstrap-Italia Design Kit with NextJS app

Roberto Conte Rosito — Sat, 05 Nov 2022 00:00:00 +0000

This guide will help you throught the process of configuring bootstrap-italia in your NextJS app

During this process you will probably face errors related to unrecognized static assets (csv, svg, fonts etc.) that can be solved quite simply in NextJS app using specific webpack config and NextJS components.

Install Bootstrap-Italia

First of all, install bootstrap-italia in your NextJS app:

npm i -S bootstrap-italia

Configure Webpack

Now you have to install a plugin, copy-webpack-plugin, required to move static assets from the node_modules folder (boostrap-italia) in to the project build folder using webpack:

npm i --save-dev copy-webpack-plugin

This will install a new plugin that we have to use inside our next.config.js:

const CopyPlugin = require("copy-webpack-plugin");
const path = require("path");
// Prepare from/to
const from = "node_modules/bootstrap-italia/dist/";
const to = path.join(__dirname, "./public/bootstrap-italia/dist");
const nextConfig = {
 webpack: (config, { buildId, dev, isServer, defaultLoaders, webpack }) => {
 config.plugins.push(
 new CopyPlugin({
 patterns: [{ from, to }],
 })
 );
 return config;
 },
};
module.exports = nextConfig;

Next time you run your app you will see a new folder, inside public, called bootstrap-italia with all assets required to work with NextJS app.

You can choose to copy these files statically executing a Copy/Paste operation but I prefer this solution to automatically include updates when necessary (when the bootstrap-italia package will be updated).

Configure _app.js

Import basic CSS globally in _app.js adding this code on top of the file:

import "bootstrap-italia/dist/css/bootstrap-italia.min.css";

const App = ({ Component, pageProps }) => {
 return <Component {...pageProps} />;
};
export default App;

Configure Script TAG

Supposing you have a common set of elements necessary to display your page and there exists a file called Layout where all the UX logic starts, you have to add a link to the javascript bundle required by bootstrap-italia to work correctly with you project, like in the example below:

import Header from "./Header";
import React from "react";
import Script from "next/script";
const Layout = ({ children }) => (
 <div>
 <Header />
 {children}
 <Script src="bootstrap-italia/dist/js/bootstrap-italia.bundle.min.js" />
 </div>
);
export default Layout;

Now you are ready to see a running NextJS app with bootstrap-italia UX.