DEV Community

Cover image for 1 + N Queries for One Feed Page: Finding and Fixing the N+1 Problem in Spring Boot
Marouane En-naas
Marouane En-naas

Posted on

1 + N Queries for One Feed Page: Finding and Fixing the N+1 Problem in Spring Boot

I'm a student at Zone01 Oujda. My latest project, 01blog, is a social blogging platform built with Spring Boot and Angular: users write posts with tags, follow each other, like and comment.

Everything worked. The feed loaded, the tests passed, my peers approved it. Then I turned on SQL logging and saw this for a single page of 10 posts:

select p.* from post p order by p.created_at desc limit 10
select u.* from users u where u.id=?
select u.* from users u where u.id=?
select u.* from users u where u.id=?
... (x10)
select t.* from tag t join post_tag pt on ... where pt.post_id=?
select t.* from tag t join post_tag pt on ... where pt.post_id=?
... (x10)
Enter fullscreen mode Exit fullscreen mode

21 queries to render one page. This is the famous N+1 problem, and it's the most common performance bug in Spring Data JPA apps. It never fails a test, and it gets worse as your data grows.

This post covers how to see it, how to fix it properly, the trap I fell into while fixing it, and how to make sure it never comes back.

The setup

Simplified, my entities look like this:

@Entity
public class Post {
    @Id @GeneratedValue
    private Long id;

    private String title;
    private Instant createdAt;

    @ManyToOne(fetch = FetchType.LAZY)
    private User author;

    @ManyToMany
    private Set<Tag> tags = new HashSet<>();

    // getters...
}
Enter fullscreen mode Exit fullscreen mode

And the feed endpoint:

@Transactional(readOnly = true)
public Page<PostDto> feed(Pageable pageable) {
    return postRepository.findAll(pageable)
                         .map(post -> new PostDto(
                                 post.getId(),
                                 post.getTitle(),
                                 post.getAuthor().getUsername(),       // <- touches the author
                                 post.getTags().stream()               // <- touches the tags
                                     .map(Tag::getName).toList()));
    }
Enter fullscreen mode Exit fullscreen mode

It looks innocent. The bug is invisible in the code.

Why it happens

findAll(pageable) loads the 10 posts: 1 query. But author and tags are lazy, so Hibernate only loads them when the code touches them. The .map(...) touches them once per post, so Hibernate fires a separate query each time.

Before: one query for the posts, then two more queries for every post

That's 1 + N + N queries, where N is the page size. Double the page size, double the queries:

Query count vs page size: 11, 21, 41, 101, 201 before the fix, constant after

With 10 posts you might not notice. With 100 posts per page, or a slow database connection, it becomes your bottleneck.

Step 1: Make the problem visible

You can't fix what you can't see. Add this to application.properties:

# Log every SQL statement
logging.level.org.hibernate.SQL=DEBUG

# Let Hibernate count statements (we'll use this in a test later)
spring.jpa.properties.hibernate.generate_statistics=true
Enter fullscreen mode Exit fullscreen mode

Now call your endpoint and count the select lines. If the number changes with the page size, you have an N+1.

Add your own screenshot here: your real console output of the "before" run, with the queries highlighted. It's the most convincing proof in the post.

Step 2: Fix the @ManyToOne side with an entity graph

The author is a single-valued relation, so the natural fix is to load it in the same query with a join. Spring Data JPA makes this a one-liner:

public interface PostRepository extends JpaRepository<Post, Long> {

    @EntityGraph(attributePaths = "author")
    Page<Post> findAllByOrderByCreatedAtDesc(Pageable pageable);
}
Enter fullscreen mode Exit fullscreen mode

This produces one query that selects the posts and their authors. The 10 author queries are gone.

Step 3: The trap, JOIN FETCH on the collection

The tags are still lazy, so my first idea was to fetch them in the same query too:

@Query("select p from Post p join fetch p.author join fetch p.tags")
Page<Post> findFeed(Pageable pageable);
Enter fullscreen mode Exit fullscreen mode

The number of queries dropped to 1. I was happy for about five minutes, until I saw this in the logs:

HHH90003004: firstResult/maxResults specified with collection fetch; applying in memory
Enter fullscreen mode Exit fullscreen mode

The trap: fetching a collection with pagination makes Hibernate load everything and paginate in memory

What happened: joining a collection multiplies rows. A post with 3 tags becomes 3 rows, so SQL's LIMIT 10 would cut posts in half. Hibernate refuses to use LIMIT in that case. It loads every matching row from the table into memory and paginates there.

With 13 posts in my database, I'd never have noticed. With 100,000 posts, the "optimization" would have crashed the server. My fix was worse than the bug.

The rule I took away:

Fetch *ToOne relations with a join. Never combine collection fetching with pagination.

Step 4: Batch-load the collections instead

For collections, tell Hibernate to load them for many parents at once using @BatchSize:

@BatchSize(size = 50)
@ManyToMany
private Set<Tag> tags = new HashSet<>();
Enter fullscreen mode Exit fullscreen mode

Now, when the mapper touches the first post's tags, Hibernate loads the tags of up to 50 posts in a single query:

select ... from post_tag pt join tag t on ... where pt.post_id in (?, ?, ?, ?, ?, ?, ?, ?, ?, ?)
Enter fullscreen mode Exit fullscreen mode

1 + 1 instead of 1 + N. You can also enable this globally for every lazy collection and proxy:

spring.jpa.properties.hibernate.default_batch_fetch_size=50
Enter fullscreen mode Exit fullscreen mode

Step 5: Don't load a collection just to count it

A feed usually shows a like count. The lazy way is post.getLikes().size(), which loads every like of every post just to count them. Ask the database to count instead, with a projection:

public interface LikeCount {
    Long getPostId();
    Long getCount();
}

public interface PostLikeRepository extends JpaRepository<PostLike, Long> {

    @Query("""
           select l.post.id as postId, count(l) as count
           from PostLike l
           where l.post.id in :postIds
           group by l.post.id
           """)
    List<LikeCount> countByPostIds(Collection<Long> postIds);
}
Enter fullscreen mode Exit fullscreen mode

Then collect the counts into a map and use it while building the DTOs:

Map<Long, Long> likes = postLikeRepository.countByPostIds(ids).stream()
        .collect(Collectors.toMap(LikeCount::getPostId, LikeCount::getCount));
Enter fullscreen mode Exit fullscreen mode

The result

After: entity graph, batch fetching and a count projection give a constant number of queries

Queries for one page of 10 posts
Naive findAll + mapping 21
Entity graph for author + @BatchSize for tags 3 (posts+authors, tags, like counts)

The count no longer grows with the page size.

Add your own numbers here: the real query count and, if you measured it, the response time before and after on your machine. Use your own measurements rather than mine.

Step 6: Make sure it never comes back

An N+1 is a silent bug: everything still works, just slower. So write a test that fails if someone reintroduces it. Hibernate's statistics make this easy:

@DataJpaTest
class FeedQueryCountTest {

    @Autowired EntityManager em;
    @Autowired PostRepository postRepository;

    @Test
    void feedUsesAConstantNumberOfQueries() {
        // ...insert 20 posts, each with an author and a few tags...
        em.flush();
        em.clear(); // start from an empty persistence context

        Statistics stats = em.getEntityManagerFactory()
                             .unwrap(SessionFactory.class)
                             .getStatistics();
        stats.setStatisticsEnabled(true);
        stats.clear();

        var page = postRepository.findAllByOrderByCreatedAtDesc(PageRequest.of(0, 10));
        page.forEach(p -> {
            p.getAuthor().getUsername();   // touch the lazy relations like the mapper does
            p.getTags().size();
        });

        assertThat(stats.getPrepareStatementCount()).isLessThanOrEqualTo(4);
    }
}
Enter fullscreen mode Exit fullscreen mode

If anyone adds a new lazy relation to the mapper and forgets to fetch it, this test turns red instead of your production database.

A note on open-in-view

Spring Boot enables spring.jpa.open-in-view=true by default. It keeps the database session open while the controller renders the response, which hides lazy loading problems like this one: your JSON serializer can trigger queries from the view layer without any error.

I switched it off:

spring.jpa.open-in-view=false
Enter fullscreen mode Exit fullscreen mode

Now a forgotten lazy relation throws a LazyInitializationException during development instead of quietly running extra queries in production. It's annoying at first, and it's exactly the feedback you want.

What I took away

  1. Turn on SQL logging in development. N+1 is invisible unless you look at the queries.
  2. Join-fetch *ToOne relations (@EntityGraph is the cleanest way).
  3. Never join-fetch a collection with pagination. Use @BatchSize or default_batch_fetch_size.
  4. Let the database count. Don't load a collection to call .size().
  5. Test the query count, so the fix is guaranteed to stay fixed.

What's next

test with a much bigger dataset.

The full project is on GitHub: [link]

I'm still learning, so if you spot something I could do better, tell me in the comments. Have you ever shipped an N+1 without knowing?

Top comments (0)