DEV Community: Fred Feng

EasyJPA – Your Best Partner for JPA Development!

Fred Feng — Mon, 24 Mar 2025 22:24:03 +0000

It's time to say goodbye to JPA Criteria API complexity! EasyJPA makes your code sleek, simple, and powerful!

EasyJPA elegantly streamlines JPA's Criteria API with a fully Lambda-based and developer-friendly API, making dynamic queries both intuitive and efficient. It significantly reduces SQL/JPQL complexity, accelerates development, and improves code readability, ensuring a clean and concise query experience.

With comprehensive support for complex SQL queries, EasyJPA enables seamless execution of multi-table joins (INNER, LEFT, RIGHT, CROSS JOINs), subqueries, aggregations, computed columns, and filtering operations. Its fluent API with full Lambda expression support allows developers to construct queries programmatically, eliminating the need for raw SQL while retaining maximum flexibility.

EasyJPA GitHub repository: paganini2008/easyjpa.

Features

Dynamic Queries
Computed and Function-Based Columns
Grouping and Filtering
Sorting
List Queries and Pagination
Complex Subqueries
Inner Join, Left Join, Right Join, and Cross Join
Update and Delete Operations
Lambda Expression Support
Hibernate Implementation by Default

Examples

1. Query All Users

@Autowired
private UserDao userDao;

@BeforeAll
public void saveRandomUsers() {
    List.of(new User("Jack", "123456", "Jack001@jpatest.com"),
            new User("Petter", "123456", "Petter002@jpatest.com"),
            new User("Scott", "123456", "scott003@jpatest.com"),
            new User("Lee", "123456", "lee004@jpatest.com"),
            new User("Terry", "123456", "terry005@jpatest.com")
           ).forEach(user -> {
                userDao.save(user);
           });
    log.info("Total users: {}", userDao.count());
}

/**
Hibernate: 
    select
        u1_0.id,
        u1_0.email,
        u1_0.password,
        u1_0.username 
    from
        example_user u1_0
**/
@Test
public void testSelectAll() {
    userDao.query()
           .selectThis()
           .list()
           .forEach(u -> {
                log.info(u.toString());
           });
}

2. Basic Search Conditions

/**
Hibernate: 
    select
        u1_0.id,
        u1_0.email,
        u1_0.password,
        u1_0.username 
    from
        example_user u1_0 
    where
        u1_0.username=? 
        and u1_0.password=?
**/
@Test
public void testGetUserByUsernameAndPassword() {
    User user = userDao.query().filter(new FilterList()
                                          .eq(User::getUsername, "Jack")
                                          .eq(User::getPassword, "123456")
                                       ).selectThis()
                                       .one();
    log.info("Load user: {}", user);
    assertTrue(user != null);
}

/**
Hibernate: 
    select
        u1_0.username,
        u1_0.password 
    from
        example_user u1_0 
    where
        u1_0.email like ? escape '' 
    offset
        ? rows 
    fetch
        first ? rows only
**/
@ParameterizedTest
@ValueSource(strings = {"scott003", "lee004"})
public void testGetUserByEmail(String email) {
    User user = userDao.query()
                       .filter(new FilterList()
                                  .like(User::getEmail, email)
                       ).select(new ColumnList(
                                   User::getUsername, 
                                   User::getPassword)
                       ).first();
    log.info("Load user: {}", user);
    assertTrue(user != null);

}

/**
Hibernate: 
    select
        u1_0.username,
        u1_0.password 
    from
        example_user u1_0 
    where
        u1_0.email like ? escape '' 
    offset
        ? rows 
    fetch
        first ? rows only
**/
@ParameterizedTest
@ValueSource(strings = {"abc009"})
public void testGetUserNotFoundByEmail(String email) {
    User user = userDao.query()
                       .filter(new FilterList()
                               .like(User::getEmail, email)
                       ).select(new ColumnList(
                                    User::getUsername, 
                                    User::getPassword, 
                                    User::getEmail)
                       ).first();
   log.info("Load user: {}", user);
   assertTrue(user == null);
}

3. Nested Query Conditions & Sorting & Computed Columns

@Autowired
private ProductDao productDao;

/**
Hibernate: 
    select
        p1_0.name,
        p1_0.location,
        p1_0.price,
        (p1_0.price*p1_0.discount) 
    from
        example_product p1_0 
    where
        p1_0.price>=? 
        and (
            p1_0.location=? 
            or p1_0.location=?
        ) 
    order by
        4 desc
 **/
@Test
public void test2() {
    productDao.query(ProductVo.class)
              .filter(new FilterList()
                          .gte(Product::getPrice, BigDecimal.valueOf(200))
                              .and(() -> new FilterList()
                                         .eq(Product::getLocation, "Australia")
                                             .or()
                                         .eq(Product::getLocation, "Thailand")
                               )
              ).sort(JpaSort.desc(Fields.toInteger(4))
              ).select(new ColumnList(
                            Product::getName, 
                            Product::getLocation, 
                            Product::getPrice
                       ).addColumns(
                            Fields.multiply(Product::getPrice, Product::getDiscount).as("actualPrice"))
                       ).list().forEach(vo -> {
                            log.info(vo.toString());
                       });
}

4. Grouping & Aggregation & Filtering

/**
Hibernate: 
    select
        p1_0.location,
        max(p1_0.price),
        min(p1_0.price),
        avg(p1_0.price),
        count(1) 
    from
        example_product p1_0 
    group by
        p1_0.location 
    having
        avg(p1_0.price)>? 
    order by
        4 desc 
    offset
        ? rows
**/
@Test
public void test4() {
    productDao.customQuery()
              .groupBy(new FieldList(Product::getLocation))
              .having(Restrictions.gt(Fields.avg(Product::getPrice), 50d))
              .sort(JpaSort.desc(4))
              .select(new ColumnList(Product::getLocation).addColumns(
                   Fields.max(Product::getPrice).as("maxPrice"),
                   Fields.min(Product::getPrice).as("minPrice"),
                   Fields.avg(Product::getPrice).as("avgPrice"), 
                   Fields.count(1).as("amount"))
               ).setTransformer(Transformers.asBean(ProductAggregationVo.class))
               .list().forEach(vo -> {
                    log.info(vo.toString());
               });
}

5. Using Function in Columns

/**
Hibernate: 
    select
        ((max(p1_0.price)||?)||min(p1_0.price)),
        p1_0.location 
    from
        example_product p1_0 
    group by
        p1_0.location 
    offset
        ? rows
**/
@Test
public void test5() {
    productDao.customQuery()
              .groupBy("location")
              .select(new ColumnList().addColumns(
                          Fields.concat(Fields.concat(Fields.max("price", String.class), "/"),
                                        Fields.min("price", String.class)).as("repr")
                          ).addColumns(Product::getLocation))
              .setTransformer(Transformers.asBean(ProductAggregationVo.class))
              .list().forEach(vo -> {
                  log.info(vo.toString());
              });
}

/**
Hibernate: 
    select
        lower(p1_0.name),
        upper(p1_0.location) 
    from
        example_product p1_0 
    offset
        ? rows 
    fetch
        first ? rows only
**/
@Test
public void test6() {
    productDao.customQuery()
              .select(new ColumnList().addColumns(
                   Function.build("LOWER", String.class, Product::getName).as("name"),
                   Function.build("UPPER", String.class,Product::getLocation).as("location"))
              ).list(10).forEach(t -> {
                   log.info(t.toString());
              });
}

/**
Hibernate: 
    select
        case p1_0.location 
            when ? 
                then cast(? as varchar) 
            when ? 
                then cast(? as varchar) 
            when ? 
                then cast(? as varchar) 
            when ? 
                then cast(? as varchar) 
            when ? 
                then cast(? as varchar) 
            when ? 
                then cast(? as varchar) 
            when ? 
                then cast(? as varchar) 
            when ? 
                then cast(? as varchar) 
            else cast(? as varchar) 
    end,
    p1_0.location 
from
    example_product p1_0
**/
@Test
public void test7() {
     IfExpression<String, String> ifExpression = new IfExpression<String, String>(Product::getLocation)
                .when("Indonesia", "Asia")
                .when("Japan", "Asia")
                .when("China", "Asia")
                .when("Singapore", "Asia")
                .when("Vietnam", "Asia")
                .when("Thailand", "Asia")
                .when("Australia", "Oceania")
                .when("New Zealand", "Oceania")
                .otherwise("Other");
     productDao.customQuery().select(new ColumnList()
                                        .addColumns(ifExpression.as("area"))
                                        .addColumns(Product::getLocation)
                                    ).list().forEach(t -> {
                                         log.info(t.toString());
                                    });
}

6. Inner Join & Pagination

@Autowired
private OrderDao orderDao;

@Autowired
private OrderProductDao orderProductDao;

/**
Hibernate: 
    select
        o1_0.id,
        o1_0.order_date,
        o1_0.total_price,
        o1_0.user_id,
        u1_0.id,
        u1_0.email,
        u1_0.password,
        u1_0.username 
    from
        example_order o1_0 
    join
        example_user u1_0 
            on u1_0.id=o1_0.user_id 
    where
        u1_0.username=? 
    order by
        o1_0.order_date desc
**/
@ParameterizedTest
@ValueSource(strings = {"Petter", "Jack"})
public void test3(String username) {
    orderDao.customQuery().join(Order::getUser, "u", null)
                          .filter(Restrictions.eq(User::getUsername, username))
                          .sort(JpaSort.desc(Order::getOrderDate))
                          .select(new ColumnList()
                                  .addFields(Fields.root())
                                  .addTableAlias("u")
                          ).list().forEach(t -> {
                              Order order = (Order) t.get(0);
                              User user = (User) t.get(1);
                              log.info("Order: " + order + ", User: " + user);
                          });
}

/**
Hibernate: 
    select
        count(1) 
    from
        example_order o1_0 
    join
        example_user u1_0 
            on u1_0.id=o1_0.user_id 
    where
        o1_0.order_date between ? and ? 
    group by
        o1_0.order_date,
        u1_0.username 
    having
        avg(o1_0.total_price)>?
Hibernate: 
    select
        u1_0.username,
        o1_0.order_date,
        avg(o1_0.total_price) 
    from
        example_order o1_0 
    join
        example_user u1_0 
            on u1_0.id=o1_0.user_id 
    where
        o1_0.order_date between ? and ? 
    group by
        o1_0.order_date,
        u1_0.username 
    having
        avg(o1_0.total_price)>? 
    order by
        o1_0.order_date desc 
    offset
        ? rows 
    fetch
        first ? rows only
**/
@Test
public void test4() {
    orderDao.customPage().join(Order::getUser, "u", null)
                         .filter(Restrictions.between(Order::getOrderDate,
                                      LocalDate.of(2025, 2, 1).atStartOfDay(),
                                      LocalDate.of(2025, 2, 28).atStartOfDay())
                          ).groupBy(new FieldList()
                                     .addFields(Order::getOrderDate)
                                     .addFields(User::getUsername)
                          ).having(Restrictions.gt(Fields.avg(Order::getTotalPrice), 20000D))
                         .sort(JpaSort.desc(Order::getOrderDate))
                         .select(new ColumnList()
                                     .addColumns(User::getUsername)
                                     .addColumns(Order::getOrderDate)
                                     .addFields(Fields.avg(Order::getTotalPrice))
                          ).setTransformer(Transformers.asCaseInsensitiveMap())
                         .paginate(PageRequest.of(5))
                         .forEachPage(eachPage -> {
                             log.info(String.format(
              "====================== PageNumber/TotalPage: %s/%s  Total Records: %s =====================",
                             eachPage.getPageNumber(), eachPage.getTotalPages(),
                             eachPage.getTotalRecords()));
                             eachPage.getContent().forEach(vo -> {
                                 log.info(vo.toString());
                             });
                          });
}

7. Left Join & Pagination

/**
Hibernate: 
    select
        count(1) 
    from
        example_order o1_0 
    left join
        example_order_product op1_0 
            on o1_0.id=op1_0.order_id
Hibernate: 
    select
        o1_0.id,
        o1_0.order_date,
        o1_0.total_price,
        o1_0.user_id,
        p1_0.id,
        p1_0.discount,
        p1_0.location,
        p1_0.name,
        p1_0.price,
        p1_0.produce_date 
    from
        example_order o1_0 
    left join
        example_order_product op1_0 
            on o1_0.id=op1_0.order_id 
    left join
        example_product p1_0 
            on p1_0.id=op1_0.product_id 
    order by
        o1_0.order_date desc 
    offset
        ? rows 
    fetch
        first ? rows only
**/
@Test
public void test5() {
    orderDao.customPage()
            .leftJoin(Order::getOrderProducts, "op", null)
            .leftJoin(OrderProduct::getProduct, "p", null)
            .sort(JpaSort.desc(Order::getOrderDate))
            .select(new ColumnList()
                    .addFields(Fields.root())
                    .addTableAlias("p")
            ).setTransformer(Transformers.asMap())
            .paginate(PageRequest.of(10))
            .forEachPage(eachPage -> {
                 log.info(String.format(
             "====================== PageNumber/TotalPage: %s/%s  Total Records: %s =====================",
                 eachPage.getPageNumber(), eachPage.getTotalPages(),
                 eachPage.getTotalRecords()));
                 eachPage.getContent().forEach(vo -> {
                     log.info(vo.toString());
                 });
             });
}

8. Right Join & Pagination

/**
Hibernate: 
    select
        count(1) 
    from
        example_order o1_0 
    right join
        example_order_product op1_0 
            on o1_0.id=op1_0.order_id 
    right join
        example_product p1_0 
            on p1_0.id=op1_0.product_id
Hibernate: 
    select
        o1_0.id,
        o1_0.total_price,
        o1_0.order_date,
        op1_0.amount,
        p1_0.name,
        p1_0.location 
    from
        example_order o1_0 
    right join
        example_order_product op1_0 
            on o1_0.id=op1_0.order_id 
    right join
        example_product p1_0 
            on p1_0.id=op1_0.product_id 
    order by
        o1_0.order_date desc,
        op1_0.amount desc 
    offset
        ? rows 
    fetch
        first ? rows only
**/
@Test
public void test6() {
    orderDao.customPage()
            .rightJoin(Order::getOrderProducts, "op", null)
            .rightJoin(OrderProduct::getProduct, "p", null)
            .sort(JpaSort.desc(Order::getOrderDate), 
                  JpaSort.desc(OrderProduct::getAmount)
             ).select(new ColumnList()
                  .addColumns(Order::getId, 
                              Order::getTotalPrice, 
                              Order::getOrderDate)
                  .addColumns(OrderProduct::getAmount)
                  .addColumns(Product::getName, 
                              Product::getLocation)
             ).setTransformer(Transformers.asMap())
              .paginate(PageRequest.of(10))
              .forEachPage(eachPage -> {
                   log.info(String.format(
            "====================== PageNumber/TotalPage: %s/%s  Total Records: %s ======================",
                   eachPage.getPageNumber(), 
                   eachPage.getTotalPages(),
                   eachPage.getTotalRecords()));
                   eachPage.getContent().forEach(vo -> {
                       log.info(vo.toString());
                   });
             });
}

9. Cross Join & Pagination

/**
Hibernate: 
    select
        count(1) 
    from
        example_product p1_0,
        example_stock s1_0 
    where
        s1_0.product_id=p1_0.id
Hibernate: 
    select
        p1_0.id,
        p1_0.name,
        s1_0.amount 
    from
        example_product p1_0,
        example_stock s1_0 
    where
        s1_0.product_id=p1_0.id 
    offset
        ? rows 
    fetch
        first ? rows only
**/
@Test
public void test8() {
    productDao.customPage()
              .crossJoin(Stock.class, "a")
              .filter(new FilterList()
                      .eq(Stock::getProductId, Product::getId)
              ).select(new ColumnList()
                       .addColumns(Product::getId, 
                                   Product::getName
                       ).addColumns(Stock::getAmount)
              ).setTransformer(Transformers.asBean(ProductStockVo.class))
               .paginate(PageRequest.of(10))
               .forEachPage(eachPage -> {
                    log.info(String.format(
              "====================== PageNumber/TotalPage: %s/%s  Total Records: %s =====================",
                    eachPage.getPageNumber(), 
                    eachPage.getTotalPages(),
                    eachPage.getTotalRecords()));
                    eachPage.getContent().forEach(vo -> {
                        log.info(vo.toString());
                    });
              });
}

10. Subquery & Join

/**
Hibernate: 
    select
        distinct o1_0.user_id 
    from
        example_order o1_0 
    where
        exists(select
            1 
        from
            example_user u2_0 
        where
            u2_0.id=o1_0.user_id)
**/
@Test
public void test1() {
    JpaQuery<Order, Tuple> jpaQuery = orderDao.customQuery();
    JpaSubQuery<User, Long> jpaSubQuery = jpaQuery.subQuery(User.class, "u", Long.class)
                                                  .filter(Restrictions.eq(User::getId, Order::getUser))
                                                  .select(Fields.toLong(1L));
    jpaQuery.filter(Restrictions.exists(jpaSubQuery))
            .distinct()
            .select(new ColumnList(Order::getUser))
            .list().forEach(m -> {
                 log.info(m.toString());
            });
}

/**
Hibernate: 
    select
        op1_0.order_id,
        op1_0.product_id,
        op1_0.amount,
        p1_0.name,
        u1_0.username 
    from
        example_order_product op1_0 
    left join
        example_product p1_0 
            on p1_0.id=op1_0.product_id 
    join
        example_order o1_0 
            on o1_0.id=op1_0.order_id 
    join
        example_user u1_0 
            on u1_0.id=o1_0.user_id 
    where
        exists(select
            p3_0.id 
        from
            example_product p3_0 
        where
            p3_0.id=op1_0.product_id 
            and p3_0.name=?) 
    offset
        ? rows 
    fetch
        first ? rows only
**/
@ParameterizedTest
@ValueSource(strings = {"Microwave oven", "Coffee maker"})
public void test2(String itemName) {
     JpaQuery<OrderProduct, Tuple> jpaQuery = orderProductDao.customQuery();
     JpaSubQuery<Product, Long> jpaSubQuery = jpaQuery.subQuery(Product.class, "p", Long.class)
                                              .filter(new FilterList()
                                              .eq(Product::getId, OrderProduct::getProduct)
                                                  .and()
                                              .eq(Product::getName, itemName)
                                               ).select(Product::getId);
     jpaQuery.leftJoin(OrderProduct::getProduct, "p", null)
             .join(Order.class, "o", null)
             .join(User.class, "u", null)
             .filter(Restrictions.exists(jpaSubQuery))
             .select(new ColumnList(
                                    OrderProduct::getOrder, 
                                    OrderProduct::getProduct,
                                    OrderProduct::getAmount
                                   ).addColumns(Product::getName)
                                    .addColumns(User::getUsername)
             ).setTransformer(Transformers.asMap())
              .list(10)
              .forEach(m -> {
                  log.info(m.toString());
              });
}

11. Update with Subquery

/**
Hibernate: 
    update
        example_stock s1_0 
    set
        amount=(s1_0.amount+cast(? as integer)) 
    where
        s1_0.product_id in ((select
            p1_0.id 
        from
            example_product p1_0 
        where
            p1_0.location=?))
**/
@ParameterizedTest
@ValueSource(strings = {"Australia", "New Zealand"})
public void test9(String location) {
    JpaSubQuery<Product, Long> subQuery = stockDao.update().subQuery(Product.class, Long.class)
                .filter(Restrictions.eq(Product::getLocation, location)).select(Product::getId);
    stockDao.update()
            .setField(Stock::getAmount, Fields.plusValue(Stock::getAmount, 1000))
            .filter(Restrictions.in(Stock::getProductId, subQuery))
            .execute();
}

12. Delete with Subquery

/**
Hibernate: 
    delete 
    from
        example_order o1_0 
    where
        exists(select
            op1_0.order_id 
        from
            example_order_product op1_0 
        join
            example_product p1_0 
                on p1_0.id=op1_0.product_id 
        where
            p1_0.id=op1_0.product_id 
            and p1_0.name in (?, ?))
**/
@ParameterizedTest
@CsvSource({"'Flashlight,Iron'"})
public void test7(String str) {
     String[] itemNames = str.split(",");
     JpaSubQuery<OrderProduct, Order> subQuery =
                orderDao.query()
                        .subQuery(OrderProduct.class, "o", Order.class)
                        .join(OrderProduct::getProduct, "p", null)
                        .filter(new FilterList()
                                .eq(Product::getId, OrderProduct::getProduct)
                                .in(Product::getName, List.of(itemNames)))
                        .select(OrderProduct::getOrder);
     int rows = orderDao.delete().filter(Restrictions.exists(subQuery)).execute();
     log.info("Affected rows: {}", rows);
}

Get Started

JDK 17 or later
Spring Boot 3.x or latest preferred
H2, Postgresql, MySQL perfect supported
pom.xml

<dependency>
    <groupId>com.github.paganini2008</groupId>
    <artifactId>easyjpa-spring-boot-starter</artifactId>
    <version>1.0.0-RC1</version>  <!-- use the latest version here -->
</dependency>

Java Configuration

@EntityScan(basePackages = {"com.github.easyjpa.test.entity"})
@EnableJpaRepositories(repositoryFactoryBeanClass = EntityDaoFactoryBean.class,
        basePackages = {"com.github.easyjpa.test.dao"})
@Configuration(proxyBeanMethods = false)
public class JpaConfig {

}

Entity Definition

@Entity
@Table(name = "example_user")
public class User {
   ...
}

@Entity
@Table(name = "example_product")
public class Product {
    ...
}

@Entity
@Table(name = "example_order")
public class Order {
    ...
}

DAO (Repository) Definition

UserDao

public interface UserDao extends EntityDao<User, Long> {

}

OrderDao

public interface OrderDao extends EntityDao<Order, Long> {

}

ProductDao

public interface ProductDao extends EntityDao<Product, Long> {

}

Contribution and License

This project is open source and licensed under the MIT License.

Project Link

For more information, visit the EasyJPA GitHub repository: paganini2008/easyjpa.

Cronsmith - The Ultimate Cron Expression Generator & Parser

Fred Feng — Mon, 10 Mar 2025 02:04:50 +0000

Cronsmith

Are cron expressions sometimes difficult to understand? Let's try a different approach to creating scheduled tasks.

Cronsmith is a powerful and versatile Java utility library designed for handling complex cron expressions in an intuitive, object-oriented manner. It offers a highly flexible and comprehensive API for generating, parsing, and scheduling cron-based tasks with ease.

Built for seamless integration, Cronsmith provides full support for both Spring and Quartz cron expressions, ensuring compatibility with widely used scheduling frameworks. Moreover, it extends traditional cron syntax by introducing advanced patterns — such as multiple numbers combined with 'L' (last) and 'W' (weekday)—which were previously unsupported, offering greater precision and flexibility in scheduling.

Cronsmith Github Repository
paganini2008/cronsmith

Features

1. Object-Oriented `CronExpression` Builder

Cronsmith allows developers to construct complex cron expressions in an intuitive, object-oriented manner. This approach enables easy customization and avoids the need for manual string construction.

Example:

@Test
    public void testA() {
        CronExpression cronExpression = new CronBuilder().everySecond(5);
        System.out.println(cronExpression.toString());
        assertEquals("*/5 * * * * ?", cronExpression.toString());
    }

    @Test
    public void testB() {
        CronExpression cronExpression = new CronBuilder().everyMinute(5).second(5).andSecond(10)
                .toSecond(30).andSecond(32).toSecond(59, 2);
        System.out.println(cronExpression.toString());
        assertEquals("5,10-30,32/2 */5 * * * ?", cronExpression.toString());
    }

    @Test
    public void testC() {
        CronExpression cronExpression = new CronBuilder().everyMonth().day(10).andDay(15).andDay(16)
                .andLastDay().everyHour(2).everyMinute(5);
        System.out.println(cronExpression.toString());
        assertEquals("0 */5 */2 10,15,16,L * ?", cronExpression.toString());
    }

    @Test
    public void testD() {
        CronExpression cronExpression = new CronBuilder().everyMonth(3).day(10).andLastWeekday()
                .hour(12).minute(1).toMinute(15, 1);
        System.out.println(cronExpression.toString());
        assertEquals("0 1-15 12 10,LW */3 ?", cronExpression.toString());
    }

    @Test
    public void testE() {
        CronExpression cronExpression =
                new CronBuilder().everyMonth().everyWeek().Mon().toFri().at(15, 10);
        System.out.println(cronExpression.toString());
        assertEquals("0 10 15 ? * MON-FRI", cronExpression.toString());
    }

    @Test
    public void testF() {
        CronExpression cronExpression = new CronBuilder().everyMonth().dayOfWeek(3, 6).everyHour(2);
        System.out.println(cronExpression.toString());
        assertTrue(cronExpression.toString().equals("0 0 */2 ? * SAT#3"));
    }

    @Test
    public void testG() {
        CronExpression cronExpression =
                new CronBuilder().year(2025).Mar().toSept().everyWeek().everyWeekday().at(9, 10);
        System.out.println(cronExpression.toString());
        assertTrue(cronExpression.toString().equals("0 10 9 ? MAR-SEP MON-FRI 2025"));
    }

    @Test
    public void testH() {
        CronExpression cronExpression =
                new CronBuilder().year(2025).toYear(2028).everyMonth(2).lastDay().hour(12);
        System.out.println(cronExpression.toString());
        assertEquals("0 0 12 L */2 ? 2025-2028", cronExpression.toString());
    }

    @Test
    public void testI() {
        CronExpression cronExpression = new CronBuilder().everyYear().June().andJuly().andAug()
                .latestWeekday(15).hour(10).toHour(18, 2);
        System.out.println(cronExpression.toString());
        assertEquals("0 0 10-18/2 15W JUN,JUL,AUG ?", cronExpression.toString());
    }

    @Test
    public void testJ() {
        CronExpression cronExpression = new CronBuilder().everyYear(2).Mar().andApr().andMay()
                .toDec().everyWeek(2).Tues().toFri().hour(10).andHour(12).toHour(22).everyMinute()
                .second(10).andSecond(20).andSecond(30);
        System.out.println(cronExpression.toString());
        assertEquals("10,20,30 * 10,12-22 ? MAR,APR,MAY-DEC TUE-FRI 2025/2",
                cronExpression.toString());
    }

    @Test
    public void testK() {
        CronExpression cronExpression = new CronBuilder().year(2025).toYear(2030).andYear(2035)
                .toEnd(2).everyMonth(2, 2).dayOfWeek(2, DayOfWeek.TUESDAY)
                .and(3, DayOfWeek.WEDNESDAY).andLastFri().hour(2).andHour(3).andHour(4)
                .toHour(17, 2).minute(0).toMinute(12, 3).andMinute(15).toMinute(40, 2).andMinute(46)
                .andMinute(48).andMinute(50).everySecond(5);
        System.out.println(cronExpression.toString());
        assertEquals(
                "*/5 0-12/3,15-40/2,46,48,50 2,3,4-17/2 ? FEB-DEC/2 TUE#2,WED#3,5L 2025-2030,2035/2",
                cronExpression.toString());
    }

    @Test
    public void testL() {
        CronExpression cronExpression = new CronBuilder().everyYear(2025, 4).everyMonth(5, 1)
                .day(10).andDay(15).andDay(20).andLastDay(2).hour(10).toHour(15, 1).at(10, 0)
                .andSecond(15).andSecond(30).andSecond(45);
        System.out.println(cronExpression.toString());
        assertEquals("0,15,30,45 10 10-15 10,15,20,L-2 MAY-DEC ? 2025/4",
                cronExpression.toString());
    }

    @Test
    public void testM() {
        CronExpression cronExpression = new CronBuilder().year().andYear(2026).andYear(2030)
                .toEnd(2).Mar().andJuly().andSept().dayOfWeek(1, DayOfWeek.SATURDAY)
                .and(2, DayOfWeek.THURSDAY).andLast(DayOfWeek.FRIDAY).hour(0).toHour(12, 3)
                .everyMinute(10).second(0).andSecond(15).andSecond(30).andSecond(45).toSecond(59);
        System.out.println(cronExpression.toString());
        assertEquals("0,15,30,45/1 */10 0-12/3 ? MAR,JUL,SEP SAT#1,THU#2,5L 2025,2026,2030/2",
                cronExpression.toString());
    }

2. Cron Expression String Parsing and Reverse Engineering to `CronExpression`

Cronsmith includes a powerful parser built with ANTLR, allowing you to parse an existing cron expression string and convert it back into a structured CronExpression object.

Example:

    @Test
    public void testA() {
        String cron = "0 0 12 * * ?";
        CronExpression cronExpression = CRON.parse(cron);
        System.out.println(cronExpression);
        assertEquals(cron, cronExpression.toString());
    }

    @Test
    public void testB() {
        String cron = "0 15 10 ? * MON-FRI";
        CronExpression cronExpression = CRON.parse(cron);
        System.out.println(cronExpression);
        assertEquals(cron, cronExpression.toString());
    }

    @Test
    public void testC() {
        String cron = "0 10,20,30 9-17 L * ?";
        CronExpression cronExpression = CRON.parse(cron);
        System.out.println(cronExpression);
        assertEquals(cron, cronExpression.toString());
    }

    @Test
    public void testD() {
        String cron = "1,3,5,7,9 3-30/3 12-16 ? * TUE#1";
        CronExpression cronExpression = CRON.parse(cron);
        System.out.println(cronExpression);
        assertEquals(cron, cronExpression.toString());
    }

    @Test
    public void testE() {
        String cron = "0 0 6 ? 1-5 MON,WED,FRI";
        CronExpression cronExpression = CRON.parse(cron);
        System.out.println(cronExpression);
        assertEquals(cron, cronExpression.toString());
    }

    @Test
    public void testF() {
        String cron = "*/5 1 12,15,18-22 LW * ? 2025";
        CronExpression cronExpression = CRON.parse(cron);
        System.out.println(cronExpression);
        assertEquals(cron, cronExpression.toString());
    }

    @Test
    public void testG() {
        String cron = "0 1 0-12,15,16-22/2 ? * 1-5 2025-2028";
        CronExpression cronExpression = CRON.parse(cron);
        System.out.println(cronExpression);
        assertEquals(cron, cronExpression.toString());
    }

    @Test
    public void testH() {
        String cron = "0 1,5,7,13,29,45 12/2 5,10,27W,L-1 APR-NOV ?";
        CronExpression cronExpression = CRON.parse(cron);
        System.out.println(cronExpression);
        assertEquals(cron, cronExpression.toString());
    }

    @Test
    public void testI() {
        String cron = "5/1 */5 12 ? 1-9 3#1,5#2,6L";
        CronExpression cronExpression = CRON.parse(cron);
        System.out.println(cronExpression);
        assertEquals(cron, cronExpression.toString());
    }

    @Test
    public void testJ() {
        String cron = "*/5 1,3,5/1 12-16 1,3,20,LW MAR-SEP ? 2026/1";
        CronExpression cronExpression = CRON.parse(cron);
        System.out.println(cronExpression);
        assertEquals(cron, cronExpression.toString());
    }

    @Test
    public void testK() {
        String cron = "5-30/7 0-12/3,15-45/2 2,3,4-17/2 ? JAN-JUL MON-THU/2 2025-2033";
        CronExpression cronExpression = CRON.parse(cron);
        System.out.println(cronExpression);
        assertEquals(cron, cronExpression.toString());
    }

    @Test
    public void testL() {
        String cron = "0,15,30,45/1 */10 0-12/3 ? MAR,JUL,SEP SAT#1,THU#2,5L 2025,2026,2030/2";
        CronExpression cronExpression = CRON.parse(cron);
        System.out.println(cronExpression);
        assertEquals(cron, cronExpression.toString());
    }

    @Test
    public void testM() {
        String cron = "*/2 0 12 1,5,9,22,L */2 ? 2025/2";
        CronExpression cronExpression = CRON.parse(cron);
        System.out.println(cronExpression);
        assertEquals(cron, cronExpression.toString());
    }

Parse Tree:

cron
    second
        secondField
            rangeWithStep
                5
                -
                30
                /
                7
    <missing SPACE>
    minute
        minuteField
            rangeWithStep
                0
                -
                12
                /
                3
        ,
        minuteField
            rangeWithStep
                15
                -
                45
                /
                2
    <missing SPACE>
    hour
        hourField
            2
        ,
        hourField
            3
        ,
        hourField
            rangeWithStep
                4
                -
                17
                /
                2
    <missing SPACE>
    dayOfMonth
        dayOfMonthField
            ?
    <missing SPACE>
    month
        monthField
            monthRange
                monthName
                    JAN
                -
                monthName
                    JUL
    <missing SPACE>
    dayOfWeek
        dayOfWeekField
            weekdayRangeWithStep
                dayOfWeekName
                    MON
                -
                dayOfWeekName
                    THU
                /
                2
    year
        yearField
            yearRange
                2025
                -
                2030
    <EOF>

Retrieve next date and times from CronExpression

public static void main(String[] args) {
   int N = 100; // Retrieve 100 items
   String cron = "10 0 12 LW 1/2 ?";
   CronExpression cronExpression = CRON.parse(cron);
   System.out.println(cronExpression);
   cronExpression.consume(l -> {
       System.out.println(l);
   }, N);
}

// Console:
// 2025-05-30T12:00:10
// 2025-07-31T12:00:10
// 2025-09-30T12:00:10
// 2025-11-28T12:00:10
// 2026-01-30T12:00:10
// 2026-03-31T12:00:10
// 2026-05-29T12:00:10
// 2026-07-31T12:00:10
// 2026-09-30T12:00:10
// 2026-11-30T12:00:10
// 2027-01-29T12:00:10
// 2027-03-31T12:00:10
// 2027-05-31T12:00:10
// 2027-07-30T12:00:10
// 2027-09-30T12:00:10
// 2027-11-30T12:00:10
// 2028-01-31T12:00:10
// 2028-03-31T12:00:10
// 2028-05-31T12:00:10
// ...

3. Built-in Scheduler

Cronsmith provides a built-in scheduler that allows you to execute tasks at specified intervals based on cron expressions. This feature makes it easy to schedule repetitive tasks efficiently.

Example: Scheduling a Task Every 5 Seconds

private ScheduledExecutorService scheduledExecutorService;

@Before
public void start() {
    scheduledExecutorService =
                Executors.newScheduledThreadPool(Runtime.getRuntime().availableProcessors() * 2);
}

@Test
public void testSchedulerAndRunTenTimes() {
    int N = 10; // Run 10 times
    final CountDownLatch latch = new CountDownLatch(N);
    final AtomicInteger counter = new AtomicInteger();

    CronFuture future = new CronBuilder()
        .everySecond(5)
        .scheduler(scheduledExecutorService)
        .setDebuged(false)
        .runTask(() -> {
            System.out.println("Run task_" + counter.incrementAndGet());
            latch.countDown();
        }, N);

    try {
        latch.await();
    } catch (InterruptedException e) {
        Thread.currentThread().interrupt();
    }

    future.cancel(true);
    assertTrue(future.isDone() && counter.get() == N);
}

@After
public void release() {
    scheduledExecutorService.shutdown();
}

4. Advanced Part

Can specify day of year

public static void main(String[] args) {
   int N = 1000;
   DateTimeFormatter dtf = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
   new CronBuilder().setZoneId(ZoneId.of("UTC")).year(2025).day(208).andDay(330).toLastDay()
                .at(12, 0).consume(ldt -> {
                    System.out.println(ldt.format(dtf));
                }, N);
}
// Console: 
// 2025-07-27 12:00:00
// 2025-11-26 12:00:00
// 2025-11-27 12:00:00
// 2025-11-28 12:00:00
// 2025-11-29 12:00:00
// 2025-11-30 12:00:00
// 2025-12-01 12:00:00
// 2025-12-02 12:00:00
// 2025-12-03 12:00:00
// 2025-12-04 12:00:00
// 2025-12-05 12:00:00
// 2025-12-06 12:00:00
// ...

Can specify week of year

public static void main(String[] args) {
    int N = 1000;
    DateTimeFormatter dtf = DateTimeFormatter.ofPattern("yyyy-MM-dd HH:mm:ss");
    new CronBuilder().setZoneId(ZoneId.of("UTC")).everyYear().week(40).andWeek(45).Mon().toFri()
                .at(12, 0).consume(ldt -> {
                    System.out.println(ldt.format(dtf));
                }, N);
}
// Console: 
// 2025-09-29 12:00:00
// 2025-09-30 12:00:00
// 2025-10-01 12:00:00
// 2025-10-02 12:00:00
// 2025-10-03 12:00:00
// 2025-11-03 12:00:00
// 2025-11-04 12:00:00
// 2025-11-05 12:00:00
// 2025-11-06 12:00:00
// 2025-11-07 12:00:00
// 2026-09-28 12:00:00
// 2026-09-29 12:00:00
// 2026-09-30 12:00:00
// 2026-10-01 12:00:00
// 2026-10-02 12:00:00
// ...

Installation

Support Jdk1.8 or later

To use Cronsmith in your Java project, add the following dependency to your pom.xml (if using Maven):

<dependency>
    <groupId>com.github.paganini2008</groupId>
    <artifactId>cronsmith</artifactId>
    <version>1.0.0-RC1</version>
</dependency>

If using Gradle:

dependencies {
    implementation 'com.github.paganini2008:cronsmith:1.0.0-RC1'
}

Getting Started

Add Cronsmith as a dependency in your project.
Use CronBuilder to create complex cron expressions.
Parse existing cron expressions using CRON.parse().
Schedule tasks using the built-in scheduler.

License

This project is licensed under the MIT License - see the LICENSE file for details.

Cronsmith Github Repository

paganini2008/cronsmith

Introduction to Greenfinger - High Performance Distributed Web Crawler Framework

Fred Feng — Sun, 02 Feb 2025 06:03:17 +0000

GreenFinger is a high-performance, highly scalable distributed web crawler built in Java. Designed for both enterprise and individual users, it offers an intuitive user interface and minimal configuration, enabling seamless and efficient web resource extraction. As an open-source solution, GreenFinger provides a powerful yet user-friendly approach to large-scale web crawling and data acquisition.

🌟Features:

Seamless Spring Boot Integration
Natively integrates with Spring Boot, ensuring effortless configuration, deployment, and maintenance.
Scalable, High-Throughput Distributed Crawling
Architected for distributed environments, enabling seamless horizontal scaling to handle massive workloads efficiently.
Optimized Network Communication with Netty
Leverages Netty for ultra-low-latency networking, with additional support for Mina and Grizzly for flexible communication strategies.
Enterprise-Grade URL Deduplication
Implements billion-scale deduplication using Bloom Filter and RocksDB, ensuring optimal storage efficiency and crawl accuracy.
Granular URL Customization
Supports fine-grained control over URL selection, allowing users to define initial URLs, retain only relevant URLs, and exclude undesired links dynamically.
Advanced Fault Tolerance & Crawler Constraints
Incorporates intelligent retry mechanisms, configurable timeouts, target URL limits, and maximum crawl depth enforcement for robust error handling.
Multi-Engine Web Content Extraction
Integrates Playwright, Selenium, and HtmlUnit to capture and process dynamic web content efficiently.
Strict Adherence to Robots.txt
Fully complies with the Robots Exclusion Protocol, ensuring ethical and responsible web crawling.
Comprehensive Developer API
Exposes a rich set of APIs, enabling seamless customization, extension, and integration into diverse ecosystems.
Automated Authentication Handling
Supports intelligent login and logout workflows, facilitating seamless authentication across secured web portals.
Version-Controlled Web Document Management
Assigns unique versioning to crawled documents, enabling multi-version indexing for enhanced content tracking and retrieval.
Intuitive Angular-Based Web Interface
Provides a modern, interactive dashboard built with Angular, empowering users with real-time monitoring, configuration, and management capabilities.

🚀 Technology Stack

Technology	Version Requirement	Description
☕ JDK	17 or later	Core Java runtime environment
🌱 Spring Boot	2.7.18	Backend framework for microservices and rapid development
⚡ Netty	4.x	High-performance asynchronous networking framework
🔥 Redis	7.x or later	In-memory data store for caching and message queuing
🐘 PostgreSQL	9.x or later	High-performance, open-source relational database
🔍 ElasticSearch	7.16.2 or later	Distributed search and analytics engine
🕷 Selenium	4.x	Web automation framework for headless and UI-based scraping
🎭 Playwright	1.48	Modern browser automation tool for scraping and testing
📄 HtmlUnit	2.6	Lightweight headless browser for quick HTML processing
🌐 Angular	19.x	Frontend framework for building interactive web applications
🎨 Angular Material	Latest	UI component library for modern, responsive designs

Install:

Git Repository： https://github.com/paganini2008/greenfinger.git
Directory Structure：

📂 **greenfinger**  
├── 📂 **greenfinger-ui**  
│   ├── 📜 pom.xml  
│   ├── 📂 src  
│   │   ├── 📂 config  # Configuration files  
│   │   ├── 📂 db      # Database-related scripts and configurations  
│   │   └── ...  
├── 📂 **greenfinger-spring-boot-starter**  
│   ├── 📜 pom.xml  
│   ├── 📂 src  
│   └── ...  
├── 📜 LICENSE  
├── 📜 pom.xml  
└── 📜 README.md

Steps:

Modify configuration:

spring:
  redis:
    database: 0
    host: 127.0.0.1
    port: 6379
    password: 123456
  elasticsearch:
    rest:
      uris: http://127.0.0.1:9200
      connection-timeout: 10000
      read-timeout: 60000
  datasource:
    driver-class-name: org.postgresql.Driver
    url: jdbc:postgresql://localhost:5432/test?characterEncoding=utf8&allowMultiQueries=true&useSSL=false&stringtype=unspecified
    username: admin
    password: 123456
# Binding host name is preferred
doodler:
  transmitter:
    nio:
      server:
        bindHostName: 127.0.0.1
# Internal Work ThreadPool Threads      
greenfinger:
  workThreads: 1000

Create database and import table scripts execute db/crawler.sql
mvn clean install
run jar with java --add-opens=java.base/java.lang=ALL-UNNAMED -jar greenfinger-ui-service-1.0.0-SNAPSHOT-sources.jar
Open the Web UI
http://localhost:6120/ui/index.html

Greenfinger UI Guide

Catalog Management

Create a catalog

Edit a catalog

📌 Parameters

Parameter	Description
ID	Unique identifier for the catalog entry.
Name	The name of the catalog item.
URL	The target website to be crawled.
Category	User-defined category for classification.
Start URL	The initial entry point for crawling.
Included Path Pattern	Defines URL patterns to be included in the crawl.
Excluded Path Pattern	Defines URL patterns to be excluded from crawling.
Page Encoding	Character encoding of the webpage (e.g., UTF-8).
Max Fetch Size	The maximum number of URLs to fetch in a single crawl session.
Fetch Duration	The total duration (in seconds) the crawl process should run.
Fetch Depth	Defines how deep the crawler should go (-1 means unlimited depth).
Fetch Interval	Time interval (in milliseconds) between consecutive requests.
Counting Type	Determines how crawled data is counted:
	- Total Fetched Count (all requests made)
	- Filtered Count (URLs removed by filters)
	- Failed Count (requests that failed)
	- Saved Resource Count (stored data)
	- Indexed Count (successfully indexed pages)
	- Duplicate Count (repeated entries detected)
Max Retry Count	The number of times a failed request will be retried.
URL Path Filter	Duplicate URL filtering mechanism:
	- Redis BloomFilter (efficient large-scale deduplication)
	- RocksDB (persistent local storage-based deduplication)
HTML Extractor	Webpage extraction engine:
	- Playwright (modern headless browser automation)
	- Selenium (full-browser-based interaction)
	- HtmlUnit (lightweight Java-based HTML parsing)
URL Path Acceptor	Defines custom URL patterns to be accepted during crawling.
Credential Handler	Custom Login Handler (full class name) for handling authentication logic.
Indexed Checkbox	Determines whether the crawled data should be indexed (`Checked = Yes`).
Last Modified	Timestamp of the last modification.
Save Button	Confirms and saves the catalog settings.
Cancel Button	Discards changes and exits the editor.

Run web crawler

Icon	Action	Description
🧹 Clean	Remove unnecessary or outdated data to free up space.
🌐 Crawl	Start a new crawling session to gather updated data.
🛠 Rebuild	Restart the crawler to reprocess and fetch data again.
📄 Summary	View a high-level summary of the crawling results and statistics.
❌ Interrupt	Stop an ongoing crawling session immediately.
🔄 Sync	Synchronize data with external sources or distributed nodes.
🚀 Upgrade	Perform system upgrades, applying new features or patches.

Monitor

Query

Customize your application

Application Integration

Step1: add dependency in your pom.xml:

<dependency>
  <groupId>com.github.paganini2008</groupId>
  <artifactId>greenfinger-spring-boot-starter</artifactId>
  <version>1.0.0-SNAPSHOT</version>
</dependency>

Step2: add @EnableGreenFingerServer on the main:

@EnableAsync(proxyTargetClass = true)
@EnableScheduling
@EnableGreenfingerServer
@SpringBootApplication
public class GreenFingerServerConsoleMain {

    public static void main(String[] args) {
        SpringApplication.run(GreenFingerServerConsoleMain.class, args);
    }
}

Step3: Run it

Extensible points:

Interface	Description
ResourceIndexManager	Manages indexing operations for crawled resources, ensuring efficient storage, retrieval, and versioning of indexed data.
ResourceManager	Handles the lifecycle of crawled URLs, including storage, deduplication, and scheduling for further processing.
ExistingUrlPathFilter	Implements a URL deduplication mechanism to prevent redundant crawling of previously processed URLs.
GlobalStateManager	Maintains and synchronizes the global state of the crawler across distributed nodes, ensuring coordinated execution.
InterruptionChecker	Monitors execution status and provides mechanisms to gracefully interrupt crawling operations when necessary.
UrlPathAcceptor	Defines a custom business logic filter for URL acceptance, allowing precise control over which URLs are processed.
ExtractorCredentialHandler	Provides a pluggable authentication handler to automate login flows for accessing restricted content.
Extractor	Defines the component responsible for extracting page content, supporting multiple implementations such as Playwright, Selenium, and HtmlUnit.

DEV Community: Fred Feng

EasyJPA – Your Best Partner for JPA Development!

It's time to say goodbye to JPA Criteria API complexity! EasyJPA makes your code sleek, simple, and powerful!

Features

Examples

Get Started

UserDao

OrderDao

ProductDao

Contribution and License

Project Link

Cronsmith - The Ultimate Cron Expression Generator & Parser

Cronsmith

Are cron expressions sometimes difficult to understand? Let's try a different approach to creating scheduled tasks.

Features

1. Object-Oriented CronExpression Builder

Example:

2. Cron Expression String Parsing and Reverse Engineering to CronExpression

Example:

3. Built-in Scheduler

Example: Scheduling a Task Every 5 Seconds

4. Advanced Part

Installation

Getting Started

License

Cronsmith Github Repository

Introduction to Greenfinger - High Performance Distributed Web Crawler Framework

🌟Features:

🚀 Technology Stack

Install:

Steps:

Greenfinger UI Guide

Catalog Management

Create a catalog

Edit a catalog

📌 Parameters

Run web crawler

Monitor

Query

Customize your application

Application Integration

Extensible points:

1. Object-Oriented `CronExpression` Builder

2. Cron Expression String Parsing and Reverse Engineering to `CronExpression`