DEV Community: Mallikarjun H T

Lucene V/S KQL

Mallikarjun H T — Thu, 01 May 2025 14:20:10 +0000

A Tour of KQL vs Lucene

Elasticsearch is a search engine, and Kibana can be used to search documents in Elasticsearch.
A search is executed by sending a query to Elasticsearch. A query can answer many different types of questions.

For example:

Who are the customers with the first name Edison?
What are the names of customers in India?
Are there any orders for Men’s Clothing OR Electronics section?

Should I use KQL or Lucene in the Kibana query bar?

It Depends, Both KQL and Lucine have a quite a lot in common.
I recoment to start with KQL then switch to Lucine for Specific Aggrigation use Case.

Learn By Executing.

in order to learn use the Free trial or download the pre prepared data sets to run locally if cloud trial expires.

KQL Features.

once you load your data you can use kibana to explore and learn about the features.

NOTE: KQL is case in sencitive and Lucine operaters are All Caps only

Autocomplete - KQL
Full Testsearch - KQL and Lucine

Exact Match

You can search for documents that contain an exact term in a provided field of data type keyword(opens in a new tab), which will not be analyzed. Not analyzed strings are case sensitive.

Example: Customer_first_name.keyword: Elyssa

Note: keyword is a type that only supports Exact search, if not given while searching then anything close to the word is returned.

Phrase search

To search text fields where the search terms are in the order provided, surround the value in double quotation marks, as follows:

Example: “Elyssa Underwood” - this is all field search
Example: Customer_full_name: “Elyssa Underwood” - this ia field specific search

Fuzzy Search only available in Lucine

arjun~1 - will match any word with 1ch differance, the number determines the no of charecters that can differ
arj*n - whild charecter search one or more charecters can occur in between arj and n
arj?n - single charecter only

NOTE: bad idea to use ** or ?* as this can cause perfomance issues.
to allow change the settings in elastic search.

Regex

A regular expression is a way to match patterns in data using placeholder characters called operators. It returns documents that contain terms matching a regular expression. Syntax is based on the Lucene regular expression engine.

Example: /.+91[0-9]{9}./

this will match number with +91 xxx xxx xxx

Ranges

To search documents that contain terms within a provided range, use KQL’s range syntax. You can use the range syntax for string values, IP addresses, and timestamps.

KQL Supports the following syntax >, >=, <, and <=

Lucene supports bracketed range syntax. Ranges can be specified for date, numeric or string fields. Inclusive ranges are specified with square brackets [min TO max] and exclusive ranges with curly brackets {min TO max}.

 date:[2012-01-01 TO 2012-12-31]
 count:[1 TO 5]
 tag:{alpha TO omega}

## Proximity Search
Lucine Exclusive
Proximity search(opens in a new tab) allows the specified words to be further apart or in a different order, a proximity search allows us to specify a maximum edit distance of words in a phrase. The closer the text in a field is to the original order specified in the query string, the more relevant that document is considered to be

“open data”~1

The above example will find all the documents where “open data” is found as a phrase and also “open source data” is a hit where one more term is present in between open and data.

Boosting

Lucene

Use the boost operator ^ to make one term more relevant than another. For instance, if we want to find all documents about foxes, but we are especially interested in quick foxes:

quick^2 fox

The default boost value is 1, but can be any positive floating point number. Boosts between 0 and 1 reduce relevance.

Boosts can also be applied to phrases or to groups:

"john smith"^2 (foo bar)^4

Searching for IP addresses

KQL Lucene

Use "" around IP addresses while using Lucene syntax.
Lucene also supports CIDR notation.

Example: "192.168.03.01"

Nested fields

KQL

Querying nested fields requires a special syntax.

Example: user.names:{ first: "Alyssa" and last: "Underwood" }

Runtime fields

KQL Lucene

A runtime field is a field that is evaluated at query time.

KQL and syntex

Mallikarjun H T — Wed, 23 Apr 2025 14:07:07 +0000

Should I use KQL or Lucene in the Kibana query bar?

It depends! KQL and Lucene have quite a lot in common, but there are some differences. Let's explore some key-features one by one to find out the best use-case for each language.

For new users, we recommend starting with KQL(opens in a new tab) and switching to Lucene(opens in a new tab) if any particular feature is unavailable in KQL.

KQL

Kibana Query Language (KQL) is a simple text-based query language for filtering data.
KQL only filters data, and has no role in aggregating, transforming, or sorting data.

Semi-structured search

free text search with field-based search.

The semi-structured search will filter documents for matches, and only return matching documents.

Filter for documents where a field exists

To filter documents for which an indexed value exists for a given field, use the * operator.

For example, to filter for documents where the http.request.method field exists, use the following syntax:

http.request.method: *

This checks for any indexed value, including an empty string.

Filter for documents that match a value

Use KQL to filter for documents that match a specific number, text, date, or boolean value. For example, to filter for documents where the http.request.method is GET, use the following query:
http.request.method: GET

to search all fields for “Hello”, use the following:

Hello

To querying keyword, numeric, date, or boolean fields, the value must be an exact match, including punctuation and case.

To Search text field that has "null pointer" then querry using http.request.body.content: "null pointer" if the \" are not used then you will get all documents that has null pointer OR Pointer null. This is called term querry.

to search urls use \ or you will find syntax error OR try with ".
Example:

http.request.referrer: "https://example.com"  
http.request.referrer: https\://example.com

[!Note]
\():<>"* excape These Charecters with \

Filter for documents within a range

to search for all documents for which http.response.bytes is less than 10000, use the following syntax:
http.response.bytes < 10000

to search In Range use this
http.response.bytes > 10000 and http.response.bytes <= 20000

Filter for documents using wildcards

To search for documents matching a pattern, use the wildcard syntax. For example, to find documents where http.response.status_code begins with a 4, use the following syntax:

http.response.status_code: 4*
By default, leading wildcards are not allowed for performance reasons. You can modify this with the query:allowLeadingWildcards advanced setting.

[!Note]
Only * is currently supported. This matches zero or more characters.

Negating a query

use NOT in front of querry like:
NOT http.request.method: GET

Querying nested fields

Use . operater to query nested fields like:
user.names:{ first: "Alice" and last: "White" }

Installing Elastic Search 7.14 with docker

Mallikarjun H T — Tue, 10 Sep 2024 04:52:53 +0000

Welcome to the ultimate guide on Elastic Search and Kibana! If you're looking for a powerful, open-source tool that can help you search, analyze, and visualize your data like never before, then look no further. In this comprehensive guide, we'll take you through everything you need to know about Elastic Search and Kibana - from what they are and how they work together to their installation with Docker. Whether you're a developer or just getting started with these tools, get ready to learn everything there is to know about using Elastic Search and Kibana with Docker in this exciting blog post!

Introduction to Elasticsearch and Kibana

Elasticsearch is a powerful open source search and analytics engine that makes data easy to explore. Kibana is an open source data visualization platform that allows you to interact with your data through beautiful visualizations. In this guide, we'll show you how to use Docker to install and run Elasticsearch and Kibana on your local machine.

What is Elastic Search?

Elastic Search is a powerful, open source, distributed search and analytics engine. It is built on top of the Apache Lucene search library and supports full text search, as well as structured data search with SQL-like query syntax. Elastic Search can be used to power search for websites, applications, and enterprise search solutions. It is also commonly used for log analysis, monitoring, and security analytics.

What is Kibana?

Kibana is an open source data visualization plugin for the Elasticsearch stack. It provides a rich set of tools to help you visualize and explore your data in Elasticsearch. Kibana is a great way to get started with visualizing your data in Elasticsearch.

How to Install Elastic Search and Kibana with Docker

version: '2.2'
services:
  node01:
    image: docker.elastic.co/elasticsearch/elasticsearch:7.14.1
    container_name: node01
    environment:
      - node.name=node01
      - cluster.name=es-my-cluster
      - discovery.type=single-node
      - bootstrap.memory_lock=true
      - "ES_JAVA_OPTS=-Xms512m -Xmx512m"
    ulimits:
      memlock:
        soft: -1
        hard: -1
    volumes:
      - data01:/usr/share/elasticsearch/data
    ports:
      - 9200:9200
    networks:
      - elastic
  kib01test:
    image: docker.elastic.co/kibana/kibana:7.14.1
    container_name : kib01test
    environment:
      - ELASTICSEARCH_HOSTS=http://node01:9200
    ports:
      - 5601:5601
    networks: 
      - elastic

volumes:
  data01:
    driver: local

networks:
  elastic:
    driver: bridge

ES - Node APIs

Mallikarjun H T — Tue, 13 Aug 2024 09:40:44 +0000

# If no filters are given, the default is to select all nodes
curl -X GET "localhost:9200/_nodes?pretty"
# Explicitly select all nodes
curl -X GET "localhost:9200/_nodes/_all?pretty"
# Select just the local node
curl -X GET "localhost:9200/_nodes/_local?pretty"
# Select the elected master node
curl -X GET "localhost:9200/_nodes/_master?pretty"
# Select nodes by name, which can include wildcards
curl -X GET "localhost:9200/_nodes/node_name_goes_here?pretty"
curl -X GET "localhost:9200/_nodes/node_name_goes_*?pretty"
# Select nodes by address, which can include wildcards
curl -X GET "localhost:9200/_nodes/10.0.0.3,10.0.0.4?pretty"
curl -X GET "localhost:9200/_nodes/10.0.0.*?pretty"
# Select nodes by role
curl -X GET "localhost:9200/_nodes/_all,master:false?pretty"
curl -X GET "localhost:9200/_nodes/data:true,ingest:true?pretty"
curl -X GET "localhost:9200/_nodes/coordinating_only:true?pretty"
# Select nodes by custom attribute (e.g. with something like `node.attr.rack: 2` in the configuration file)
curl -X GET "localhost:9200/_nodes/rack:2?pretty"
curl -X GET "localhost:9200/_nodes/ra*:2?pretty"
curl -X GET "localhost:9200/_nodes/ra*:2*?pretty"

ES - CAT APIs

Mallikarjun H T — Tue, 13 Aug 2024 09:38:43 +0000

cat APIs

Introduction

All the cat commands accept a query string parameter help to see all the headers and info they provide, and the /_cat command alone lists all the available commands.

Common parameters

Verbose - curl -X GET "localhost:9200/_cat/master?v&pretty"
Help - curl -X GET "localhost:9200/_cat/master?help&pretty"
Headers - curl -X GET "localhost:9200/_cat/nodes?h=ip,port,heapPercent,name&pretty"
Numeric formats - curl 'localhost:9200/_cat/indices?bytes=b' | sort -rnk8
Response as text, json, smile, yaml or cbor - curl 'localhost:9200/_cat/indices?format=json&pretty'
Sort - curl -X GET "localhost:9200/_cat/templates?v&s=order:desc,index_patterns&pretty"

cat aliases

curl -X GET "localhost:9200/_cat/aliases?v&pretty"

cat allocation

curl -X GET "localhost:9200/_cat/allocation?v&pretty"

cat count

curl -X GET "localhost:9200/_cat/count?v&pretty"

curl -X GET "localhost:9200/_cat/count/twitter?v&pretty"

cat fielddata

curl -X GET "localhost:9200/_cat/fielddata?v&pretty"

cat health

curl -X GET "localhost:9200/_cat/health?v&pretty"

cat indices

curl -X GET "localhost:9200/_cat/indices?pretty"

Apply Filter

curl -X GET "localhost:9200/_cat/indices/twi*?pretty"

cat un healthy index

indices which are not ready

curl -X GET "localhost:9200/_cat/indices?v&health=yellow&pretty"

sort by dock count

curl -X GET "localhost:9200/_cat/indices?v&s=docs.count:desc&pretty"

cat master

curl -X GET "localhost:9200/_cat/master?v&pretty"

cat nodes

curl -X GET "localhost:9200/_cat/nodes?v&pretty"

Documentation

cat pending tasks

curl -X GET "localhost:9200/_cat/pending_tasks?v&pretty"

cat plugins

curl -X GET "localhost:9200/_cat/plugins?v&s=component&h=name,component,version,description&pretty"

cat recovery

curl -X GET "localhost:9200/_cat/recovery?v&pretty"

cat repositories

curl -X GET "localhost:9200/_cat/repositories?v&pretty"

cat thread pool

curl -X GET "localhost:9200/_cat/thread_pool?pretty"

for field data visit Documentation

cat shards

curl -X GET "localhost:9200/_cat/shards?pretty"

Reasion for un assigned shared look Documentation

INDEX_CREATED | Unassigned as a result of an API creation of an index.

CLUSTER_RECOVERED | Unassigned as a result of a full cluster recovery.

INDEX_REOPENED | Unassigned as a result of opening a closed index.

DANGLING_INDEX_IMPORTED | Unassigned as a result of importing a dangling index.

NEW_INDEX_RESTORED | Unassigned as a result of restoring into a new index.

EXISTING_INDEX_RESTORED | Unassigned as a result of restoring into a closed index.

REPLICA_ADDED | Unassigned as a result of explicit addition of a replica.

ALLOCATION_FAILED | Unassigned as a result of a failed allocation of the shard.

NODE_LEFT | Unassigned as a result of the node hosting it leaving the cluster.

REROUTE_CANCELLED | Unassigned as a result of explicit cancel reroute command.

REINITIALIZED | When a shard moves from started back to initializing, for example, with shadow replicas.

REALLOCATED_REPLICA | A better replica location is identified and causes the existing replica allocation to be cancelled.

cat segments

this where your data is saved on the FS.

curl -X GET "localhost:9200/_cat/segments?v&pretty"

snapshots

curl -X GET "localhost:9200/_cat/snapshots/repo1?v&s=id&pretty"

templates

curl -X GET "localhost:9200/_cat/templates?v&s=name&pretty"

ES - Index API - part 4

Mallikarjun H T — Mon, 05 Aug 2024 08:04:39 +0000

Indices Stats

curl -X GET "localhost:9200/_stats?pretty"

Indices Segments

curl -X GET "localhost:9200/test/_segments?pretty"

curl -X GET "localhost:9200/test/_segments?verbose=true&pretty"

Indices Recovery

curl -X GET "localhost:9200/_recovery?human&pretty"

curl -X GET "localhost:9200/_recovery?human&detailed=true&pretty"

Indices Shard Stores

# return information of only index test
curl -X GET "localhost:9200/test/_shard_stores?pretty"
# return information of only test1 and test2 indices
curl -X GET "localhost:9200/test1,test2/_shard_stores?pretty"
# return information of all indices
curl -X GET "localhost:9200/_shard_stores?pretty"

Clear Cache

curl -X POST "localhost:9200/twitter/_cache/clear?pretty"

Flush

curl -X POST "localhost:9200/twitter/_flush?pretty"

Synced Flush

curl -X POST "localhost:9200/twitter/_flush/synced?pretty"

Refresh

curl -X POST "localhost:9200/twitter/_refresh?pretty"

Force Merge

curl -X POST "localhost:9200/twitter/_forcemerge?pretty"

ES Index API - part 3

Mallikarjun H T — Mon, 05 Aug 2024 07:47:49 +0000

PUT Mappings

create index with mappings

curl -X PUT "localhost:9200/twitter?pretty" -H 'Content-Type: application/json' -d'
{}'

curl -X PUT "localhost:9200/twitter/_mapping/_doc?pretty" -H 'Content-Type: application/json' -d'
{
  "properties": {
    "email": {
      "type": "keyword"
    }
  }
}'

update existing index

curl -X PUT "localhost:9200/my_index?pretty" -H 'Content-Type: application/json' -d'
{
  "mappings": {
    "_doc": {
      "properties": {
        "name": {
          "properties": {
            "first": {
              "type": "text"
            }
          }
        },
        "user_id": {
          "type": "keyword"
        }
      }
    }
  }
}'

curl -X PUT "localhost:9200/my_index/_mapping/_doc?pretty" -H 'Content-Type: application/json' -d'
{
  "properties": {
    "name": {
      "properties": {
        "last": { 
          "type": "text"
        }
      }
    },
    "user_id": {
      "type": "keyword",
      "ignore_above": 100 
    }
  }
}'

Get Mappings

curl -X GET "localhost:9200/twitter/_mapping/_doc?pretty"

GET field mapping

curl -X GET "localhost:9200/publications/_mapping/_doc/field/title?pretty"

ES Index API- part 2

Mallikarjun H T — Mon, 05 Aug 2024 07:33:31 +0000

Delete Index

curl -X DELETE "localhost:9200/twitter?pretty"

Get Index

this will return mappings and settings if index exist

curl -X GET "localhost:9200/twitter?pretty"

check if index exist

curl -I "localhost:9200/twitter?pretty"

Open / Close Index API

use this with caution - like updating or to prevent new document insertion - search will work but indexing will not work

curl -X POST "localhost:9200/my_index/_close?pretty"
curl -X POST "localhost:9200/my_index/_open?pretty"

Shrink Index

use this to reduce the number of shards for version < 7 it is 5 and above it is 1 shard.

Preparing an index for shrinking

curl -X PUT "localhost:9200/my_source_index/_settings?pretty" -H 'Content-Type: application/json' -d'
{
  "settings": {
    "index.routing.allocation.require._name": "shrink_node_name", 
    "index.blocks.write": true 
  }
}
'

Shrinking an index

curl -X POST "localhost:9200/my_source_index/_shrink/my_target_index?copy_settings=true&pretty" -H 'Content-Type: application/json' -d'
{
  "settings": {
    "index.routing.allocation.require._name": null, 
    "index.blocks.write": null 
  }
}
'

Shrinking with settings

curl -X POST "localhost:9200/my_source_index/_shrink/my_target_index?copy_settings=true&pretty" -H 'Content-Type: application/json' -d'
{
  "settings": {
    "index.number_of_replicas": 1,
    "index.number_of_shards": 1, 
    "index.codec": "best_compression" 
  },
  "aliases": {
    "my_search_indices": {}
  }
}
'

Rollover Index

Defining the new index

curl -X PUT "localhost:9200/logs-000001?pretty" -H 'Content-Type: application/json' -d'
{
  "aliases": {
    "logs_write": {}
  }
}'

curl -X POST "localhost:9200/logs_write/_rollover?pretty" -H 'Content-Type: application/json' -d'
{
  "conditions" : {
    "max_age": "7d",
    "max_docs": 1000,
    "max_size": "5gb"
  },
  "settings": {
    "index.number_of_shards": 2
  }
}'

Dry run

curl -X PUT "localhost:9200/logs-000001?pretty" -H 'Content-Type: application/json' -d'
{
  "aliases": {
    "logs_write": {}
  }
}'

curl -X POST "localhost:9200/logs_write/_rollover?dry_run&pretty" -H 'Content-Type: application/json' -d'
{
  "conditions" : {
    "max_age": "7d",
    "max_docs": 1000,
    "max_size": "5gb"
  }
}'

ES index API - part 1

Mallikarjun H T — Mon, 05 Aug 2024 07:13:56 +0000

Indices APIs

Index management:

Create Index
Delete Index
Get Index
Indices Exists
Open / Close Index API
Shrink Index
Split Index
Rollover Index

### Mapping management:

Put Mapping
Get Mapping
Get Field Mapping
Types Exists

Alias management:

Index Aliases

Index settings:

Update Indices Settings
Get Settings
Analyze
Index Templates

Monitoring:

Indices Stats
Indices Segments
Indices Recovery
Indices Shard Stores

Status management:

Clear Cache
Refresh
Flush
Force Merge

Create Index

index creation without any mappings

curl -X PUT "localhost:9200/twitter?pretty"

NOTE:

Index name limitations

There are several limitations to what you can name your index. The complete list of limitations are:

Lowercase only
Cannot include \, /, *, ?, ", <, >, |, ` ` (space character), ,, #
Indices prior to 7.0 could contain a colon (:), but that’s been deprecated and won’t be supported in 7.0+
Cannot start with -, _, +
Cannot be . or ..
Cannot be longer than 255 bytes (note it is bytes, so multi-byte characters will count towards the 255 limit faster)

Index Settings

To update index use this, in this payload we are setting number of shards and no of replicas

curl -X PUT "localhost:9200/twitter?pretty" -H 'Content-Type: application/json' -d'
{
    "settings" : {
        "number_of_shards" : 3,
        "number_of_replicas" : 2
    }
}
'

Mappings

setting mappings to an index while creation

curl -X PUT "localhost:9200/test?pretty" -H 'Content-Type: application/json' -d'
{
    "settings" : {
        "number_of_shards" : 1
    },
    "mappings" : {
        "_doc" : {
            "properties" : {
                "field1" : { "type" : "text" }
            }
        }
    }
}
'

setting Aliases on existing index

curl -XPUT localhost:9200/index1/_alias/index2
multiple Aliases and filtering

curl -X PUT "localhost:9200/test?pretty" -H 'Content-Type: application/json' -d'
{
    "aliases" : {
        "alias_1" : {},
        "alias_2" : {
            "filter" : {
                "term" : {"user" : "kimchy" }
            },
            "routing" : "kimchy"
        }
    }
}
'

java prep - part 3

Mallikarjun H T — Tue, 18 Jun 2024 02:42:49 +0000

can we disable the default behaviour of spring explain with example

Yes, in Spring Framework, you can disable or customize default behaviors by overriding configurations or using specific annotations to alter how Spring manages components and processes requests. Let's explore a couple of examples where you might want to disable default behaviors in Spring:

Example 1: Disabling Default Spring Security Configuration

By default, Spring Security applies a set of security configurations to your application, including basic authentication and authorization rules. You might want to disable these default behaviors if you have custom security requirements.

Step-by-step Example:

Disable Security Auto-Configuration: To disable Spring Security's default configuration, you can exclude the SecurityAutoConfiguration class from auto-configuration in your main application class or configuration class.

   import org.springframework.boot.SpringApplication;
   import org.springframework.boot.autoconfigure.SpringBootApplication;
   import org.springframework.boot.autoconfigure.security.servlet.SecurityAutoConfiguration;

   @SpringBootApplication(exclude = {SecurityAutoConfiguration.class})
   public class MyApplication {

       public static void main(String[] args) {
           SpringApplication.run(MyApplication.class, args);
       }
   }

In this example:

@SpringBootApplication is annotated with exclude = {SecurityAutoConfiguration.class}, which disables Spring Security's auto-configuration.

Customize Security Configuration: After disabling default configuration, you can then customize security settings by creating your own SecurityConfig class and configuring it based on your specific requirements using @EnableWebSecurity and extending WebSecurityConfigurerAdapter.

   import org.springframework.context.annotation.Configuration;
   import org.springframework.security.config.annotation.web.configuration.EnableWebSecurity;
   import org.springframework.security.config.annotation.web.configuration.WebSecurityConfigurerAdapter;

   @Configuration
   @EnableWebSecurity
   public class SecurityConfig extends WebSecurityConfigurerAdapter {

       @Override
       protected void configure(HttpSecurity http) throws Exception {
           // Customize your security configuration here
           http
               .authorizeRequests()
                   .antMatchers("/public/**").permitAll()
                   .anyRequest().authenticated()
                   .and()
               .formLogin()
                   .loginPage("/login")
                   .permitAll()
                   .and()
               .logout()
                   .permitAll();
       }
   }

This SecurityConfig class provides customized security configurations, overriding the default behavior that was disabled.

Example 2: Disabling Spring Boot Banner

Spring Boot displays a banner by default during application startup. You may want to disable this banner for specific reasons, such as cleaner log output.

Step-by-step Example:

Disable Banner Programmatically: You can disable the banner programmatically by setting spring.main.banner-mode property to OFF in your application.properties or application.yml.

   spring.main.banner-mode=OFF

This configuration instructs Spring Boot not to display the banner during application startup.

Disable Banner via Configuration Class: Alternatively, you can disable the banner by creating a configuration class and using SpringApplication.setBannerMode(Banner.Mode.OFF) in your main application class.

   import org.springframework.boot.Banner;
   import org.springframework.boot.SpringApplication;
   import org.springframework.boot.autoconfigure.SpringBootApplication;

   @SpringBootApplication
   public class MyApplication {

       public static void main(String[] args) {
           SpringApplication app = new SpringApplication(MyApplication.class);
           app.setBannerMode(Banner.Mode.OFF);
           app.run(args);
       }
   }

Here, SpringApplication.setBannerMode(Banner.Mode.OFF) disables the banner explicitly for the MyApplication.

Conclusion:

Spring Framework provides flexibility to disable default behaviors by excluding auto-configurations, overriding configuration classes, or configuring properties. This approach allows developers to customize Spring applications according to specific requirements, ensuring that the framework's default behaviors align with application needs.

How caching works with example/ archtecture diagram as code

Caching in software systems, including those built with Spring Boot, involves storing frequently accessed data in a faster access storage to improve performance and reduce load on backend systems.

import org.springframework.cache.annotation.Cacheable;
import org.springframework.stereotype.Service;

@Service
public class UserService {

    @Cacheable(value = "usersCache", key = "#userId")
    public User getUserDetails(Long userId) {
        // Simulate fetching user details from a database
        User user = userRepository.findById(userId);
        return user;
    }
}

# Enable caching
spring.cache.type=ehcache

with Redis example

import java.io.Serializable;

public class Book implements Serializable {
    private String isbn;
    private String title;
    private String author;

    // Getters and setters (omitted for brevity)

    @Override
    public String toString() {
        return "Book{" +
                "isbn='" + isbn + '\'' +
                ", title='" + title + '\'' +
                ", author='" + author + '\'' +
                '}';
    }
}

import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.data.redis.core.RedisTemplate;
import org.springframework.stereotype.Service;

@Service
public class BookService {

    private static final String REDIS_KEY = "books";

    @Autowired
    private RedisTemplate<String, Book> redisTemplate;

    public void saveBook(Book book) {
        redisTemplate.opsForHash().put(REDIS_KEY, book.getIsbn(), book);
    }

    public Book getBook(String isbn) {
        return (Book) redisTemplate.opsForHash().get(REDIS_KEY, isbn);
    }

    public void deleteBook(String isbn) {
        redisTemplate.opsForHash().delete(REDIS_KEY, isbn);
    }
}

java interview prep part 2

Mallikarjun H T — Tue, 18 Jun 2024 02:25:52 +0000

Explain restcontroller annotation in springboot

In Spring Boot, @RestController is a specialized version of the @Controller annotation. It is used to indicate that the class is a RESTful controller that handles HTTP requests and directly maps them to the methods inside the class.

import org.springframework.web.bind.annotation.GetMapping;
import org.springframework.web.bind.annotation.RestController;

@RestController
public class HelloController {

    @GetMapping("/hello")
    public String hello() {
        return "Hello, World!";
    }
}

Purpose of @RestController:
API Endpoint Handling:

@RestController combines @Controller and @ResponseBody. It is primarily used to create RESTful web services where HTTP requests (GET, POST, PUT, DELETE, etc.) are mapped directly to the methods in the class.
Automatic JSON/XML Conversion:

Methods in a @RestController return domain objects directly. Spring Boot automatically converts these objects into JSON or XML responses (based on the Accept header of the request) using Jackson or another configured message converter.
Simplification of Controller Code:

By annotating a class with @RestController, developers can streamline the code as they no longer need to add @ResponseBody to every request handling method.

Explain microstructure architecture

Microservices Architecture:
Microservices Architecture is an architectural style that structures an application as a collection of loosely coupled services. Each service is self-contained, independently deployable, and typically focuses on performing a single business function.

Benefits of Microservices Architecture:
Scalability: Individual microservices can be scaled independently based on demand.
Flexibility and Agility: Enables rapid development, deployment, and updates of services.
Improved Fault Isolation: Failures are contained within a single service, reducing impact on the overall system.
Technology Diversity: Allows for the use of different technologies and frameworks for different services.
Enhanced Maintainability: Smaller, focused services are easier to understand, modify, and maintain compared to monolithic applications.

Challenges:
Increased Complexity: Managing a distributed system introduces complexity in deployment, testing, monitoring, and debugging.
Data Management: Maintaining consistency and managing data across multiple services can be challenging.
Service Discovery and Communication: Ensuring reliable communication and discovering services dynamically in a distributed environment.
Operational Overhead: Requires robust infrastructure and operational practices to support and monitor microservices effectively.

Microservices architecture can be categorized into several types or patterns, each emphasizing different aspects of service decomposition, communication patterns, and deployment strategies. Here are some common types of microservices architecture patterns:

1. Monolithic Application Decomposition:

Description: In this approach, a monolithic application is decomposed into smaller services, but each service may still be relatively large and cohesive compared to other microservices architectures.
Characteristics:
- Services are typically larger and might handle multiple related functionalities.
- Often uses synchronous communication (e.g., RESTful APIs) between services.
- Deployment and scaling are often handled per service, but still may share some components (e.g., databases).

2. Layered Architecture:

Description: Services are organized into layers, where each layer represents a set of related functionalities or responsibilities.
Characteristics:
- Each layer may be implemented as one or more microservices.
- Communication between layers can be synchronous or asynchronous.
- Promotes separation of concerns and scalability within each layer.

3. Event-Driven Architecture:

Description: Services communicate through events (messages) asynchronously. Events represent state changes or significant actions within the system.
Characteristics:
- Services are decoupled and communicate through message brokers or event buses.
- Enables loose coupling and scalability by allowing services to react to events without direct dependencies.
- Supports eventual consistency and fault isolation.

4. API Gateway Pattern:

Description: An API Gateway acts as a single entry point for clients to interact with multiple microservices.
Characteristics:
- Provides a unified interface for clients, routing requests to appropriate microservices.
- May handle authentication, rate limiting, and request aggregation.
- Improves client-side performance and simplifies the client's view of the system architecture.

5. Service Mesh:

Description: A dedicated infrastructure layer for handling service-to-service communication, including load balancing, service discovery, encryption, and monitoring.
Characteristics:
- Enhances visibility, reliability, and security of microservices communication.
- Typically implemented using a sidecar proxy (e.g., Envoy) alongside each microservice instance.
- Allows centralized management of microservices communication policies.

6. Saga Pattern:

Description: Handles long-lived transactions that span multiple microservices, ensuring eventual consistency without distributed transactions.
Characteristics:
- Uses a series of local transactions (compensating actions) within each microservice.
- Coordination and orchestration are typically managed by a Saga orchestrator.
- Enables rollback and compensation in case of failures, ensuring data consistency across microservices.

7. Containerization and Orchestration:

Description: Microservices are deployed as lightweight, isolated containers (e.g., Docker) and orchestrated using platforms like Kubernetes.
Characteristics:
- Simplifies deployment, scaling, and management of microservices.
- Provides infrastructure automation and supports microservices resilience.
- Enables efficient resource utilization and scalability through container orchestration.

Conclusion:

Each type of microservices architecture pattern offers distinct advantages and is suitable for different scenarios based on scalability needs, communication requirements, and operational considerations. The choice of architecture pattern depends on factors such as application complexity, team expertise, scalability requirements, and operational goals.

Java prep for 3+ years

Mallikarjun H T — Tue, 18 Jun 2024 02:09:00 +0000

Tell me the difference between Method Overloading and Method
Overriding in Java. with example

Method Overloading:
Method Overloading refers to defining multiple methods in the same class with the same name but different parameters. The methods can differ in the number of parameters, type of parameters, or both.

Key Points:

Method overloading is achieved within the same class.
It is also known as compile-time polymorphism or static polymorphism.
The return type may or may not be different, but the parameters must differ in type, sequence, or number.

public class OverloadingExample {

    // Method with the same name but different parameters
    public int add(int a, int b) {
        return a + b;
    }

    // Overloaded method with different parameter types
    public double add(double a, double b) {
        return a + b;
    }

    // Overloaded method with different number of parameters
    public int add(int a, int b, int c) {
        return a + b + c;
    }

    // Overloaded method with different sequence of parameters
    public int add(int a, double b) {
        return (int) (a + b);
    }
}

Method Overriding:
Method Overriding occurs when a subclass provides a specific implementation of a method that is already provided by its superclass. The method in the subclass has the same name, parameter list, and return type as the method in the parent class.

Key Points:

Method overriding is achieved between a superclass and its subclass.
It is also known as runtime polymorphism or dynamic polymorphism.
The method signatures (name, parameters, return type) must be identical in both the superclass and subclass.

// Superclass
class Animal {
    public void makeSound() {
        System.out.println("Animal makes a sound");
    }
}

// Subclass overriding the makeSound() method
class Dog extends Animal {
    @Override
    public void makeSound() {
        System.out.println("Dog barks");
    }
}

public class Main {
    public static void main(String[] args) {
        Animal animal = new Animal();
        animal.makeSound(); // Output: Animal makes a sound

        Dog dog = new Dog();
        dog.makeSound(); // Output: Dog barks
    }
}

What do you mean by the class loader in Java?

It seems like you're summarizing the types of built-in class loaders in Java, and you've provided an overview of each type. Let's elaborate on each type of class loader based on your description:

1. Bootstrap Classloader

- **Description**: The Bootstrap Classloader is the first class loader that initiates when the JVM starts up. It is responsible for loading core Java classes from the bootstrap classpath, which typically includes the `rt.jar` file (or `classes.jar` in older JDK versions). These classes form the Java Standard Edition libraries.

- **Loaded Classes**: Classes loaded by the Bootstrap Classloader include fundamental Java classes like those in `java.lang`, `java.net`, `java.util`, `java.io`, `java.sql`, etc.

### 2. Extension Classloader

- **Description**: The Extension Classloader is the child class loader of the Bootstrap Classloader. It loads classes from the JDK's extension directories (`$JAVA_HOME/jre/lib/ext` by default). These directories contain optional extensions to the standard core Java classes.

- **Loaded Classes**: Classes loaded by the Extension Classloader typically include classes from JAR files located in `$JAVA_HOME/jre/lib/ext`.

### 3. System Application Classloader (Application Classloader)

- **Description**: The System Application Classloader, also known as the Application Classloader, is the child class loader of the Extension Classloader. It loads classes from the application classpath, which is set by the `CLASSPATH` environment variable or specified explicitly using the `-cp` or `-classpath` command-line option.

- **Loaded Classes**: Classes loaded by the Application Classloader include application-specific classes and third-party libraries that are included in the classpath.

### Class Loading Hierarchy Summary:

- **Bootstrap Classloader**: Loads core Java classes (`java.lang`, `java.util`, etc.) from the bootstrap classpath.
- **Extension Classloader**: Loads classes from extension directories (`$JAVA_HOME/jre/lib/ext`).
- **Application Classloader**: Loads classes from the application classpath (specified by `CLASSPATH` or `-cp` option).

### Why Understanding Class Loaders is Important:

- **Class Loading Flexibility**: Java's class loading mechanism allows applications to dynamically load classes at runtime, enabling features like plugin systems, dynamic class loading, and hot swapping.
- **Security and Isolation**: Class loaders provide a level of security and isolation, ensuring that classes from different sources (system libraries, extensions, applications) do not interfere with each other.
- **Customization**: Developers can create custom class loaders to load classes from non-standard sources or apply specific loading policies.

Overall, understanding the role of each built-in class loader in Java helps developers grasp how classes are loaded and managed within the JVM environment, facilitating effective application development and runtime behavior management.

What do you mean by inheritance in Java?

In Java, inheritance is a mechanism through which an object acquires all the properties and behavior of another class. It is usually used for Method Overriding and Code Reusability.

The concept of inheritance in Java is based on the fact that it creates new classes that are built upon the existing classes. Therefore, these methods and fields of the parent class can be reused if we inherit from an existing class. And also, we can add new methods and fields to our current class.

In Java, the inheritance is of five types-

Hybrid Inheritance
Hierarchical Inheritance
Single-level Inheritance
Multi-level Inheritance
Multiple Inheritance

// Superclass
class Animal {
    // Field
    protected String name;

    // Constructor
    public Animal(String name) {
        this.name = name;
    }

    // Method
    public void eat() {
        System.out.println(name + " is eating.");
    }
}

// Subclass inheriting from Animal
class Dog extends Animal {
    // Constructor
    public Dog(String name) {
        super(name); // Call to superclass constructor
    }

    // Additional method specific to Dog
    public void bark() {
        System.out.println(name + " is barking.");
    }
}

// Main class to demonstrate inheritance
public class Main {
    public static void main(String[] args) {
        // Create an instance of Dog
        Dog myDog = new Dog("Buddy");

        // Access inherited method from Animal
        myDog.eat(); // Output: Buddy is eating.

        // Access method specific to Dog
        myDog.bark(); // Output: Buddy is barking.
    }
}

Why can we not override the static method in Java?

The reasons why we cannot override the static method in Java are :

(a) The static method does not belong to the object level. Instead, it belongs to the class level. The object decides which method can be called in the method overriding.

(b) In the static method, which is the class level method, the type reference decides on which method is to be called without referring to the object. It concludes that the method that is called is determined at the compile time.

If any child class defines the static method with the same signature as the parent class, then the method in the child class hides the method in the parent class.

class SuperClass {
    public static void staticMethod() {
        System.out.println("Static method in SuperClass");
    }
}

class SubClass extends SuperClass {
    public static void staticMethod() {
        System.out.println("Static method in SubClass");
    }
}

public class Main {
    public static void main(String[] args) {
        SuperClass.staticMethod(); // Output: Static method in SuperClass
        SubClass.staticMethod();   // Output: Static method in SubClass
    }
}

Can you tell us what the Dynamic Method Dispatch is in Java?

Dynamic Method Dispatch in Java refers to the mechanism by which the correct method implementation is determined at runtime, based on the object type rather than the reference type. It enables polymorphic behavior in Java, allowing a subclass to provide a specific implementation of an inherited method.

class Animal {
    public void makeSound() {
        System.out.println("Animal makes a sound");
    }
}

class Dog extends Animal {
    @Override
    public void makeSound() {
        System.out.println("Dog barks");
    }
}

class Cat extends Animal {
    @Override
    public void makeSound() {
        System.out.println("Cat meows");
    }
}

public class Main {
    public static void main(String[] args) {
        Animal animal1 = new Dog(); // Dog object
        Animal animal2 = new Cat(); // Cat object

        animal1.makeSound(); // Output: Dog barks
        animal2.makeSound(); // Output: Cat meows
    }
}