DEV Community

Mallikarjun H T
Mallikarjun H T

Posted on • Edited on

Bulk insert data to ES

Elastic Search Version 6.2 is used for this example

Elasticdump

take index mappings dump to json

/usr/bin/elasticdump \
  --type=mapping \
  --input=http://localhost:9200/index_mame \
  --output "/file/to/write/index_name.mapping.json"
Enter fullscreen mode Exit fullscreen mode

take index data dump to json

/usr/bin/elasticdump \
  --type=data \
  --concurrency=100 \
  --intervalCap=500 \
  --input=http://localhost:9200/index_mame \
  --output "/file/to/write/index_name.data.json"
Enter fullscreen mode Exit fullscreen mode

create index with mappings

/usr/bin/elasticdump \
 --type=mapping \
  --input "/path/to/maping-file.mapping.json"  \
  --output=http://localhost:9200/index_name
Enter fullscreen mode Exit fullscreen mode

restore data from other index

/usr/bin/elasticdump \
  --type=data \
  --concurrency=100 \
  --intervalCap=500 \
  --input "/file/to/data-dump.data.json"  \
  --output=http://localhost:9200/index_name
Enter fullscreen mode Exit fullscreen mode

_source only backup

/usr/bin/elasticdump \
  --type=data \
  --concurrency=100 \
  --intervalCap=500 \
  --sourceOnly=true \
  --input=http://localhost:9200/index_name \
  --output "/file/to/data-dump.data.json"
Enter fullscreen mode Exit fullscreen mode

_bulk API

curl -XPOST localhost:9200/index_name/Document_Type/_bulk -H 'Content-Type: application/json' --data-binary @/path/to/file
Enter fullscreen mode Exit fullscreen mode

Image of Timescale

🚀 pgai Vectorizer: SQLAlchemy and LiteLLM Make Vector Search Simple

We built pgai Vectorizer to simplify embedding management for AI applications—without needing a separate database or complex infrastructure. Since launch, developers have created over 3,000 vectorizers on Timescale Cloud, with many more self-hosted.

Read full post →

Top comments (0)

Postmark Image

Speedy emails, satisfied customers

Are delayed transactional emails costing you user satisfaction? Postmark delivers your emails almost instantly, keeping your customers happy and connected.

Sign up