DEV Community: Kiran (AK) Adapa

Refresh Your Neon Cloud Database from Local Postgres (Without the Data API)

Kiran (AK) Adapa — Sat, 07 Feb 2026 16:46:58 +0000

If you need to load your Neon Postgresql database with data, then is simple and you can make it easy for yourself on one of two ways.

Here is a situation: You’ve been iterating on your app against a local PostgreSQL database and now you want to push that data up to your Neon cloud instance. Your first thought might be: “Neon has a Data API—I’ll use that.” For a one-off or periodic bulk refresh, there’s a simpler and faster way.

Why Skip the Data API for This?

Neon’s Data API is great for application-driven CRUD over HTTP—think frontends or serverless functions talking to the database with optional JWT auth and Row-Level Security. For that, it’s the right tool.

For syncing an entire table (or several) from local to cloud, you’d end up reading every row locally, then sending them over the wire via many HTTP requests. You’d also need to handle auth, batching, and rate limits. That’s a lot of moving parts for a job that’s really “copy data from A to B.”

Neon speaks standard PostgreSQL. You can use normal connection strings and standard tooling. No Data API required for bulk refresh.

Two Better Options

Option 1: pg_dump + psql

Export from local (data-only, if the schema already exists in Neon):

pg_dump -h localhost -U your_user -d your_db -t your_table --data-only -F p -f data.sql

Then import into Neon using your Neon connection string:

psql "postgresql://user:password@your-project.neon.tech/neondb?sslmode=require" -f data.sql

Truncate the target table in Neon first if you’re replacing data to avoid duplicate-key errors. For multiple tables with foreign keys, export in dependency order or adjust the dump/restore sequence.

Option 2: A small script with two connections

Use your language of choice (e.g. Python with psycopg2) to open two connections: one to local Postgres, one to Neon. Read from the local table(s), truncate the target table(s) in Neon, then bulk-insert (e.g. execute_values or COPY). No HTTP layer, no JWT—just two Postgres connections and a straightforward copy. This approach is easy to rerun and fits well into scripts or small CLI tools.

Option 2 is what I use. There are times that I need to refresh the schema as well as load the data. I have a generic script that I use from time to time. If the schema changes or the data refresh is required, I point my friendly coding agent to it with a few instructions and my local .env and viola it is all taken care of!

TL;DR

For bulk refresh from local Postgres to Neon, use pg_dump/psql or a script with two DB connections. Reserve the Data API for runtime CRUD from your app. You’ll get the job done with less setup and better performance.

Hope this is helpful. Let me know on how you have used Neon Data API for your projects.

Note taking using CLI. I like to stay in the terminal or at the command line. I use `jrnl` now. With genai powered coding tools galore, wanted to build something that would better suit my needs. Lets see how it goes. I will write in here with my updates.

Kiran (AK) Adapa — Fri, 02 Jan 2026 23:30:28 +0000

Slide decks with Markdown

Kiran (AK) Adapa — Tue, 18 Nov 2025 13:04:37 +0000

If you are a developer or developer minded and creating presentations or slides with Powerpoint or Google Slides, then give Marp.app a try. Once you use it, you will never go back.

Write and version all your presentations in git!

If you love writing in Markdown, Marp.app lets you create beautiful slide decks quickly and easily using Markdown syntax. Here's how to get started and see your changes in real-time:

Write Your Slides in Markdown
Use --- (three dashes) to separate slides. Marp supports all common Markdown features like headers, lists, images, and code blocks. Add a YAML frontmatter at the top with marp: true to enable slide rendering, plus options like theme and pagination.

Preview Your Slides Instantly
Use the Marp CLI or Visual Studio Code Marp extension for live preview. Run marp -w yourslides.md with the CLI to open a localhost preview in your browser that updates as you edit your Markdown.

Apply Themes and Custom Styles
Marp comes with built-in themes like default, gaia, and uncover. You can set the theme in the frontmatter (theme: uncover). For custom styling, Marp supports adding CSS styles directly in your Markdown or creating your own theme.

Export to Multiple Formats
When ready, export your slides to PDF, HTML, or PowerPoint with one command or click, making it easy to share or present.

Marp.app turns your Markdown documents into presentation-ready slides with minimal fuss, perfect for developers who prefer editing text over slide design tools!

Try it out: create a .md file, add --- between slides, open in VS Code with Marp extension or use the CLI, and start presenting from Markdown!

Great Video on how to work with Marp by Doug Mercer

https://www.youtube.com/watch?v=EzQ-p41wNEE

Excalidraw - Browser based app for hand drawn like diagrams

Kiran (AK) Adapa — Sat, 01 Nov 2025 19:46:23 +0000

Online Whiteboard made simple. That's what Excalidraw promotes its app as and it is true. Excalidraw is a great app to create and share diagrams for your next project.

I use it locally on my Mac and love its simplicity.

Excalidraw positions itself as a "simple online whiteboard," and its architecture reflects that philosophy. Built with React and TypeScript, Excalidraw provides a minimal, collaborative environment for diagramming—ideal for technical teams, architects, and anyone who needs to quickly visualize system designs, workflows, or data models.

As an open-source project, Excalidraw can be deployed locally, giving teams full control over data privacy and customization. Here’s how to set it up for local use:

Clone the repository
Execute git clone https://github.com/excalidraw/excalidraw.git to download the source code.

Install dependencies
Navigate to the project directory and run npm install to install all required Node.js packages.

Launch the development server
Start the app with npm start. By default, Excalidraw will be available at http://localhost:3000.

Begin diagramming and sharing
Use the web interface to create diagrams, export them in multiple formats.

Right out of the base installation, you can even use Mermaid (https://mermaid.js.org/) to use text to draw diagrams.

How to Push Changes from a Cloned GitHub Repo to Your Own Repository

Kiran (AK) Adapa — Tue, 08 Jul 2025 13:15:58 +0000

You starred a great github (open source) project. You love this project and wanted to work on it on your own.

You clone that project from github to your local.
Now, what?

Make a brand new github repository under your own account and start using that repo.

For many, it is a vary basic activity and requires just a couple of steps. But, how do you do it?

When you clone a repository from GitHub, the remote named origin points to the source repository. If you want to make changes and push them to your own GitHub repository (not the original), follow these steps

Create a new repo on GitHub.
Change the remote URL in your local clone.
Push your changes to your new repo.
Confirm everything on GitHub.

That second step is the key step.

When you clone a repo, the project's git remote is still pointed to the

$ git remote -v

origin  https://github.com/open-source-org/great-project.git (fetch)
origin  https://github.com/open-source-org/great-project.git (push)

You update the origin to your newly created github repository url!

$ git remote set-url origin https://github.com/your-gh-profile/great-project.git

Now, if you display the git remote using the following command
$ git remote -v

You see this...

origin  https://github.com/your-gh-profile/great-project.git (fetch)
origin  https://github.com/your-gh-profile/great-project.git (push)

That's it!

From now on, whenever you make changes and push changes from your local these updates/additions go to your github repository.

Reference(s):

https://docs.github.com/en/get-started/git-basics/managing-remote-repositories

ER Diagrams for your Database

Kiran (AK) Adapa — Sat, 07 Jun 2025 15:19:10 +0000

DbSchema is a good tool to generate ER Diagrams for your database

https://dbschema.com/

A screenshot of Table Properties within the DbSchema

Working with Parquet files

Kiran (AK) Adapa — Sat, 05 Apr 2025 17:11:31 +0000

Parquet files offer significant advantages over traditional formats like CSV or JSON. This is more relevant in analytical workloads and processing.

Tools like parquet-tools and DuckDB make it easy to create, manipulate, and query these files.

parquet-tools https://github.com/hangxie/parquet-tools
DuckDB https://duckdb.org/docs/stable/data/parquet/overview

1) parquet-tool
display data to terminal in json (default) or jsonl, or csv

parquet-tools cat data_file.parquet | jq .

display in jsonl only two lines/records

parquet-tools cat --format jsonl --limit 2 data_file.parquet

Get the meta data about the parquet file

parquet-tools meta data_file.parquet

**2) DuckDB

DuckDB is an embedded SQL database that supports reading and writing Parquet files.

Example:

Generate a Parquet file:

COPY (SELECT 'example' AS col1) TO 'data_file.parquet' (FORMAT 'parquet');

Read a Parquet file:

SELECT * FROM read_parquet('data_file.parquet');

Lately, I have been using DuckDB for most of my analytics (dealing with Gigabytes of data) and it can handle both local and cloud-based files efficiently.

What is the Parquet file format?

Read about the Apache project's Overview/Motivation page https://parquet.apache.org/docs/overview/motivation/ and the project Documentation

Parquet is built to support very efficient compression and encoding schemes. Multiple projects have demonstrated the performance impact of applying the right compression and encoding scheme to the data. Parquet allows compression schemes to be specified on a per-column level, and is future-proofed to allow adding more encodings as they are invented and implemented.

For a good easy reading, go to this great blog article:https://blog.matthewrathbone.com/2019/12/20/parquet-or-bust.html

DevOps with AWS CodePipeline

Kiran (AK) Adapa — Fri, 14 Mar 2025 01:34:23 +0000

Automating DevOps with GitLab and AWS CodePipeline Integration

As a developer or DevOps engineer, streamlining your workflow is critical for efficient delivery of high-quality software. Integrate GitLab/Github repositories with AWS CodePipeline to get a robust solution for automating the CI/CD (Continuous Integration/Continuous Deployment) process. I use this setup for my DevOps workflows and highly recommend it to others looking to simplify their software development lifecycle.

My latest illustration of AWS CodePipeline Stages:

A Visual representation AWS CodePipeline of the CI/CD process, divided into three key stages: Source, Build, and Deploy.

One of the most powerful features of this integration is its ability to automatically trigger pipeline actions whenever code changes are pushed to GitLab. I configure a webhook between GitLab and AWS CodePipeline, and any new commit to the 'develop'/'release' branch in the repository initiates the pipeline workflow. Super convenient and increases your iteration speeds.

Get the benefits of automation and scalability using GitLab with AWS CodePipeline for DevOps.

I’ve found this setup invaluable for managing backend repositories efficiently in my DevOps workflows. If you’re looking to simplify your CI/CD process while maintaining high reliability, integrating GitLab with AWS CodePipeline is an excellent choice.

By adopting this approach, you can focus more on writing great code while letting automation handle the heavy lifting of building and deploying your applications.

How to Sync two GIT repositories?

Kiran (AK) Adapa — Tue, 04 Mar 2025 12:43:50 +0000

Steps for synching one Github repository with another Github repository using your local Macbook

Before you start…

Lets say, on your local macbook, you have a folder called “CODE”.
You already have the first or source repo (ak-repo1) on your local (on a feature branch — ‘feat/sync-1’ for the below example)
You don’t have a second or target repo created yet

Step 0) Create a Target (blank repository)

Create a new repository (ak-repo2) on Github
In this step, this ak-repo2 can stay as a blank repo

Step 1) Clone target repository to your local

Clone ak-repo2 from github to your local CODE folder
git clone https://github.com/akaak/ak-repo2.git

Verify the ‘remote’ for this target repository

CODE/ak-repo2 - <main> $ git remote -v
origin  https://github.com/akaak/ak-repo2.git (fetch)
origin  https://github.com/akaak/ak-repo2.git (push)

Step 2) Set Repo2 (target) as “remote” for Repo1 (source) on your local

From Repo1 (ak-repo1), set remote for repo2
CODE/ak-repo1 - <main> $ git remote add ak-repo2 https://github.com/akaak/ak-repo2.git

Verify the remotes on your Source repo

CODE/ak-repo1 - <feat/sync-1> $ git remote -v
ak-repo2    https://github.com/akaak/ak-repo2.git (fetch)
ak-repo2    https://github.com/akaak/ak-repo2.git (push)
origin  https://github.com/akaak/ak-repo1.git (fetch)
origin  https://github.com/akaak/ak-repo1.git (push)
CODE/ak-repo1 - <feat/sync-1> $

As you can see, there are two remotes on this source (ak-repo1). One for the newly added ‘ak-repo2’ and other other for ‘origin’

Step 3) Push from repo1 to repo2

From the Repo1 execute the command “git push ” to push to Repo 2

While on the branch ‘feat/sync-1’ of repo1, push repo1 branch contents to repo2

CODE/ak-repo1 - <feat/sync-1> $ git push ak-repo2 feat/sync-1

Here the repo2 name of ak-repo2 is recognized by this ‘git push’ command and pushes the code to github’s repo2. repo2 is set up as remote in Step 2 above.

the above command gives an output similar to the one given below…

CODE/ak-repo1 - <feat/sync-1> $ git push ak-repo2 feat/sync-1
Enumerating objects: 213, done.
Counting objects: 100% (213/213), done.
Delta compression using up to 8 threads
Compressing objects: 100% (155/155), done.
Writing objects: 100% (210/210), 365.70 KiB | 121.90 MiB/s, done.
Total 210 (delta 38), reused 210 (delta 38), pack-reused 0
remote: Resolving deltas: 100% (38/38), done.
remote:
remote: Create a pull request for 'feat/sync-1' on GitHub by visiting:
remote:      https://github.com/akaak/ak-repo2/pull/new/feat/sync-1
remote:
To https://github.com/akaak/ak-repo2.git
# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.

Step 4) Verify Repo 2 on Github

Go to Repo2 on Github and you should see the feature ‘branch’ feat/sync-1 show up as a new branch.

Step 5) Refresh (Fetch) Repo2 on your local

Go to your Repo2 folder on your local

CODE/ak-repo2 - <main> $ git fetch
remote: Enumerating objects: 214, done.
remote: Counting objects: 100% (214/214), done.
remote: Compressing objects: 100% (156/156), done.
remote: Total 211 (delta 38), reused 210 (delta 38), pack-reused 0 (from 0)
Receiving objects: 100% (211/211), 366.55 KiB | 4.17 MiB/s, done.
Resolving deltas: 100% (38/38), done.
From https://github.com/akaak/ak-repo2
   1f31f5b..592155f  main        -> origin/main
 * [new branch]      feat/sync-1 -> origin/feat/sync-1
CODE/ak-repo2 - <main>

Switch to the feature branch on your local
You can switch to the pushed feature branch on your local:

CODE/ak-repo2 - <main> $ git checkout feat/sync-1
branch 'feat/sync-1' set up to track 'origin/feat/sync-1'.
Switched to a new branch 'feat/sync-1'
CODE/ak-repo2 - <feat/sync-1> $

You may also use ‘git log’ command on to check and compare the git log on both the repositories to ensure that they both look the same.

CODE/ak-repo1 - <feat/sync-3> $ git log --oneline

That’s it. Now, both Repo1 and Repo2 are synched both at Github and on your Local!

Work with data files from your terminal - visidata

Kiran (AK) Adapa — Thu, 27 Feb 2025 01:23:26 +0000

If you are working with data files and want to wrangle data then you should add 'visidata' to your arsenal.

I have been using it for sometime and really love this tool.

from https://www.visidata.org/ website:

Data exploration at your fingertips.

VisiData is an interactive multitool for tabular data. It combines the clarity of a spreadsheet, the efficiency of the terminal, and the power of Python, into a lightweight utility which can handle millions of rows with ease.

Javascript projects and optimize space on your laptop

Kiran (AK) Adapa — Mon, 24 Feb 2025 02:22:29 +0000

(last updated Apr 5, 2025)

If you are a javascript/nodejs developer, then changes are node_modules from your projects is going out of control.

Get your space back on your macbook or to better manage space consumed by "node_modules" for various Node.js and React.js applications, you can try one of two things given here:

Remove Unused node_modules

Use npkill

This tool allows you to easily identify and remove unnecessary node_modules folders.

   npx npkill

This command will go through all the projects/directories and lists the "Releasable Space" in GB. It lists all node_modules directories and their sizes, allowing you to delete them selectively. You may use the up/down arrow on your keyboard to select a project and just hit the 'space bar' on the select that project's node_modules be cleaned up. All the folders that are deleted are marked and the "Space saved" in GB is updated.

See the screenshot here:

Use a shell script:

Create a script to automatically delete node_modules folders in inactive projects.

   #!/bin/bash
   find . -name "node_modules" -type d -mtime +30 -exec rm -rf {} +

This script removes node_modules folders that haven't been modified in the last 30 days.

Useful too to work with your JSON files - jq

Kiran (AK) Adapa — Thu, 26 Dec 2024 16:42:00 +0000

If you are working with JSON files then jq could be a very valuable tool. jq has proven to be a valuable tool for JSON processing. I've integrated it into several projects to streamline API response handling and data transformation tasks.
In our CI/CD pipelines, jq helps automate configuration updates across environments. If you have JSON-formatted log files, then you can greatly benefit from using 'jq' to search those log files.

jq has its own query language and it is quite powerful. Mastering jq's query language takes some time. But, with some basic queries you can get a lot using this tool and significantly improves your productivity when working with JSON data. I have been using it and added it to my toolkit, complementing other command-line utilities for data manipulation and analysis.

"jq is a lightweight and flexible command-line JSON processor" from the jq https://stedolan.github.io/jq/

References:

[1] jq manual
https://jqlang.github.io/jq/manual/

[2] Reference 2
https://www.example.com