DEV Community

Cover image for Transfer Data from CSV files to Deta Bases + Alternative solution.
Fredy Somy
Fredy Somy

Posted on

Transfer Data from CSV files to Deta Bases + Alternative solution.

What are we going to do.

First of all , lets go through this in two phases.

  • Convert csv to json
  • Upload converted json to Deta base

Prerequisites

  • Create a free account in Deta
  • Create a new project (Note down the project key, we need it)

Lets start.

Imagine having a csv file as

name,age
fredy,34
god,34345345
mannu,34
Enter fullscreen mode Exit fullscreen mode

We want it like:

[{'name': 'fredy', 'age': '34'},{'name': 'god', 'age': '34345345'}
,{'name': 'maanu', 'age': '34'}
]
Enter fullscreen mode Exit fullscreen mode

Lets start by importing csv module

import csv
Enter fullscreen mode Exit fullscreen mode

Read the file contents:

arr=[]
with open (path) as csvFile:
    csvReader = csv.DictReader(csvFile)
    for csvRow in csvReader:
        arr.append(csvRow)
Enter fullscreen mode Exit fullscreen mode
  • We are making a list named arr for appending all the json extracted from csv.
csvReader = csv.DictReader(csvFile)
Enter fullscreen mode Exit fullscreen mode
  • This creates a csvreader object which maps each row to a dictionary.

  • We can now use the csvreader to extract the dictionary.

for csvRow in csvReader:
      arr.append(csvRow)
Enter fullscreen mode Exit fullscreen mode
  • The above loop takes each dictionary from csvreader object and append it to list arr,which i mentioned earlier.

  • We can now print the list arr to see the json data.

>> print(arr)
[{'name': 'fredy', 'age': '34'},{'name': 'god', 'age': '34345345'}
,{'name': 'maanu', 'age': '34'}
]
Enter fullscreen mode Exit fullscreen mode

Csv file is converted , lets upload it to Deta Base

  • Install deta module.
pip install deta
Enter fullscreen mode Exit fullscreen mode
  • Lets import it.
from deta import Deta
Enter fullscreen mode Exit fullscreen mode
  • Initialize with a Project Key( which we get while creating a project)
deta = Deta(projectkey)
database = deta.Base(db)
for each in arr:
    datafile.insert(each)
    print("Uploaded {}".format(each))

Enter fullscreen mode Exit fullscreen mode
  • Connect or create a database in Deta
database = deta.Base(db)
Enter fullscreen mode Exit fullscreen mode
for each in arr:
    datafile.insert(each)
    print("Uploaded {}".format(each))

Enter fullscreen mode Exit fullscreen mode
  • We are iterating through the list arr and datafile.insert(each) inserts each data into the database.

  • Finally we have this in Deta Base UI.

deta

Final Code

def csvtodeta(id,path,db):
    deta = Deta(id)
    datafile = deta.Base(db)
    print("Reading data from {}".format(path))
    arr=[]
    with open (path) as csvFile:
        csvReader = csv.DictReader(csvFile)
        for csvRow in csvReader:
           arr.append(csvRow)
    for each in arr:
        datafile.insert(each)
        print("Uploaded {}".format(each))
    print("succesfully uploaded {} data to {} Base".format(len(arr),db))

Enter fullscreen mode Exit fullscreen mode

Alternative Solution

I created a python CLI tool to do the same.

pip install csvtodeta
Enter fullscreen mode Exit fullscreen mode
  • With just one command the data of csv file is uploaded to DetaπŸŽ‰.
$ csvtodeta --id yourprojectkey --path path/to/csv.csv --db detabasename
Enter fullscreen mode Exit fullscreen mode
  • id is your project key from deta
  • Provide the path name of csv file.
  • Provide database name of the Base you want to create or connect.
  • Tada...πŸŽ‰πŸŽ‰ The data is uploaded.
$ csvtodeta --id 45345dhsgh3rjdf2ur34hhwf --path src/deta.csv --db detabasename
Reading data from src/deta.csv
Uploaded {'name': 'fredy', 'age': '34'}
Uploaded {'name': 'god', 'age': '34345345'}
Uploaded {'name': 'maanu', 'age': '34'}
succesfully uploaded 3 data to detabasename Base

Enter fullscreen mode Exit fullscreen mode

Github Repo

Thank you for reading this blog. If you have any doubts or if there is any wrong explanation please comment down here.

Top comments (0)