Hello Riccardo
First of all congratulations for your post!
I want to ask you something. I am importing data in my mongodb collection.Every file contains 105.000 rows and we have in total 11 files.So we have 1.2m rows
The code i used for that is this:
bulk_request=[]
for file in sorted_files:
df = process_file(file)
for row, item in df.iterrows():
data_dict = item.to_dict()
bulk_request.append(UpdateOne(
{"nsamples": {"$lt":12}},
{
"$push": {"samples": data_dict},
"$inc": {"nsamples": 1}
},
upsert=True
))
result = mycol1.bulk_write(bulk_request)
The problem is that this is very slow.Do you know how can i have faster insertion?
Thanks in advance!
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Hello Riccardo
First of all congratulations for your post!
I want to ask you something. I am importing data in my mongodb collection.Every file contains 105.000 rows and we have in total 11 files.So we have 1.2m rows
The code i used for that is this:
The problem is that this is very slow.Do you know how can i have faster insertion?
Thanks in advance!