My professional interests are cloud and big data technologies, machine learning, and software development. I like to read source code and research papers to understand how stuff works.
Pronoun: He
Can I get better upload performance if I connect to the oracle database?
Most likely, as you are anyway "connecting to the Oracle DB" to generate the CSV file. By reading directly from Oracle you save the CSV generation and parsing step. This step not only takes time but is also error-prone as all schema information is lost and all variables are converted to String.
It might depend on the load and frequency of these ETL jobs and data format.
Dumping a table allows you to decouple the extraction and insertion steps, which means extraction could be done by a serial job and insertion to the destination DB could be done in parallel. Granted this can also be accomplished by using an intermediate programming language but ETL tools are normally equipped at handling massive CSV.
Depending on how the data is it might not matter to have data type conversion in place.
For further actions, you may consider blocking this person and/or reporting abuse
We're a place where coders share, stay up-to-date and grow their careers.
Most likely, as you are anyway "connecting to the Oracle DB" to generate the CSV file. By reading directly from Oracle you save the CSV generation and parsing step. This step not only takes time but is also error-prone as all schema information is lost and all variables are converted to String.
It might depend on the load and frequency of these ETL jobs and data format.
Dumping a table allows you to decouple the extraction and insertion steps, which means extraction could be done by a serial job and insertion to the destination DB could be done in parallel. Granted this can also be accomplished by using an intermediate programming language but ETL tools are normally equipped at handling massive CSV.
Depending on how the data is it might not matter to have data type conversion in place.