Writing data flow to postgresql

Question

I know that by doing:

COPY test FROM '/path/to/csv/example.txt' DELIMITER ',' CSV;

I can import csv data to postgresql.

However, I do not have a static csv file. My csv file gets downloaded several times a day and it includes data which has previously been imported to the database. So, to get a consistent database I would have to leave out this old data.

My bestcase idea to realize this would be something like above. However, worstcase would be a java program manually checks each entry of the db with the csv file. Any recommendations for the implementation?

I really appreciate your answer!

You can COPY full file to temp table and then INSERT ... SELECT ... only new record to the actual table. — Ihor Romanchenko, Commented Jan 17, 2014 at 12:56
Sounds like you're looking for a bulk data merge, i.e. copy ignoring duplicates, or copy upsert. If so, see stackoverflow.com/q/13947327/398670, stackoverflow.com/q/17267417/398670 — Craig Ringer, Commented Jan 18, 2014 at 2:02

Parthi P · Accepted Answer · 2014-01-17 17:16:23Z

1

You can dump latest data to the temp table using COPY command and MERGE temp table with the live table.

If you are using JAVA program for execute COPY command, then try CopyManager API.

answered Jan 17, 2014 at 17:16

Parthi P

651 silver badge8 bronze badges

Add a comment |

Collectives™ on Stack Overflow

Writing data flow to postgresql

1 Answer 1

Your Answer

Linked

Hot Network Questions

Collectives™ on Stack Overflow

1 Answer 1

Your Answer

Sign up or log in

Post as a guest

Linked

Related