current community

your communities

more stack exchange communities

company blog
Stack Exchange Inbox Reputation and Badges
tour help
stack overflow careers
Sign up ×
Stack Overflow is a community of 4.7 million programmers, just like you, helping each other. Join them; it only takes a minute:
up vote 1 down vote favorite

I'm a little confused about the amount of possibilities to import csv-data into a postgres database.

My issue: I have a csv-file with about 70 columns, but I only want to import specific columns of these 70.

I've tried the psycopg2.copy_from(), but when I specify my columns like this:

psycopg2.copy_from(csv_in, 'stamm_data', sep=';', columns = 'col_1', 'col_4', 'col_7')

I get: 

Error: extra data after last expected column

So, another solution is to work with dictionaries with the column-name as key, the data as value and then import the data with psycopg2.execute("INSERT INTO..") into the database.

I also could write my needed columns into a new csv-file and import it with psycopg2.copy_from().

Or I use psycopg2.execute("COPY ..") or psycopg2.copy_expert().

So, whats the most effective way for my issue?

share|improve this question
    
You need to pass an iterable ['col_1','col_4','col_7'] - and I do not think you can select individual columns like that (from the documentation): "The length and types should match the content of the file to read. If not specified, it is assumed that the entire table matches the file structure." but try with the iterable first. – Burhan Khalid Mar 5 at 8:57
    
I've tried that already and it doesn't work - I get the same Error as I mentioned above. When column-option isn't for picking up individual columns, then I don't get the sense behind this option. The documentation sound like this option is for picking up individual columns imho. – jwi Mar 5 at 9:19
    
I believe the columns option is for enumerating which columns in your already-created database table will receive the data imported from the file. For example, if you are importing this data into a table with 90 columns, you would need this option to specify which 70 are going to be populated. Since copy_from is a wrapper around the native PostgreSQL COPY functionality, it probably can't do anything short of importing the entirety of each row it encounters. – rchang Mar 5 at 11:10

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Browse other questions tagged or ask your own question.