Database Administrators Stack Exchange is a question and answer site for database professionals who wish to improve their database skills and learn from others in the community. It's 100% free, no registration required.

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I have a very big CSV file with the following format:

TAG,TIME,VALUE

as an example row:

footag,2014-06-25 08:00:00.0,3400.0

I used to import it easily inside PostgreSQL 9.3 using the following: COPY datapoints FROM '/home/foo/my.csv' DELIMITER ',' CSV; where datapoints is a table with the three corresponding columns.

Now, the database into which I need to import has changed. It has two tables. Here are \d+ outputs:

                                          Table "public.tags"
 Column |       Type        |                     Modifiers                     | Storage  | Stats target | Description
--------+-------------------+---------------------------------------------------+----------+--------------+-------------
 id     | integer           | not null default nextval('tags_id_seq'::regclass) | plain    |              |
 tag    | character varying | not null                                          | extended |              |
Indexes:
    "tags_pkey" PRIMARY KEY, btree (id)
    "tags_tag_idx" btree (tag)
Referenced by:
    TABLE "tag_values" CONSTRAINT "tag_values_tag_fkey" FOREIGN KEY (tag_id) REFERENCES tags(id)
Has OIDs: no

                                Table "public.tag_values"
 Column |            Type             | Modifiers | Storage | Stats target | Description
--------+-----------------------------+-----------+---------+--------------+-------------
 tag_id | integer                     | not null  | plain   |              |
 time   | timestamp without time zone | not null  | plain   |              |
 value  | double precision            |           | plain   |              |
Indexes:
    "tag_values_tag_time_idx" btree (tag_id, "time")
Foreign-key constraints:
    "tag_values_tag_fkey" FOREIGN KEY (tag_id) REFERENCES tags(id)
Has OIDs: no

So essentially, the original table is split into two. As you see, there's a foreign key constraint now (between tags(id) and tag_valued(tag_id)) Also, tags(tag) are supposed to be distinct. What is the easiest way to ingest the same .CSV files into such structure?

Thanks!

share|improve this question
1  
Your best bet is probably to import it into a temp table then move it to the two new tables. – Neil McGuigan Oct 1 '14 at 0:41
    
That's what actually I ended up doing, thanks for the confirmation though =) I was wondering if there were a better way. – kolistivra Oct 1 '14 at 2:08
    
@NeilMcGuigan Are you interested in posting an answer? – dezso Feb 10 at 13:44

I find the most reliable way to import CSV data into PostgreSQL is to create a table just for the import (you can delete it after). Often there are wonky values in some columns, so I make all the column types text and then convert them on insert into the "real" table.

The \copy command in psql (different than copy) makes it easy to copy files to a remote server, and insert into ... select makes it easy to copy to another table.

share|improve this answer

The best way to import data from multiple tables is to use COPY:

COPY (write your query here) FROM '/home/foo/my.csv' DELIMITER ',' CSV;
share|improve this answer
2  
And this is what the OP used to use, until they started having two tables... – dezso Feb 10 at 13:43
    
This does not provide an answer to the question. To critique or request clarification from an author, leave a comment below their post - you can always comment on your own posts, and once you have sufficient reputation you will be able to comment on any post. - From Review – Julien Vavasseur Feb 10 at 14:04

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.