Stack Overflow is a community of 4.7 million programmers, just like you, helping each other.

Join them; it only takes a minute:

Sign up
Join the Stack Overflow community to:
  1. Ask programming questions
  2. Answer and help your peers
  3. Get recognized for your expertise

I am new to node js.

I have a csv file in my local system that I want to upload it local PostgreSQL Database using node js.

I am trying the following code:

var csv = require('csv-stream');
var request = require('request');
var fs = require('fs');

// All of these arguments are optional.
var options = {
    delimiter : '\t', // default is ,
    endLine : '\n', // default is \n,
// by default read the first line and use values found as columns 
   // columns : ['Settlement Ref No.', 'Order Type','Fulfilment Type','Seller SKU','wsn'],
    escapeChar : '"', // default is an empty string
    enclosedChar : '"' // default is an empty string
}

var csvStream = csv.createStream(options);
 fs.createReadStream('C:\\Users\\YAM\\Documents\\fk_starchi.csv').pipe(csvStream)
    .on('error',function(err){
        console.error(err);
    })
    .on('data',function(data){
        // outputs an object containing a set of key/value pair representing a line found in the csv file.
       // console.log(data);
    })
    .on('column',function(key,value){
        // outputs the column name associated with the value found
      // console.log('#' + key + ' = ' + value);
        console.log('# '   + value);

    })

Its reading data . now i want to import it on postgrsql database.

Where can I get a tutorial or any other help to do this.

share|improve this question
    
Why you need node if postgres has a built-in function? Referer: stackoverflow.com/questions/2987433/… – vanadium23 May 7 '15 at 7:51
    
I also have to create a file dialog, that will not create in postgresql – user3946530 May 7 '15 at 7:53
    
May be you can write query COPY zip_codes FROM '/path/to/csv/ZIP_CODES.txt' DELIMITER ',' CSV; without csv module? – vanadium23 May 7 '15 at 7:54
    
it should be http path mycsv.com/file.csv like that – Rajat Modi May 7 '15 at 7:59
    
@RajatModi if i need the local path then what changes should i do? – user3946530 May 7 '15 at 8:15

I understand you want to import this cvs file into Postgres.

There's two steps. Reading the file. Writing the data.

1) Reading the file you've done with csv-stream. I don't quite understand what the column event does, but it looks like the 'data' event is where to start. So add your code there.

2) Writing the data.

There's two routes for this:

a) Quick and dirty. In the 'data' event, craft the SQL using strings, then run them with a thin library like node-postgres.

var sql = 'INSERT INTO table VALUES (' data.this + ',' + data.that + ',' + data.theotherthing + ');';

Check out this example for a structure to start. You're already familiar with streams, so you'll just need to manage the callbacks.

You're csv-stream will produce SQL statements faster than postgress will handle them, so you could run into 1000's of simultaneous requests. You might want to + the query strings together in batches, and/or use through2 to query, wait, then query.

The reason NOT to do this is someone could put a SQL injection into the CSV and trash your database.

b) The smart way to do this (especially if dealing with unknown CSV's) is to use an ORM like sequelize.

There isn't a copy/paste and done. A good place to start is reading their homepage.

share|improve this answer
    
how i can run this for demo – user3946530 May 7 '15 at 10:01

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.