Tell me more ×
Stack Overflow is a question and answer site for professional and enthusiast programmers. It's 100% free, no registration required.

I'm using scrapy to scrape a webpage which I then add to a postgres database. The first INSERT statements works fine, and I can select the items from the database. The second one seems to insert data but all the fields are blank

           date            | count 
---------------------------+-------
 04/2013                   | 
 03/2013                   | 
 02/2013                   | 

Here is my code:

#Database init
    self.conn = psycopg2.connect("dbname='dataproject' user='xxxx' host='localhost' password='xxxxxx'")
    self.cursor = self.conn.cursor()    

    #CSV files
    self.DatavisItemCsv = csv.writer(open('DatavisTable.csv', 'wb'))
    self.DatavisItemCsv.writerow(['dates', 'counts'])

def process_item(self, item, spider):
    self.DatavisItemCsv.writerow([item['dates'], item['counts']])

    date_list = item['dates']
    count_list = item['counts']

    for s in date_list:
        self.cursor.execute('INSERT INTO ufo_info(date) VALUES (%s);', [s])

    for c in count_list:
        self.cursor.execute('INSERT INTO ufo_info(count) VALUES (%s);', [c])

    self.conn.commit()

Does this have anything to do with my for loops? data race?

share|improve this question

1 Answer

up vote 1 down vote accepted
for s, c in zip(date_list, count_list):
    self.cursor.execute(
        'INSERT INTO ufo_info(date, count) VALUES (%s, %s);'
        , (s, c)
    )
share|improve this answer
very smooth, than you sir! – enkitosh May 8 at 15:24

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.