I am trying to import csv files into a sqlite3 database which I'm writing using python. I'm fairly new to importing data this way and to sqlite3 in general.
The data files I've got tend to have an irregular format at the beginning, but the rows I want will always begin with a date field (dd/mm/yy) and time field (hh:mm:ss). Example file is below:
Hello I am a file, , , ,
I am a type of csv file, , , ,
Date, Time, ID number, Message
12/12/2012, 13:12:13, 1, Hello World
13/12/2012, 13:12:13, 2, Goodbye
So I want to just import the 4th and 5th rows (and all subsequent rows with a date), but skip the first 3 which describe the file and column headers.
So far I've got the code which reads in the data and then puts it into a table along with a unique identifier (logID), but at the moment I'm just reading from line 4 onwards - I'd like a more general solution for the case when a file doesn't have the 3 leading rows of blurb.
#code above here opens the database and creates a cursor
with open(file) as csvfile:
reader = csv.reader(csvfile, dialect='excel')
for row in reader:
if reader.line_num >= 4:
values = logID + str(row[0]) + str(row[1]) + str(row[2]) + str(row[3])
cursor.execute("INSERT INTO table VALUES(" +values+ ");")
Thanks in advance for your help!