I am making a utility in which I need to extract tweets for a user and store them in database. Twitter API only sends 20 tweets in one call and to retrieve more tweets, you need to page through the tweets 20 at a time. So instead of synchronously reading tweets from Twitter API and waiting to insert into DB, I want to start the database inserts asynchronously to optimize the process.
How can I achieve this with Python and Mysql?
Pseudo-code for this can be written as (not checked for syntax validity):-
def readTweets():
x=0
while true:
tweets= twitterAPI.getusertimeline(id='twitterUser',count=20,page=x)
#Need to know how to call the below function asynchronously
callDBSaveAsynchronously(tweets)
if len(tweets) < 20: break
x=x+1
def callDBSaveAsynchronously(tweets):
for tweet in tweets:
mysqldb.insertTweet(tweet)
Thanks in advance!
SELECT * FROM twitter WHERE from_user='twitterUser';– Mike T Jun 4 '13 at 23:21