Since it is a MyISAM table you are loading, please be aware of three(3) aspects of bulk loading:
ASPECT #1 : Bulk Insert Buffer
What is the bulk insert buffer? According to the MySQL Documentation
MyISAM uses a special tree-like cache to make bulk inserts faster for INSERT ... SELECT, INSERT ... VALUES (...), (...), ..., and LOAD DATA INFILE when adding data to nonempty tables. This variable limits the size of the cache tree in bytes per thread. Setting it to 0 disables this optimization. The default value is 8MB.
In light of this, you need to set it fairly high. I would go with 256M just to be conservative, but you could probably go with 1G.
ASPECT #2 : Disabling/Enabling Indexes
The MyISAM Storage Engine allows you to do the following:
ALTER TABLE mytable DISABLE KEYS;
ALTER TABLE mytable ENABLE KEYS;
What do they ?
ALTER TABLE mytable DISABLE KEYS;
disables all secondary non-unique indexes from receiving updates. This dis not affect PRIMARY KEY and UNIQUE KEY indexes.
ALTER TABLE mytable ENABLE KEYS;
does two things
- locates all secondary non-unique indexes that have not been updated and performs a linear rebuild of all such indexes
- enables all secondary non-unique indexes to resume receiving updates
Note how a mysqldump prepares a table for reload with these two commands:
--
-- Table structure for table `A`
--
DROP TABLE IF EXISTS `A`;
SET @saved_cs_client = @@character_set_client;
SET character_set_client = utf8;
CREATE TABLE `A` (
`DATE` date NOT NULL default '0000-00-00',
`OPEN` float default NULL,
`HIGH` float default NULL,
`LOW` float default NULL,
`CLOSE` float default NULL,
`VOLUME` int(11) default NULL
) ENGINE=MyISAM DEFAULT CHARSET=latin1;
SET character_set_client = @saved_cs_client;
--
-- Dumping data for table `A`
--
LOCK TABLES `A` WRITE;
/*!40000 ALTER TABLE `A` DISABLE KEYS */;
INSERT INTO `A` VALUES ('
.
.
.
/*!40000 ALTER TABLE `A` ENABLE KEYS */;
UNLOCK TABLES;
ASPECT #3 : Increase Sort Buffer Size
The MyISAM key cache comes into play when the sort buffer size is not large enough to run ENABLE KEYS. When this happens, the processlist will show Repair by keycache.
, which show that ENABLE KEYS
will take much longer (believe me, you do not want that). Therefore, increase the sort_buffer_size
to accommodate. If Repair by keycache
occurs no matter what, you will have to increase key_buffer_size
.
EPILOGUE
In your particular case, you should be doing the following:
ALTER TABLE yourtable DISABLE KEYS;
LOAD DATA INFILE ... INTO yourtable ... ;
ALTER TABLE yourtable ENABLE KEYS;
Give it a Try !!!