[ensembl-dev] Problem importing huge file into MySQL

[Toni Hermoso Pulido]

Hello,

I'm trying to import into a MySQL DB an ENSEMBL file (concretely, 
homo_sapiens_variation_66_37 allele.txt.gz ~ 20 GB) using MySQL LOAD 
DATA INFILE.
However, it always gets stuck when the index file is around 5 GB. At 
that time, the process status is 'Repair with keycache'.

After discovering this file:
ftp://ftp.ensembl.org/pub/ensembl-functgenomics/scripts/DAS/load_bed_source.pl 

I've been trying different values of myisam_sort_buffer_size and 
myisam_max_sort_file_size parameters with no luck.

MySQL version is 5.1.45.

Has anyone experienced problem like this?

Thanks in advance,

-- 
Toni Hermoso Pulido
Bioinformatician

Bioinformatics Core Facility
http://biocore.crg.cat
CRG - Centre de Regulació Genòmica (Room 439)
Parc de Recerca Biomèdica de Barcelona (PRBB)
Doctor Aiguader, 88
08003 Barcelona

toni.hermoso at crg.cat
Phone: +34 93 316 02 02