[ensembl-dev] './ensembl_compara_75/member.frm' (errno: 24)

[Andy Yates]

Hi,

In the past we've seen this kind of behaviour linked to code which connects & disconnects from a database too often. MySQL traditionally assigns a connection to a linux process. Later versions of MySQL (5.6 enterprise), MariaDB & Percona Server have switched to using a pool of threads IIRC which does alleviate the problem.

Anyway going back to Ensembl. I've seen the issue appear because the disconnect_if_idle flag is turned on in a DBConnection object. Of course this maybe your intended behaviour as otherwise you are responsible for cleaning up the connections you create and filling up a MySQL server with idle connections. At any rate raising open-files-limit is a good way to alleviate this. To give you an idea our servers are normally set to 132,106.

Andy

------------
Andrew Yates - Ensembl Support Coordinator
European Bioinformatics Institute (EMBL-EBI)
European Molecular Biology Laboratory
Wellcome Trust Genome Campus
Hinxton
Cambridge CB10 1SD
Tel: +44-(0)1223-492538
Fax: +44-(0)1223-494468
http://www.ensembl.org/

On 27 May 2014, at 16:02, Ed Gray <gray_ed at hotmail.com> wrote:

> Good info Stuart.  We have very few users and connections.  It would seem to me though that the root problem might be not destroying an object properly eh?
>  
> Anyone encounter/fix problems with a long running perl script using the ensembl api that generates too many file handles?
>  
> From: Stuart.Watt at oicr.on.ca
> To: dev at ensembl.org
> Date: Tue, 27 May 2014 14:32:42 +0000
> Subject: Re: [ensembl-dev] './ensembl_compara_75/member.frm' (errno: 24)
> 
> I’ve hit that one before with MySQL running out of file handles. See: http://stackoverflow.com/questions/11390213/1016-cant-open-file-database-name-sql-38f-36aa-frm-errno-24. You can usually set the server variable open-files-limit (IIRC, by default in many configs, it’s set low). 
> 
> All the best
> Stuart
> 
> 
> On May 27, 2014, at 10:22 AM, Ed Gray <gray_ed at hotmail.com> wrote:
> 
>  
> strange error:
>  
> here is what I did:
>  
> my @homs = @{$gene->get_all_homologous_Genes};
> 
> DBD::mysql::st execute failed: Can't open file: './ensembl_compara_75/member.frm' (errno: 24) at /usr/local/share/perl/5.12.4/ensembl-compara/modules/Bio/EnsEMBL/Compara/DBSQL/BaseAdaptor.pm line 168.
> -------------------- EXCEPTION --------------------
> MSG: Detected an error whilst executing SQL 'SELECT m.member_id, m.source_name, m.stable_id, m.version, m.taxon_id, m.genome_db_id, m.description, m.chr_name, m.chr_start, m.chr_end, m.chr_strand, m.sequence_id, m.gene_member_id, m.canonical_member_id, m.display_label, hm.cigar_line, hm.perc_cov, hm.perc_id, hm.perc_pos FROM  (member m, homology_member hm)  WHERE hm.homology_id = ? AND m.member_id = hm.peptide_member_id ': DBD::mysql::st execute failed: Can't open file: './ensembl_compara_75/member.frm' (errno: 24) at /usr/local/share/perl/5.12.4/ensembl-compara/modules/Bio/EnsEMBL/Compara/DBSQL/BaseAdaptor.pm line 168.
> STACK Bio::EnsEMBL::Compara::DBSQL::BaseAdaptor::generic_fetch /usr/local/share/perl/5.12.4/ensembl-compara/modules/Bio/EnsEMBL/Compara/DBSQL/BaseAdaptor.pm:169
> STACK Bio::EnsEMBL::Compara::DBSQL::AlignedMemberAdaptor::fetch_all_by_Homology /usr/local/share/perl/5.12.4/ensembl-compara/modules/Bio/EnsEMBL/Compara/DBSQL/AlignedMemberAdaptor.pm:120
> STACK Bio::EnsEMBL::Compara::DBSQL::AlignedMemberAdaptor::fetch_all_by_AlignedMemberSet /usr/local/share/perl/5.12.4/ensembl-compara/modules/Bio/EnsEMBL/Compara/DBSQL/AlignedMemberAdaptor.pm:88
> STACK Bio::EnsEMBL::Compara::MemberSet::get_all_Members /usr/local/share/perl/5.12.4/ensembl-compara/modules/Bio/EnsEMBL/Compara/MemberSet.pm:443
> STACK Bio::EnsEMBL::Compara::MemberSet::get_all_GeneMembers /usr/local/share/perl/5.12.4/ensembl-compara/modules/Bio/EnsEMBL/Compara/MemberSet.pm:489
> STACK Bio::EnsEMBL::Gene::get_all_homologous_Genes /usr/local/share/perl/5.12.4/ensembl/modules/Bio/EnsEMBL/Gene.pm:857
> STACK main::process_homologs newget_gene_info.pl:423
> STACK toplevel newget_gene_info.pl:292
> Date (localtime)    = Tue May 27 12:46:41 2014
> Ensembl API version = 75
> ---------------------------------------------------
> 
> _______________________________________________
> Dev mailing list    Dev at ensembl.org
> Posting guidelines and subscribe/unsubscribe info: http://lists.ensembl.org/mailman/listinfo/dev
> Ensembl Blog: http://www.ensembl.info/