[ensembl-dev] skip lock tables for ensembl-compara
Francesco Lamanna
francesco.lamanna at gmail.com
Tue Nov 21 17:36:04 GMT 2017
Hi Mateus,
I have updated to e91 but know I get the following error:
+-------------+----------------------------+----------------+--------+---------+-----------+---------------------+-------+--------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+
| analysis_id | logic_name | log_message_id | job_id |
role_id | worker_id | when_logged | retry | status |
msg
| is_error |
+-------------+----------------------------+----------------+--------+---------+-----------+---------------------+-------+--------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+
| 9 | copy_ncbi_table | 1 | 5
| 3 | 3 | 2017-11-21 18:20:05 | 0 | WRITE_OUTPUT |
Successfully copied 1609518 'ncbi_taxa_node'
rows
| 0 |
| 9 | copy_ncbi_table | 2 | 6
| 4 | 4 | 2017-11-21 18:22:33 | 0 | WRITE_OUTPUT |
Successfully copied 2438889 'ncbi_taxa_name'
rows
| 0 |
| 18 | make_treebest_species_tree | 3 | 10
| 9 | 9 | 2017-11-21 18:29:29 | 0 | RUN | Can't
call method "build_leftright_indexing" on an undefined value at
/home/hd/hd_hd/hd_cc141/EnsEMBL/ensembl-compara/modules/Bio/EnsEMBL/Compara/RunnableDB/MakeSpeciesTree.pm
line 111. | 1 |
| 18 | make_treebest_species_tree | 4 | 10 |
12 | 12 | 2017-11-21 18:30:31 | 1 | RUN | Can't call
method "build_leftright_indexing" on an undefined value at
/home/hd/hd_hd/hd_cc141/EnsEMBL/ensembl-compara/modules/Bio/EnsEMBL/Compara/RunnableDB/MakeSpeciesTree.pm
line 111. | 1 |
| 18 | make_treebest_species_tree | 5 | 10 |
13 | 13 | 2017-11-21 18:31:37 | 2 | RUN | Can't call
method "build_leftright_indexing" on an undefined value at
/home/hd/hd_hd/hd_cc141/EnsEMBL/ensembl-compara/modules/Bio/EnsEMBL/Compara/RunnableDB/MakeSpeciesTree.pm
line 111. | 1 |
| 18 | make_treebest_species_tree | 6 | 10 |
14 | 14 | 2017-11-21 18:32:36 | 3 | RUN | Can't call
method "build_leftright_indexing" on an undefined value at
/home/hd/hd_hd/hd_cc141/EnsEMBL/ensembl-compara/modules/Bio/EnsEMBL/Compara/RunnableDB/MakeSpeciesTree.pm
line 111. | 1 |
+-------------+----------------------------+----------------+--------+---------+-----------+---------------------+-------+--------------+-------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------+----------+
it is still the "make_treebest_species_tree" analysis throwing a different
error.
Thanks,
Francesco.
2017-11-15 11:49 GMT+01:00 Mateus Patricio <mateus at ebi.ac.uk>:
> Hi Francesco,
>
> There has been some changes in that piece of code recently, which may have
> fixed this issue.
>
> Could you try updating to e91?
>
> Cheers,
>
> Mateus.
>
>
> On 15 Nov 2017, at 10:23, Francesco Lamanna <francesco.lamanna at gmail.com>
> wrote:
>
> Hi Mateus,
>
> I am using release/90 of the Compara pipeline.
>
> Cheers,
> Francesco.
>
> 2017-11-15 11:07 GMT+01:00 Mateus Patricio <mateus at ebi.ac.uk>:
>
>> Hi Francesco,
>>
>> Could you please confirm which checkout version of the compara code you
>> have?
>>
>> Cheers,
>>
>> Mateus.
>>
>> On 13 Nov 2017, at 10:50, Francesco Lamanna <francesco.lamanna at gmail.com>
>> wrote:
>>
>> Hi Mateus and Brandon,
>>
>> many thanks for your patch, it solved the issue.
>>
>> Unfortunately, I got stuck at the successive step with the following
>> error message:
>>
>> mysql> SELECT * FROM msg;
>> +-------------+----------------------------+----------------
>> +--------+---------+-----------+---------------------+------
>> -+--------------+-------------------------------------------
>> ------------------------------------------------------------
>> ------------------------------------------------------------
>> -------+----------+
>> | analysis_id | logic_name | log_message_id | job_id |
>> role_id | worker_id | when_logged | retry | status |
>> msg
>>
>> | is_error |
>> +-------------+----------------------------+----------------
>> +--------+---------+-----------+---------------------+------
>> -+--------------+-------------------------------------------
>> ------------------------------------------------------------
>> ------------------------------------------------------------
>> -------+----------+
>> | 9 | copy_ncbi_table | 1 | 5
>> | 3 | 3 | 2017-11-13 11:16:31 | 0 | WRITE_OUTPUT |
>> Successfully copied 1609518 'ncbi_taxa_node' rows
>>
>> | 0 |
>> | 9 | copy_ncbi_table | 2 | 6
>> | 4 | 4 | 2017-11-13 11:20:15 | 0 | WRITE_OUTPUT |
>> Successfully copied 2438889 'ncbi_taxa_name' rows
>>
>> | 0 |
>> | 18 | make_treebest_species_tree | 3 | 10
>> | 9 | 9 | 2017-11-13 11:25:29 | 0 | FETCH_INPUT | Can't
>> call method "get_all_subnodes" on an undefined value at
>> /home/hd/hd_hd/hd_cc141/EnsEMBL/ensembl-compara/modules/Bio/
>> EnsEMBL/Compara/Utils/SpeciesTree.pm line 150. | 1 |
>> | 18 | make_treebest_species_tree | 4 | 10
>> | 11 | 11 | 2017-11-13 11:26:29 | 1 | FETCH_INPUT | Can't
>> call method "get_all_subnodes" on an undefined value at
>> /home/hd/hd_hd/hd_cc141/EnsEMBL/ensembl-compara/modules/Bio/
>> EnsEMBL/Compara/Utils/SpeciesTree.pm line 150. | 1 |
>> | 18 | make_treebest_species_tree | 5 | 10
>> | 13 | 13 | 2017-11-13 11:27:31 | 2 | FETCH_INPUT | Can't
>> call method "get_all_subnodes" on an undefined value at
>> /home/hd/hd_hd/hd_cc141/EnsEMBL/ensembl-compara/modules/Bio/
>> EnsEMBL/Compara/Utils/SpeciesTree.pm line 150. | 1 |
>> | 18 | make_treebest_species_tree | 6 | 10
>> | 16 | 16 | 2017-11-13 11:28:32 | 3 | FETCH_INPUT | Can't
>> call method "get_all_subnodes" on an undefined value at
>> /home/hd/hd_hd/hd_cc141/EnsEMBL/ensembl-compara/modules/Bio/
>> EnsEMBL/Compara/Utils/SpeciesTree.pm line 150. | 1 |
>> +-------------+----------------------------+----------------
>> +--------+---------+-----------+---------------------+------
>> -+--------------+-------------------------------------------
>> ------------------------------------------------------------
>> ------------------------------------------------------------
>> -------+----------+
>> 6 rows in set (0.00 sec)
>>
>> Do you have any suggestion?
>>
>> Thanks,
>> Francesco
>>
>> 2017-11-09 17:25 GMT+01:00 Brandon Walts <bwalts at ebi.ac.uk>:
>>
>>> Hi Francesco
>>>
>>> We have just pushed a fix to the MySQLTransfer.pm runnable, so that it
>>> no longer locks tables by default. If you go to your eHive checkout and
>>> perform a git pull, you should get this fix.
>>>
>>> Users who wish to retain the table locking behavior can switch it on by
>>> passing a parameter 'lock_tables=1' to MySQLTransfer.pm
>>>
>>> Best
>>> -Brandon
>>>
>>> On 09/11/2017 16:20, Mateus Patricio wrote:
>>>
>>> Hi Francesco,
>>>
>>> So what’s happening here is that you are trying to lock a table in one
>>> of our servers.
>>>
>>> That can be fixed by telling mysqldump to avoid locking the table. (Like
>>> you initially suspected)
>>>
>>> You could do that directly on the eHive runnable
>>> "ehive/master/modules/Bio/EnsEMBL/Hive/RunnableDB/MySQLTransfer.pm”
>>>
>>> The dump command is constructed at:
>>> https://github.com/Ensembl/ensembl-hive/blob/6c4877d4eda1bf0
>>> 469c29bdbe58932671848cc20/modules/Bio/EnsEMBL/Hive/RunnableD
>>> B/MySQLTransfer.pm#L117
>>>
>>> I would suggest you to create a new entry to “mode_options”:
>>> https://github.com/Ensembl/ensembl-hive/blob/6c4877d4eda1bf0
>>> 469c29bdbe58932671848cc20/modules/Bio/EnsEMBL/Hive/RunnableD
>>> B/MySQLTransfer.pm#L112
>>>
>>> So that you could access directly from your pipeline config file, by
>>> setting something like:
>>>
>>> { -logic_name => 'copy_ncbi_table',
>>> -module => 'Bio::EnsEMBL::Hive::RunnableD
>>> B::MySQLTransfer',
>>> -parameters => {
>>> 'src_db_conn' => '#ncbi_db#',
>>> 'mode' => ’*SKIP_LOCK*',
>>> 'filter_cmd' => 'sed "s/ENGINE=MyISAM/ENGINE=InnoDB
>>> /"',
>>> },
>>> },
>>>
>>>
>>> Please let me know if that doesn’t work.
>>>
>>> Cheers,
>>>
>>> Mateus.
>>>
>>>
>>>
>>>
>>> On 9 Nov 2017, at 15:44, Francesco Lamanna <francesco.lamanna at gmail.com>
>>> wrote:
>>>
>>> Hi Mateus,
>>>
>>> thank you for your answer.
>>>
>>> I am using 'ncbi_db' => 'mysql://anonymous@ensembldb.e
>>> nsembl.org/ensembl_compara_90' to conncect to ncbi_db (I used the same
>>> value as in the NoMasterProteinTrees_conf.pm example).
>>>
>>> The failing analysis is copy_ncbi_table(9).
>>>
>>> Cheers,
>>> Francesco.
>>>
>>> 2017-11-09 16:29 GMT+01:00 Mateus Patricio <mateus at ebi.ac.uk>:
>>>
>>>> Hi Francesco,
>>>>
>>>> Looks like your pipeline is using the user anonymous to connect to the
>>>> database.
>>>>
>>>> I would suggest that you add the skip option within your runnable.
>>>>
>>>> Which analysis is actually failing?
>>>>
>>>> Cheers,
>>>>
>>>> Mateus.
>>>>
>>>>
>>>> On 8 Nov 2017, at 17:36, Francesco Lamanna <francesco.lamanna at gmail.com>
>>>> wrote:
>>>>
>>>> Hello,
>>>>
>>>> I am trying to apply the ProteinTrees pipeline to a set of Ensembl and
>>>> non-Ensembl genomes.
>>>>
>>>> When running the pipeline without a master database (via beekeeper.pl)
>>>> I get the following error:
>>>>
>>>> ensembl_compara_90 ncbi_taxa_node | sed "s/ENGINE=MyISAM/ENGINE=InnoDB/"
>>>> | mysql -h127.0.0.1 -P3306 -uroot -p$EHIVE_TMP_PASSWORD_0
>>>> hd_cc141_test_protein_trees_no_master_90' resulted in an error code=2
>>>> stderr is: mysql: [Warning] Using a password on the command line
>>>> interface can be insecure.
>>>> mysqldump: Got error: 1044: Access denied for user 'anonymous'@'%' to
>>>> database 'ensembl_compara_90' when doing LOCK TABLES
>>>>
>>>>
>>>> Is there a way to pass --skip-lock-tables to the pipeline (e.g. in the
>>>> conf file) in order to be able to dump the database?
>>>>
>>>> Thanks,
>>>> Francesco.
>>>>
>>>>
>>>> _______________________________________________
>>>> Dev mailing list Dev at ensembl.org
>>>> Posting guidelines and subscribe/unsubscribe info:
>>>> http://lists.ensembl.org/mailman/listinfo/dev
>>>> Ensembl Blog: http://www.ensembl.info/
>>>>
>>>>
>>>>
>>>> _______________________________________________
>>>> Dev mailing list Dev at ensembl.org
>>>> Posting guidelines and subscribe/unsubscribe info:
>>>> http://lists.ensembl.org/mailman/listinfo/dev
>>>> Ensembl Blog: http://www.ensembl.info/
>>>>
>>>>
>>> _______________________________________________
>>> Dev mailing list Dev at ensembl.org
>>> Posting guidelines and subscribe/unsubscribe info:
>>> http://lists.ensembl.org/mailman/listinfo/dev
>>> Ensembl Blog: http://www.ensembl.info/
>>>
>>>
>>>
>>>
>>> _______________________________________________
>>> Dev mailing list Dev at ensembl.org
>>> Posting guidelines and subscribe/unsubscribe info: http://lists.ensembl.org/mailman/listinfo/dev
>>> Ensembl Blog: http://www.ensembl.info/
>>>
>>>
>>>
>>> _______________________________________________
>>> Dev mailing list Dev at ensembl.org
>>> Posting guidelines and subscribe/unsubscribe info:
>>> http://lists.ensembl.org/mailman/listinfo/dev
>>> Ensembl Blog: http://www.ensembl.info/
>>>
>>>
>> _______________________________________________
>> Dev mailing list Dev at ensembl.org
>> Posting guidelines and subscribe/unsubscribe info:
>> http://lists.ensembl.org/mailman/listinfo/dev
>> Ensembl Blog: http://www.ensembl.info/
>>
>>
>>
>> _______________________________________________
>> Dev mailing list Dev at ensembl.org
>> Posting guidelines and subscribe/unsubscribe info:
>> http://lists.ensembl.org/mailman/listinfo/dev
>> Ensembl Blog: http://www.ensembl.info/
>>
>>
> _______________________________________________
> Dev mailing list Dev at ensembl.org
> Posting guidelines and subscribe/unsubscribe info:
> http://lists.ensembl.org/mailman/listinfo/dev
> Ensembl Blog: http://www.ensembl.info/
>
>
>
> _______________________________________________
> Dev mailing list Dev at ensembl.org
> Posting guidelines and subscribe/unsubscribe info:
> http://lists.ensembl.org/mailman/listinfo/dev
> Ensembl Blog: http://www.ensembl.info/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ensembl.org/pipermail/dev_ensembl.org/attachments/20171121/d2ac97a4/attachment.html>
More information about the Dev
mailing list