[ensembl-dev] ensembl-compara pipeline problems

江文恺 jiangwenkai at mail.kib.ac.cn
Mon Oct 25 16:32:05 BST 2010


Hi,
The problem is solved by change codes in CreateAlignmentChainsJobs.pm
By using fetch_by_logic_name('AlignmentChains'), the pipeline continues, 
possibly this feature is unique to 2x alignments? 


202 sub createAlignmentChainsJobs sub createAlignmentChainsJobs
203 { {
204   my $self = shift;   my $self = shift;
205    #print "CREATING ALIGNMENT CHAIN JOBS \n" ;
206   #my $analysis = $self->db->get_AnalysisAdaptor->fetch_by_logic_name('AlignmentChains');   my $analysis = $self->db->get_AnalysisAdaptor->fetch_by_logic_name('AlignmentChains');
207   my $analysis = $self->db->get_AnalysisAdaptor->fetch_by_logic_name($self->{'logic_name'});   # SMJS Reverted recent change for 2x my $analysis = $self->db->get_AnalysisAdaptor->fetch_by_logic_name($self->{'logic_name'});
208  
209   my (%qy_dna_hash, %tg_dna_hash);   my (%qy_dna_hash, %tg_dna_hash);
-----原始邮件-----
发件人: "江文恺" <jiangwenkai at mail.kib.ac.cn>
发送时间: 2010年10月25日 星期一
收件人: dev at ensembl.org
抄送:
主题: Re: [ensembl-dev] ensembl-compara pipeline problems

Hi,


The code is check out as follows:
ensembl-compara HEAD
ensembl-hive HEAD
ensembl-pipeline HEAD
ensembl-analysis HEAD
ensembl branch59




-----------------------------------------------

Hi Wenkai,

which checkout of the ensembl-compara code are you using ? Are you  
running on the head code or
did you specify a specific branch to be checked out ?

Regards,
                    Jan

On 25 Oct 2010, at 10:20, 江文恺 wrote:

> Hi, all,
>
> I have try to align 2 genomes under ensembl-compara pipeline, which  
> use the guide of 2x genome alignment,
>
> I successfully run several alignments, but the step  
> CreateAlignmentChainsJobs  (16) Fails,
>
>
>
> The output for this is:
>
> SubmitGenome               ( 1)        DONE 0:cpum job(0/2 run:0  
> fail:0 23ms) worker[0/-1] (sync'd 1 sec ago)
>
> ChunkAndGroupDna           ( 2)        DONE 0:cpum job(0/2 run:0  
> fail:0 9922ms) worker[0/-1] (sync'd 1 sec ago)
>
> StoreSequence              ( 3)        DONE 0:cpum job(0/127 run:0  
> fail:0 98747ms) worker[0/100] (sync'd 1 sec ago)
>
> CreatePairAlignerJobs      ( 4)        DONE 0:cpum job(0/2 run:0  
> fail:0 3954ms) worker[0/1] (sync'd 1 sec ago)
>
> BlastZ-29e37998            ( 5)        DONE 0:cpum job(0/3432 run:0  
> fail:0 438961ms) worker[0/600] (sync'd 1 sec ago)
>
> DumpDnaForBlastZ-29e37998  ( 6)        DONE 0:cpum job(0/1 run:0  
> fail:0 30734ms) worker[0/1] (sync'd 1 sec ago)
>
> UpdateMaxAlignmentLengthBeforeFD( 7)        DONE 0:cpum job(0/1 run: 
> 0 fail:0 865113ms) worker[0/1] (sync'd 1 sec ago)
>
> QueryFilterDuplicates-29e37998( 8)        DONE 0:cpum job(0/15858  
> run:0 fail:0 94187ms) worker[0/50] (sync'd 1 sec ago)
>
> UpdateMaxAlignmentLengthAfterFD( 9)        DONE 0:cpum job(0/1 run:0  
> fail:0 339076ms) worker[0/1] (sync'd 1 sec ago)
>
> CreateFilterDuplicatesJobs (10)        DONE 0:cpum job(0/4 run:0  
> fail:0 7680ms) worker[0/600] (sync'd 1 sec ago)
>
> TargetFilterDuplicates-29e37998(11)        DONE 0:cpum job(0/16 run: 
> 0 fail:0 572916ms) worker[0/200] (sync'd 1 sec ago)
>
> BlastZ-2782424c            (12)        DONE 0:cpum job(0/3432 run:0  
> fail:0 629626ms) worker[0/600] (sync'd 1 sec ago)
>
> DumpDnaForBlastZ-2782424c  (13)        DONE 0:cpum job(0/1 run:0  
> fail:0 32078ms) worker[0/1] (sync'd 1 sec ago)
>
> QueryFilterDuplicates-2782424c(14)        DONE 0:cpum job(0/15858  
> run:0 fail:0 1014ms) worker[0/50] (sync'd 1 sec ago)
>
> TargetFilterDuplicates-2782424c(15)        DONE 0:cpum job(0/16 run: 
> 0 fail:0 679609ms) worker[0/200] (sync'd 1 sec ago)
>
> CreateAlignmentChainsJobs  (16)      FAILED 0:cpum job(0/1 run:0  
> fail:1 0ms) worker[0/1] (sync'd 1 sec ago)
>
> AlignmentChains            (17)     BLOCKED 0:cpum job(0/0 run:0  
> fail:0 0ms) worker[0/600] (sync'd 1 sec ago)
>
> UpdateMaxAlignmentLengthAfterChain(18)     BLOCKED 0:cpum job(0/0  
> run:0 fail:0 0ms) worker[0/1] (sync'd 1 sec ago)
>
> Net-ContigAwareNet         (19)     BLOCKED 0:cpum job(0/0 run:0  
> fail:0 0ms) worker[0/600] (sync'd 1 sec ago)
>
> CreateAlignmentNetsJobs    (20)     BLOCKED 0:cpum job(1/1 run:0  
> fail:0 0ms) worker[1/1] (sync'd 1 sec ago)
>
> UpdateMaxAlignmentLengthAfterNet(21)     BLOCKED 0:cpum job(0/0 run: 
> 0 fail:0 0ms) worker[0/1] (sync'd 1 sec ago)
>
>
>
>
>
> Then i check the error output for the this job, which show me that:
>
>
>
> Job with id=38754 died in status 'RUN' for the following reason:
> -------------------- EXCEPTION --------------------
> MSG: must define analysis
> STACK Bio::EnsEMBL::Hive::DBSQL::AnalysisJobAdaptor::CreateNewJob / 
> home/SCE/jiangwenkai/ensembl/ensembl-hive/modules/Bio/EnsEMBL/Hive/DB
> SQL/AnalysisJobAdaptor.pm:90
> STACK  
> Bio 
> ::EnsEMBL 
> ::Compara 
> ::Production 
> ::GenomicAlignBlock 
> ::CreateAlignmentChainsJobs::createAlignmentChainsJobs /home/SCE/ 
> jiangwenkai/e
> nsembl/ensembl-compara/modules/Bio/EnsEMBL/Compara/Production/ 
> GenomicAlignBlock/CreateAlignmentChainsJobs.pm:281
> STACK  
> Bio 
> ::EnsEMBL 
> ::Compara 
> ::Production::GenomicAlignBlock::CreateAlignmentChainsJobs::run / 
> home/SCE/jiangwenkai/ensembl/ensembl-compara
> /modules/Bio/EnsEMBL/Compara/Production/GenomicAlignBlock/ 
> CreateAlignmentChainsJobs.pm:132
> STACK Bio::EnsEMBL::Hive::Worker::run_module_with_job /home/SCE/ 
> jiangwenkai/ensembl/ensembl-hive/modules/Bio/EnsEMBL/Hive/Worker.pm: 
> 698
> STACK (eval) /home/SCE/jiangwenkai/ensembl/ensembl-hive/modules/Bio/ 
> EnsEMBL/Hive/Worker.pm:607
> STACK Bio::EnsEMBL::Hive::Worker::run_one_batch /home/SCE/ 
> jiangwenkai/ensembl/ensembl-hive/modules/Bio/EnsEMBL/Hive/Worker.pm: 
> 606
> STACK Bio::EnsEMBL::Hive::Worker::run /home/SCE/jiangwenkai/ensembl/ 
> ensembl-hive/modules/Bio/EnsEMBL/Hive/Worker.pm:518
> STACK toplevel /home/SCE/jiangwenkai/ensembl/ensembl-hive/scripts/ 
> runWorker.pl:140
> ---------------------------------------------------
>
> All the code is similar to the examples 2x-alignment configuration  
> files, so how could this error occurs?
>
> The different might be that my query genomes dose not have  
> chromosomes, just lots of contigs range from 1kb ~ 300kb.
>
>
>
> the setting for my configuration is like:
>
>
>
>   { TYPE => PAIR_ALIGNER,
>
>
>
>     logic_name_prefix => 'BlastZ',
>
>     method_link => [10, 'BLASTZ_RAW'],
>
>     analysis_template => {
>
>         -program    => 'blastz',
>
>         -parameters => "{method_link=>'BLASTZ_RAW',options=>'T=1  
> L=3000 H=2200 M=40000000 O=400 E=30'}",
>
>         -module     =>  
> 'Bio::EnsEMBL::Compara::Production::GenomicAlignBlock::BlastZ',
>
>     },
>
>
>
>     query_collection_name     => 'oga genome',
>
>     target_collection_name    => 'rice genome',
>
>     filter_duplicates_options => 'all',
>
>     # set the following to about 50 to be kind to the mysql instances
>
>
>
>     # optional; default batch-size = 1,  3 is optimal for blastz
>
>     batch_size                => 1,
>
>     max_parallel_workers      => 600,
>
>
>
>   },
>
>
>
>   { TYPE => CHAIN_CONFIG,
>
>     method_link => [2001, 'HIVE_CHAIN'],
>
>     input_method_link_type => 'BLASTZ_RAW',
>
>
>
>     query_collection_name => 'oga genome',
>
>     target_collection_name => 'rice genome',
>
>
>
>     max_gap => 50,
>
>
>
>     max_parallel_workers => 600,
>
>   },
>
>
>
>   #
>
>   # set-up for alignment nets
>
>   #
>
>   {
>
>     TYPE => NET_CONFIG,
>
>
>
>     method_link => [3001, 'CONTIG_AWARE_NET'],
>
>     input_method_link_type => 'HIVE_CHAIN',
>
>
>
>     net_method => 'ContigAwareNet',
>
>
>
>     query_collection_name => 'oga genome',
>
>     target_collection_name => 'rice genome',
>
>
>
>     max_gap => 50,
>
>
>
>     max_parallel_workers => 600,
>
>   },
>
>
>
> Possbily i must add " analysis_template " to the  CHAIN_CONFIG and  
> NET_CONFIG? but if so, are there some templates.
>
>
>
> Thanks for all!
>
>
>
> Wenkai
>
>
>
>
>
>
>
>
>
> _______________________________________________
> Dev mailing list
> Dev at ensembl.org
> http://lists.ensembl.org/mailman/listinfo/dev











-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ensembl.org/pipermail/dev_ensembl.org/attachments/20101025/c075b83e/attachment.html>


More information about the Dev mailing list