[ensembl-dev] VEP error: Forked process failed.

Guillermo Marco Puche guillermo.marco at sistemasgenomicos.com
Tue May 14 08:28:05 BST 2013


Hello,

I just added the chr5.vcf file that is causing me the crashes to the repo.
Forget about the weird vcf format. First columns are standard the rest 
are not.
https://github.com/guillermomarco/vep_plugins_71/blob/master/chr5.vcf

On 05/14/2013 08:37 AM, Guillermo Marco Puche wrote:
> Hello,
>
> I'm not really sure which one of those plugins is causing the fork 
> error. I cannot recreate it now running each one of them separately.
>
> Here are both:
>
> https://github.com/guillermomarco/vep_plugins_71
>
> They also slow the calculating consequences process a lot. 
> aa_grantham_distance.pm is just a hardcoded plugin from one of the 
> biologists in my work. It was just a pure copy paste and adaptation to 
> make it work as a VEP plugin. Maybe the problem is in the matrix 
> definition every time the sub routine is called. I'm not running out 
> of memory nor CPU. I'm currently using it with 2 threads and 
> buffersize of 500 for a 5000 variant vcf file.
>
> I'm my honest opinion, I think one (or even both) of those plugins are 
> slowing so much the calculating process that sometimes the fork just 
> dies. Like when you have a timeout during to heavy network traffic. So 
> when you use them together with lot of other plugins like Condel, 
> Consequence, etc.. they may be causing the process to handle and die.
>
> Best regards,
> Guillermo.
>
> On 05/13/2013 03:55 PM, Duarte Molha wrote:
>> I also get this error... it is so prevalent and so difficult to 
>> pinpoint what is causing it that I have given up on forking my 
>> annotation process.
>>
>> I do think it is related to the number of forks. It seems to crash 
>> less often if you use a low number of forks... anything above 5 
>> will undoubtedly crash the script at least in my experience.
>>
>> Cheers
>>
>> Duarte
>>
>> =========================
>>      Duarte Miguel Paulo Molha
>> http://about.me/duarte
>> =========================
>>
>>
>> On Mon, May 13, 2013 at 2:50 PM, Will McLaren <wm2 at ebi.ac.uk 
>> <mailto:wm2 at ebi.ac.uk>> wrote:
>>
>>     Hi Guillermo,
>>
>>     Test each plugin individually until you find the one that causes
>>     the error. It is highly unlikely that a particular combination of
>>     plugins is causing the crash.
>>
>>     Check that there are no "print" (to STDOUT or STDERR) statements
>>     in your plugin - forking assumes that code remains silent
>>     otherwise it will throw errors like this.
>>
>>     Also, check what, if anything, is cached between runs of your
>>     plugin. If you are caching things (for example to avoid
>>     re-querying a database), you may need to write storable hooks to
>>     ensure the data is getting cached between forks - see
>>     https://github.com/ensembl-variation/VEP_plugins/blob/master/ProteinSeqs.pm
>>     for an example.
>>
>>     If you still have no luck, send me the code and an input file
>>     that recreates the problem.
>>
>>     Regards
>>
>>     Will
>>
>>
>>     On 13 May 2013 13:18, Guillermo Marco Puche
>>     <guillermo.marco at sistemasgenomicos.com
>>     <mailto:guillermo.marco at sistemasgenomicos.com>> wrote:
>>
>>         Hello,
>>
>>         I've started to recently having problems with VEP script
>>         while using different plugins (most of them own plugins).
>>
>>         2013-05-13 13:59:44 - Connected to core version 71 database and variation version 71 database
>>         2013-05-13 13:59:44 - Loaded plugin: vcf_input
>>         2013-05-13 13:59:44 - Loaded plugin: biobase
>>         2013-05-13 13:59:44 - Loaded plugin: aa_grantham_distance
>>         2013-05-13 13:59:44 - Loaded plugin: flanking_sequence
>>         2013-05-13 13:59:44 - Loaded plugin: Condel
>>         2013-05-13 13:59:44 - Output fields redefined (37 defined)
>>         2013-05-13 13:59:44 - Starting...
>>         2013-05-13 13:59:45 - Read 3888 variants into buffer
>>         2013-05-13 13:59:54 - Reading transcript data from cache and/or database
>>         [===============================================]  [ 100% ]
>>         2013-05-13 14:02:38 - Retrieved 6463 transcripts (0 mem, 0 cached, 13743 DB, 7280 duplicates)
>>         2013-05-13 14:02:38 - Calculating consequences
>>         [===================================>           ]   [ 78% ]
>>         ERROR: Forked process failed
>>
>>
>>         I'm not getting any other error message. So I cannot debug
>>         properly. I thought my plugins were OK but it's seems they
>>         don't. I think the problem occurs when I use
>>         "aa_grantham_distance plugin" together with
>>         "flanking_sequence". I've no idea what could be causing this.
>>
>>         I'm running VEP on verbose mode but I can't get any usefull
>>         information. How could I debug that?
>>
>>         Best regards,
>>         Guillermo.
>>
>>
>>         _______________________________________________
>>         Dev mailing list Dev at ensembl.org <mailto:Dev at ensembl.org>
>>         Posting guidelines and subscribe/unsubscribe info:
>>         http://lists.ensembl.org/mailman/listinfo/dev
>>         Ensembl Blog: http://www.ensembl.info/
>>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ensembl.org/pipermail/dev_ensembl.org/attachments/20130514/79aeb98a/attachment.html>


More information about the Dev mailing list