[ensembl-dev] VEP error: Forked process failed.

Duarte Molha duartemolha at gmail.com
Fri May 17 16:41:43 BST 2013


Something is definitely something wrong with the later changes ... both
using the branch version and the head version of VEP throughs thousands of
errors of the type

Use of uninitialized value $_ in scalar chomp at
/services/.array/OGTIP/duartem/workspace/VEP_dev/Bio/EnsEMBL/Variation/Utils/VEP.pm
line 1081, <$child> line XXXXXXX.

Best regards

Duarte



=========================
     Duarte Miguel Paulo Molha
         http://about.me/duarte
=========================


On Fri, May 17, 2013 at 4:13 PM, Will McLaren <wm2 at ebi.ac.uk> wrote:

> I pushed the fork changes to branch as well today as I felt they were
> stable enough.
>
> So, you can use branch and it should be fine.
>
> There will be more coming but they can wait for the next main release.
>
> Will
>
>
> On 17 May 2013 16:06, Duarte Molha <Duarte.Molha at ogt.com> wrote:
>
>> I believe you have to use the version in HEAD from the CSV repository.***
>> *
>>
>> ** **
>>
>> Please correct me if I am wrong Will.****
>>
>> ** **
>>
>> Best regards****
>>
>> ** **
>>
>> Duarte****
>>
>> ** **
>>
>> ** **
>>
>> *From:* dev-bounces at ensembl.org [mailto:dev-bounces at ensembl.org] *On
>> Behalf Of *Guillermo Marco Puche
>> *Sent:* 16 May 2013 12:30
>> *To:* Ensembl developers list
>> *Subject:* Re: [ensembl-dev] VEP error: Forked process failed.****
>>
>> ** **
>>
>> Hello,
>>
>> I got lost in this topic.
>> So if I understand well.. to fix the VEP forking issue I must can
>> re-download the VEP script from ensembl website?
>>
>> Thank you !
>>
>> Best regards,
>> Guillermo.
>>
>> On 05/15/2013 01:35 PM, Will McLaren wrote:****
>>
>> This is now fixed on branch and head too. ****
>>
>> ** **
>>
>> Will****
>>
>> ** **
>>
>> On 14 May 2013 17:30, Duarte Molha <duartemolha at gmail.com> wrote:****
>>
>> Sorry Will... today you have loads of people bugging you :-) ****
>>
>> ** **
>>
>> I just updated to the latest VEP you pointed me to ... and now the fail
>> in on line 1307****
>>
>> ** **
>>
>> Use of uninitialized value $line in join or string at
>> /NGS_Test/duarte/vep_71/Bio/EnsEMBL/Variation/Utils/VEP.pm line 1307.****
>>
>> ** **
>>
>> Cheers****
>>
>>
>> Duarte****
>>
>> ** **
>>
>>
>> ****
>>
>> =========================
>>      Duarte Miguel Paulo Molha      ****
>>
>>          http://about.me/duarte
>> =========================****
>>
>> ** **
>>
>> On Tue, May 14, 2013 at 4:26 PM, Will McLaren <wm2 at ebi.ac.uk> wrote:****
>>
>> Hi Duarte, ****
>>
>> ** **
>>
>> You are correct, it was the --individual flag causing the problem.****
>>
>> ** **
>>
>> I have committed a fix to the 71 branch (does not include forking fixes)
>> and to head (does include forking fixes, download from
>> http://cvs.sanger.ac.uk/cgi-bin/viewvc.cgi/ensembl-variation/modules/Bio/EnsEMBL/Variation/Utils/VEP.pm?revision=1.94&root=ensembl
>> ).****
>>
>> ** **
>>
>> Regards****
>>
>> ** **
>>
>> Will****
>>
>> ** **
>>
>> On 14 May 2013 15:17, Duarte Molha <duartemolha at gmail.com> wrote:****
>>
>> However I believe it must be something related to the --individual
>> [all|ind list] flag ****
>>
>> And does not seem to be related to the --forks argument since it also
>> thoughs that error when running single threaded. ****
>>
>> ** **
>>
>> However in single thread mode it simply displays the error and keeps
>> going and with multiple forks the process dies entirely.****
>>
>>   ****
>>
>> My vcf contains 3 samples genotypes and I have included the --individual
>> all option. ****
>>
>> ** **
>>
>> Best regards****
>>
>> Duarte****
>>
>> ** **
>>
>>
>> ****
>>
>> =========================
>>      Duarte Miguel Paulo Molha      ****
>>
>>          http://about.me/duarte
>> =========================****
>>
>> ** **
>>
>> On Tue, May 14, 2013 at 2:00 PM, Duarte Molha <duartemolha at gmail.com>
>> wrote:****
>>
>> Hi Will ****
>>
>> ** **
>>
>> Unfortunately the script does not report in which line it fails and I
>> cannot provide you with the entire file since it is private data.****
>>
>> Is there a way of reporting the line >****
>>
>> ** **
>>
>> Thanks****
>>
>>
>> Duarte****
>>
>> ** **
>>
>>
>> ****
>>
>> =========================
>>      Duarte Miguel Paulo Molha      ****
>>
>>          http://about.me/duarte
>> =========================****
>>
>> ** **
>>
>> On Tue, May 14, 2013 at 1:05 PM, Will McLaren <wm2 at ebi.ac.uk> wrote:****
>>
>> Hi Duarte, ****
>>
>>
>> Do you have some input that causes this error?****
>>
>> ** **
>>
>> Thanks****
>>
>> ** **
>>
>> Will****
>>
>> ** **
>>
>> On 14 May 2013 12:57, Duarte Molha <duartemolha at gmail.com> wrote:****
>>
>> Another bug using the updated version... now using the --check_alleles
>> and --check_existing options the script dies at line 4759 ****
>>
>> ** **
>>
>> Use of uninitialized value in string ne at
>> /NGS_Test/duarte/vep_71/Bio/EnsEMBL/Variation/Utils/VEP.pm line 4759.****
>>
>> ** **
>>
>> ** **
>>
>> Best regards****
>>
>> ** **
>>
>> Duarte****
>>
>>
>> ****
>>
>> =========================
>>      Duarte Miguel Paulo Molha      ****
>>
>>          http://about.me/duarte
>> =========================****
>>
>> ** **
>>
>> On Tue, May 14, 2013 at 11:22 AM, Will McLaren <wm2 at ebi.ac.uk> wrote:****
>>
>> Thanks - try
>> http://cvs.sanger.ac.uk/cgi-bin/viewvc.cgi/ensembl-variation/modules/Bio/EnsEMBL/Variation/Utils/VEP.pm?revision=1.93&root=ensembl
>> ****
>>
>> ** **
>>
>> Will****
>>
>> ** **
>>
>> On 14 May 2013 11:09, Duarte Molha <duartemolha at gmail.com> wrote:****
>>
>> It seems the problems are still there : ****
>>
>> ** **
>>
>> Here is my output:****
>>
>> ** **
>>
>> perl variant_effect_predictor.pl --config vep_human.ini -i INPUT.vcf
>> --fork 16****
>>
>> ** **
>>
>> 2013-05-14 10:33:38 - Read configuration from vep_human.ini****
>>
>> #----------------------------------#****
>>
>> # ENSEMBL VARIANT EFFECT PREDICTOR #****
>>
>> #----------------------------------#****
>>
>> ** **
>>
>> version 71****
>>
>> ** **
>>
>> By Will McLaren (wm2 at ebi.ac.uk)****
>>
>> ** **
>>
>> Configuration options:****
>>
>> ** **
>>
>> ###****
>>
>> allow_non_variant    1****
>>
>> buffer_size                 500000****
>>
>> cache                1****
>>
>> canonical            1****
>>
>> ccds                 1****
>>
>> check_alleles        1****
>>
>> check_existing       1****
>>
>> config               vep_human.ini****
>>
>> core_type            core****
>>
>> custom
>> /ReferenceData/vep_additional_annotations/Somatic_variation_phenotypes.bed.gz,Somatic,bed,exact
>>      ****
>>
>>
>> /ReferenceData/vep_additional_annotations/dbsnp135_ensembl_variation_phenotype.bed.gz,dbsnp135,bed,exact
>> ****
>>
>> dir                       /ReferenceData/vep_cache/****
>>
>> domains                 1****
>>
>> force_overwrite      1****
>>
>> fork                          16****
>>
>> gmaf                 1****
>>
>> hgnc                 1****
>>
>> host                 ensembldb.ensembl.org****
>>
>> individual           all****
>>
>> input_file           INPUT.vcf****
>>
>> numbers              1****
>>
>> plugin               Blosum62
>> Condel,/ReferenceData/vep_cache/Plugins/config/Condel/config,b  Carol****
>>
>> polyphen             b****
>>
>> port                 5306****
>>
>> protein              1****
>>
>> regulatory           1****
>>
>> sift                 b****
>>
>> species              homo_sapiens****
>>
>> stats                HASH(0x35a8000)****
>>
>> terms                SO****
>>
>> toplevel_dir         /ReferenceData/vep_cache/****
>>
>> verbose              1****
>>
>> xref_refseq          1****
>>
>> ** **
>>
>> --------------------****
>>
>> ** **
>>
>> Will only load v71 databases****
>>
>> Species 'homo_sapiens' loaded from database 'homo_sapiens_core_71_37'****
>>
>> Species 'homo_sapiens' loaded from database 'homo_sapiens_cdna_71_37'****
>>
>> Species 'homo_sapiens' loaded from database 'homo_sapiens_vega_71_37'****
>>
>> Species 'homo_sapiens' loaded from database
>> 'homo_sapiens_otherfeatures_71_37'****
>>
>> Species 'homo_sapiens' loaded from database 'homo_sapiens_rnaseq_71_37'**
>> **
>>
>> homo_sapiens_variation_71_37 loaded****
>>
>> homo_sapiens_funcgen_71_37 loaded****
>>
>> Bio::EnsEMBL::Compara::DBSQL::DBAdaptor not found so the following
>> compara databases will be ignored: ensembl_compara_71****
>>
>> ensembl_ancestral_71 loaded****
>>
>> ensembl_ontology_71 loaded****
>>
>> 2013-05-14 10:33:39 - Connected to core version 71 database and variation
>> version 71 database****
>>
>> 2013-05-14 10:33:39 - Read existing cache info****
>>
>> 2013-05-14 10:33:39 - Loaded plugin: Blosum62****
>>
>> 2013-05-14 10:33:39 - Loaded plugin: Condel****
>>
>> 2013-05-14 10:33:39 - Loaded plugin: Carol****
>>
>> 2013-05-14 10:33:40 - Starting...****
>>
>> 2013-05-14 10:33:40 - Detected format of input file as vcf****
>>
>> 2013-05-14 10:33:46 - Read 195789 variants into buffer****
>>
>> 2013-05-14 10:33:46 - Skipping 67552 non-variant loci****
>>
>> 2013-05-14 10:33:46 - Reading transcript data from cache and/or database*
>> ***
>>
>> [======================================================================================================]
>>  [ 100% ]****
>>
>> 2013-05-14 10:40:19 - Retrieved 189344 transcripts (0 mem, 202901 cached,
>> 0 DB, 13557 duplicates)****
>>
>> 2013-05-14 10:40:19 - Reading regulatory data from cache and/or database*
>> ***
>>
>> [======================================================================================================]
>>  [ 100% ]****
>>
>> 2013-05-14 10:50:09 - Retrieved 872092 regulatory features (0 mem, 872351
>> cached, 0 DB, 259 duplicates)****
>>
>> 2013-05-14 10:50:12 - Calculating consequences****
>>
>> Use of uninitialized value $_ in pattern match (m//) at
>> /NGS_Test/duarte/vep_71/Bio/EnsEMBL/Variation/Utils/VEP.pm line 1022.****
>>
>> Use of uninitialized value $_ in pattern match (m//) at
>> /NGS_Test/duarte/vep_71/Bio/EnsEMBL/Variation/Utils/VEP.pm line 1030.****
>>
>> Use of uninitialized value $_ in pattern match (m//) at
>> /NGS_Test/duarte/vep_71/Bio/EnsEMBL/Variation/Utils/VEP.pm line 1037.****
>>
>> Use of uninitialized value $_ in pattern match (m//) at
>> /NGS_Test/duarte/vep_71/Bio/EnsEMBL/Variation/Utils/VEP.pm line 1089.****
>>
>> Use of uninitialized value $_ in concatenation (.) or string at
>> /NGS_Test/duarte/vep_71/Bio/EnsEMBL/Variation/Utils/VEP.pm line 1097.****
>>
>> ** **
>>
>> ERROR: Forked process failed****
>>
>> ** **
>>
>> ** **
>>
>> ** **
>>
>>
>> ****
>>
>> =========================
>>      Duarte Miguel Paulo Molha      ****
>>
>>          http://about.me/duarte
>> =========================****
>>
>> ** **
>>
>> On Tue, May 14, 2013 at 10:42 AM, Duarte Molha <duartemolha at gmail.com>
>> wrote:****
>>
>> Thanks ****
>>
>> ** **
>>
>> Running a annotation using 16 forks... lets see how it handles :)****
>>
>> I'll report back any issues.****
>>
>> ** **
>>
>> Thanks for the update****
>>
>> ** **
>>
>> Duarte****
>>
>> ** **
>>
>>
>> ****
>>
>> =========================
>>      Duarte Miguel Paulo Molha      ****
>>
>>          http://about.me/duarte
>> =========================****
>>
>> ** **
>>
>> On Tue, May 14, 2013 at 10:16 AM, Will McLaren <wm2 at ebi.ac.uk> wrote:****
>>
>> Stuart, Guillermo, Duarte, ****
>>
>> ** **
>>
>> I'm currently working on some code as I stated above to improve stability
>> and performance under forking.****
>>
>> ** **
>>
>> I've committed some code to the HEAD of our CVS tree which should help
>> the problems you are encountering. You'd all be welcome to test this out,
>> with the obvious proviso that this is development code and may contain bugs!
>> ****
>>
>> ** **
>>
>> To use this, you should download the copy of VEP.pm from:****
>>
>> ** **
>>
>>
>> http://cvs.sanger.ac.uk/cgi-bin/viewvc.cgi/ensembl-variation/modules/Bio/EnsEMBL/Variation/Utils/VEP.pm?revision=1.92&root=ensembl
>> ****
>>
>> ** **
>>
>> and replace the VEP.pm under
>> ensembl-variation/modules/Bio/EnsEMBL/Variation/Utils (or just
>> Bio/EnsEMBL/Variation/Utils if you use INSTALL.pl) with this one.****
>>
>> ** **
>>
>> This code will appear in production in the next proper release of Ensembl.
>> ****
>>
>> ** **
>>
>> Regards****
>>
>> ** **
>>
>> Will****
>>
>> ** **
>>
>> On 14 May 2013 09:55, Stuart Meacham <sm766 at cam.ac.uk> wrote:****
>>
>> Hi,
>>
>> I certainly don't want to hijack this thread but it seemed daft to start
>> another. I am also getting forking errors. I don't use any custom plugins
>> and am using a validated VCF as input (with about 600,000 variants). Trying
>> to fork more than 4 threads is unstable even on my machine which has 64
>> cores and half a TB of RAM.
>>
>> I haven't found anything reproducible, however if I do I'll report back
>> to the list.
>>
>> Thanks
>>
>> Stuart ****
>>
>>
>>
>> On 14/05/2013 09:42, Will McLaren wrote:****
>>
>> Hello, ****
>>
>> ** **
>>
>> Your aa_grantham_distance plugin is somewhat inefficient - it retrieves
>> the peptide alleles from the HGVS annotation, which itself requires some
>> database fetching and processing to produce. This is why it is slow.****
>>
>> ** **
>>
>> You can get the peptides from the transcript variation object:****
>>
>> ** **
>>
>> my @peps = split "/", $tva->transcript_variation->pep_allele_string();***
>> *
>>
>> ** **
>>
>> This will give you single-letter AA codes, but you could either modify
>> your hash or use BioPerl to convert:****
>>
>> ** **
>>
>> $seqobj = Bio::PrimarySeq->new ( -seq => $single_letter_aa);
>> $three_letter_aa = Bio::SeqUtils->seq3($seqobj); ****
>>
>> ** **
>>
>> You should also declare your distances hash in the new() sub and store it
>> on $self; this will also marginally speed up your plugin.****
>>
>> ** **
>>
>> Regarding the forking issues, we are working on improving stability under
>> forking.****
>>
>> ** **
>>
>> Thanks for your patience****
>>
>> ** **
>>
>> Will****
>>
>> ** **
>>
>> On 14 May 2013 07:37, Guillermo Marco Puche <
>> guillermo.marco at sistemasgenomicos.com> wrote:****
>>
>> Hello,
>>
>> I'm not really sure which one of those plugins is causing the fork error.
>> I cannot recreate it now running each one of them separately.
>>
>> Here are both:
>>
>> https://github.com/guillermomarco/vep_plugins_71
>>
>> They also slow the calculating consequences process a lot.
>> aa_grantham_distance.pm is just a hardcoded plugin from one of the
>> biologists in my work. It was just a pure copy paste and adaptation to make
>> it work as a VEP plugin. Maybe the problem is in the matrix definition
>> every time the sub routine is called. I'm not running out of memory nor
>> CPU. I'm currently using it with 2 threads and buffersize of 500 for a 5000
>> variant vcf file.
>>
>> I'm my honest opinion, I think one (or even both) of those plugins are
>> slowing so much the calculating process that sometimes the fork just dies.
>> Like when you have a timeout during to heavy network traffic. So when you
>> use them together with lot of other plugins like Condel, Consequence, etc..
>> they may be causing the process to handle and die.
>>
>> Best regards,
>> Guillermo. ****
>>
>>
>>
>> On 05/13/2013 03:55 PM, Duarte Molha wrote:****
>>
>> I also get this error... it is so prevalent and so difficult to pinpoint
>> what is causing it that I have given up on forking my annotation process.
>> ****
>>
>> ** **
>>
>> I do think it is related to the number of forks. It seems to crash less
>> often if you use a low number of forks... anything above 5
>> will undoubtedly crash the script at least in my experience.****
>>
>> ** **
>>
>> Cheers****
>>
>>
>> Duarte****
>>
>>
>> ****
>>
>> =========================
>>      Duarte Miguel Paulo Molha      ****
>>
>>          http://about.me/duarte
>> =========================****
>>
>> ** **
>>
>> On Mon, May 13, 2013 at 2:50 PM, Will McLaren <wm2 at ebi.ac.uk> wrote:****
>>
>> Hi Guillermo, ****
>>
>> ** **
>>
>> Test each plugin individually until you find the one that causes the
>> error. It is highly unlikely that a particular combination of plugins is
>> causing the crash.****
>>
>> ** **
>>
>> Check that there are no "print" (to STDOUT or STDERR) statements in your
>> plugin - forking assumes that code remains silent otherwise it will throw
>> errors like this.****
>>
>> ** **
>>
>> Also, check what, if anything, is cached between runs of your plugin. If
>> you are caching things (for example to avoid re-querying a database), you
>> may need to write storable hooks to ensure the data is getting cached
>> between forks - see
>> https://github.com/ensembl-variation/VEP_plugins/blob/master/ProteinSeqs.pmfor an example.
>> ****
>>
>> ** **
>>
>> If you still have no luck, send me the code and an input file that
>> recreates the problem.****
>>
>> ** **
>>
>> Regards****
>>
>> ** **
>>
>> Will****
>>
>> ** **
>>
>> On 13 May 2013 13:18, Guillermo Marco Puche <
>> guillermo.marco at sistemasgenomicos.com> wrote:****
>>
>> Hello,
>>
>> I've started to recently having problems with VEP script while using
>> different plugins (most of them own plugins).****
>>
>> 2013-05-13 13:59:44 - Connected to core version 71 database and variation version 71 database****
>>
>> 2013-05-13 13:59:44 - Loaded plugin: vcf_input****
>>
>> 2013-05-13 13:59:44 - Loaded plugin: biobase****
>>
>> 2013-05-13 13:59:44 - Loaded plugin: aa_grantham_distance****
>>
>> 2013-05-13 13:59:44 - Loaded plugin: flanking_sequence****
>>
>> 2013-05-13 13:59:44 - Loaded plugin: Condel****
>>
>> 2013-05-13 13:59:44 - Output fields redefined (37 defined)****
>>
>> 2013-05-13 13:59:44 - Starting...****
>>
>> 2013-05-13 13:59:45 - Read 3888 variants into buffer****
>>
>>
>> 2013-05-13 13:59:54 - Reading transcript data from cache and/or database****
>>
>> [===============================================]  [ 100% ]****
>>
>> 2013-05-13 14:02:38 - Retrieved 6463 transcripts (0 mem, 0 cached, 13743 DB, 7280 duplicates)****
>>
>> 2013-05-13 14:02:38 - Calculating consequences****
>>
>> [===================================>           ]   [ 78% ]****
>>
>> ERROR: Forked process failed****
>>
>> ** **
>>
>> ** **
>>
>> I'm not getting any other error message. So I cannot debug properly. I
>> thought my plugins were OK but it's seems they don't. I think the problem
>> occurs when I use "aa_grantham_distance plugin" together with
>> "flanking_sequence". I've no idea what could be causing this.
>>
>> I'm running VEP on verbose mode but I can't get any usefull information.
>> How could I debug that?
>>
>> Best regards,
>> Guillermo.****
>>
>> ** **
>>
>> _______________________________________________
>> Dev mailing list    Dev at ensembl.org
>> Posting guidelines and subscribe/unsubscribe info:
>> http://lists.ensembl.org/mailman/listinfo/dev
>> Ensembl Blog: http://www.ensembl.info/****
>>
>>
>> _______________________________________________
>> Dev mailing list    Dev at ensembl.org
>> Posting guidelines and subscribe/unsubscribe info:
>> http://lists.ensembl.org/mailman/listinfo/dev
>> Ensembl Blog: http://www.ensembl.info/****
>>
>> ** **
>>
>>
>>
>> ****
>>
>> _______________________________________________****
>>
>> Dev mailing list    Dev at ensembl.org****
>>
>> Posting guidelines and subscribe/unsubscribe info: http://lists.ensembl.org/mailman/listinfo/dev****
>>
>> Ensembl Blog: http://www.ensembl.info/****
>>
>> ** **
>>
>>
>> _______________________________________________
>> Dev mailing list    Dev at ensembl.org
>> Posting guidelines and subscribe/unsubscribe info:
>> http://lists.ensembl.org/mailman/listinfo/dev
>> Ensembl Blog: http://www.ensembl.info/****
>>
>> ** **
>>
>>
>> _______________________________________________
>> Dev mailing list    Dev at ensembl.org
>> Posting guidelines and subscribe/unsubscribe info:
>> http://lists.ensembl.org/mailman/listinfo/dev
>> Ensembl Blog: http://www.ensembl.info/****
>>
>> ** **
>>
>> ** **
>>
>>
>> _______________________________________________
>> Dev mailing list    Dev at ensembl.org
>> Posting guidelines and subscribe/unsubscribe info:
>> http://lists.ensembl.org/mailman/listinfo/dev
>> Ensembl Blog: http://www.ensembl.info/****
>>
>> ** **
>>
>>
>> _______________________________________________
>> Dev mailing list    Dev at ensembl.org
>> Posting guidelines and subscribe/unsubscribe info:
>> http://lists.ensembl.org/mailman/listinfo/dev
>> Ensembl Blog: http://www.ensembl.info/****
>>
>> ** **
>>
>>
>> _______________________________________________
>> Dev mailing list    Dev at ensembl.org
>> Posting guidelines and subscribe/unsubscribe info:
>> http://lists.ensembl.org/mailman/listinfo/dev
>> Ensembl Blog: http://www.ensembl.info/****
>>
>> ** **
>>
>>
>> _______________________________________________
>> Dev mailing list    Dev at ensembl.org
>> Posting guidelines and subscribe/unsubscribe info:
>> http://lists.ensembl.org/mailman/listinfo/dev
>> Ensembl Blog: http://www.ensembl.info/****
>>
>> ** **
>>
>> ** **
>>
>>
>> _______________________________________________
>> Dev mailing list    Dev at ensembl.org
>> Posting guidelines and subscribe/unsubscribe info:
>> http://lists.ensembl.org/mailman/listinfo/dev
>> Ensembl Blog: http://www.ensembl.info/****
>>
>> ** **
>>
>>
>> _______________________________________________
>> Dev mailing list    Dev at ensembl.org
>> Posting guidelines and subscribe/unsubscribe info:
>> http://lists.ensembl.org/mailman/listinfo/dev
>> Ensembl Blog: http://www.ensembl.info/****
>>
>> ** **
>>
>>
>> _______________________________________________
>> Dev mailing list    Dev at ensembl.org
>> Posting guidelines and subscribe/unsubscribe info:
>> http://lists.ensembl.org/mailman/listinfo/dev
>> Ensembl Blog: http://www.ensembl.info/****
>>
>> ** **
>>
>>
>>
>>
>> ****
>>
>> _______________________________________________****
>>
>> Dev mailing list    Dev at ensembl.org****
>>
>> Posting guidelines and subscribe/unsubscribe info: http://lists.ensembl.org/mailman/listinfo/dev****
>>
>> Ensembl Blog: http://www.ensembl.info/****
>>
>> ------------------------------
>>
>> _______________________________________________
>> Dev mailing list    Dev at ensembl.org
>> Posting guidelines and subscribe/unsubscribe info:
>> http://lists.ensembl.org/mailman/listinfo/dev
>> Ensembl Blog: http://www.ensembl.info/
>>
>>
>
> _______________________________________________
> Dev mailing list    Dev at ensembl.org
> Posting guidelines and subscribe/unsubscribe info:
> http://lists.ensembl.org/mailman/listinfo/dev
> Ensembl Blog: http://www.ensembl.info/
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ensembl.org/pipermail/dev_ensembl.org/attachments/20130517/371df148/attachment.html>


More information about the Dev mailing list