[ensembl-dev] VEP script using Fork

[Duarte Molha]

I get a bunch of garbled output, probably a memory bump of some kind.
I can send you a file with the output if you wish.
However, there was another thing I notice on your email... 750 vars/sec !!! 

I am glad if I reach 75 vars/sec using cache. 

I am not sure if there is something wrong with my setup but that value is completely unreachable in my system :S

Cheers

Duarte


-----Original Message-----
From: dev-bounces at ensembl.org [mailto:dev-bounces at ensembl.org] On Behalf Of Will McLaren
Sent: 03 August 2012 11:06
To: Ensembl developers list
Subject: Re: [ensembl-dev] VEP script using Fork

Hi Duarte,

I've tested every fork number up to 30 without it crashing - what is the error you're seeing when it crashes?

I'd be surprised if you got any significant benefit using any more than 4, though, since there are overheads associated with creating each fork.

You can get a feel for the performance by looking at the status output that gives you the rate, something like:

2012-08-02 15:23:01 - Processed 16037 total variants (750 vars/sec,
927 vars/sec total)

Cheers

Will

On 3 August 2012 10:54, Duarte Molha <Duarte.Molha at ogt.co.uk> wrote:
> Hi
>
>
>
> I am trying out the new version and the fork does improve the performance.
>
>
>
> However the script crashes if I use more than 5 as the argument... 
> since my server has many more cores than that I would like to try and 
> see my much better it could run say forking at 16 (1 fork per core). Is this possible?
>
>
>
> Best regards,
>
>
>
> Duarte Molha
>
>
>
>
> _______________________________________________
> Dev mailing list    Dev at ensembl.org
> List admin (including subscribe/unsubscribe):
> http://lists.ensembl.org/mailman/listinfo/dev
> Ensembl Blog: http://www.ensembl.info/
>

_______________________________________________
Dev mailing list    Dev at ensembl.org
List admin (including subscribe/unsubscribe): http://lists.ensembl.org/mailman/listinfo/dev
Ensembl Blog: http://www.ensembl.info/