[ensembl-dev] VEP server blocks https requests

[Laurent Gil]

Dear Rokhaya,

The *homo_sapiens_merged_vep_95_GRCh37.tar *file you downloaded is what 
we call a VEP cache file and is not mean to be read/parsed outside VEP*. 
*Therefore we don't have a README file at the moment.*
*This file helps to speed up local installation of VEP, whithout the 
need to access remotely to the Ensembl databases, nor the REST API.

If you want to run VEP locally instead of using our REST API, you can 
find some documentation about the VEP installation here: 
https://www.ensembl.org/info/docs/tools/vep/script/vep_download.html*
*

And documentation about the VEP cache here: 
https://www.ensembl.org/info/docs/tools/vep/script/vep_cache.html#cache*
*

If you struggle to install VEP, we also have a Docker image available: 
https://www.ensembl.org/info/docs/tools/vep/script/vep_download.html#docker*.
*

Best regards,*
*

Laurent

On 26/03/2019 13:42, Rokhaya BA wrote:
>
> Dear Mark,
>
> Thank you very much for your answer. I tried to reduce my request 
> according to the service rate limit. Unfortunately, 15 requests per 
> second did not increase considerably my runtime, I need to submit 
> hundreds of variants per second.
>
> Anyway, I downloaded your "*/homo_sapiens_merged_vep_95_GRCh37.tar/*" 
> file and then tried to retrieve SIFT/Polyphen scores and predictions 
> locally. However, the file is a little bit tricky for me and does not 
> provide any readme.
>
> Is there any documentation that can help to understand the file 
> directories or where I can obtain information I need (SIFT/Polyphen 
> scores and predictions)?
>
> I am really grateful for your help.
>
> Best regards,
>
> Rokhaya BA
>
> PhD Student in Bioinformatics
>
> Le 21/03/2019 à 12:19, Mark McDowall a écrit :
>> Hi Rokhaya,
>>
>> A 429 error highlights that you are hitting the rate limit for a 
>> client to the service:
>>
>> https://github.com/Ensembl/ensembl-rest/wiki/HTTP-Response-Codes
>>
>> Have a look at the following documentation for the defined rate 
>> limits for the service:
>>
>> https://github.com/Ensembl/ensembl-rest/wiki/Rate-Limits
>>
>> You'll need to throttle your code by monitoring the headers to avoid 
>> getting the 503 errors
>>
>> Hope this helps,
>>
>> Mark
>>
>> On 21/03/2019 11:02, Rokhaya BA wrote:
>>> Hello,
>>>
>>> I am a PhD student in bioinformatics within the CRTI in Nantes, France.
>>>
>>> I am developing Ferret (http://limousophie35.github.io/Ferret/), a 
>>> Java application designed to retrieve annotations  from the VEP REST 
>>> API using variants id (rsid). This application submits a https 
>>> request and extract the sift/polyphen scores and predictions from 
>>> the JSON output.
>>>
>>> However, VEP server usually shut down my requests after some 
>>> iterations. Please see below some examples of errors returned by the 
>>> app:
>>>
>>> /Server returned HTTP response code: 429 for URL: 
>>> https://rest.ensembl.org/vep/human/id/rs41282276?content-type=application/json/
>>>
>>> /Server returned HTTP response code: 503 for URL: 
>>> https://rest.ensembl.org/vep/human/id/rs543148494?content-type=application/json/
>>>
>>> In practice, I use threads to speed up my runtime. Then, the 
>>> application submits many requests in a short amount of time.
>>>
>>> I wanted to know if there is any way to solve this issue?
>>>
>>> Thank you very much for your help.
>>>
>>> Best regards,
>>>
>>> Rokhaya BA
>>>
>>> PhD Student in Bioinformatics
>>>
>>>
>>>
>>> _______________________________________________
>>> Dev mailing list Dev at ensembl.org
>>> Posting guidelines and subscribe/unsubscribe info: 
>>> http://lists.ensembl.org/mailman/listinfo/dev
>>> Ensembl Blog: http://www.ensembl.info/
>>>
>>
>
> _______________________________________________
> Dev mailing list    Dev at ensembl.org
> Posting guidelines and subscribe/unsubscribe info: http://lists.ensembl.org/mailman/listinfo/dev
> Ensembl Blog: http://www.ensembl.info/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ensembl.org/pipermail/dev_ensembl.org/attachments/20190326/eb1454ea/attachment.html>