[ensembl-dev] personal Amazon hosted Ensembl database

[Thibaut Hourlier]

Hi Joseph,
One thing you can try is to know how many connections are allowed to the server, then you should have less jobs running in parallel than the maximum number of connections allowed.
As you have many short request you should also make sure that you do not disconnect each time. Each time you disconnect, the server will use a different port, so you may also quickly run out of port to connect to. If you use the Perl API, you can set -disconnect_when_inactive => 0 when creating the connection to the database.

Hope this helps
Thibaut

> On 8 Oct 2019, at 14:53, Joseph Steinberger <joseph.steinberger at weizmann.ac.il> wrote:
> 
> Dear Development community,
> 
> Is there a precedent for having a personal Amazon AWS clone of the entire Ensembl database?
> I am making many short requests and getting timed out -
> Specifically, I want the genetree id for each of a few million genes, and I cannot do it in parallel or I get a "too many requests" error. 
> 
> Thank you for your time.
> 
> Sincerely,
> Joseph Steinberger
> 
> laboratory of Naama Barkai, Weizmann Institute of Science
> _______________________________________________
> Dev mailing list    Dev at ensembl.org <mailto:Dev at ensembl.org>
> Posting guidelines and subscribe/unsubscribe info: https://lists.ensembl.org/mailman/listinfo/dev_ensembl.org <https://lists.ensembl.org/mailman/listinfo/dev_ensembl.org>
> Ensembl Blog: http://www.ensembl.info/ <http://www.ensembl.info/>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.ensembl.org/pipermail/dev_ensembl.org/attachments/20191008/c47a9158/attachment.html>