[ensembl-dev] Atlas tracks - fail to add on Gramene genome browser, method not allowed

Wed Mar 25 18:26:39 GMT 2015

Thanks, Anne. It worked this time and the source was successfully attached.

Sharon

On Mar 24, 2015, at 12:31 PM, Anne Lyle <annelyle at ebi.ac.uk<mailto:annelyle at ebi.ac.uk>> wrote:

Sorry, try again - the push to github failed the first time.

Cheers

Anne

On 24 Mar 2015, at 16:24, Wei, Xuehong <weix at cshl.edu<mailto:weix at cshl.edu>> wrote:

Hi Anne,

Was the change made on branch release/79? I just did a pull under ensembl-webcode and it is already up-to-date. But I see no changes from this morning.

[weix at paulaner ensembl-webcode]$ git status
# On branch release/79
nothing to commit (working directory clean)

[weix at paulaner ensembl-webcode]$ git pull
Already up-to-date.

ls -ltr /usr/local/ensembl-live/ensembl-webcode/modules/EnsEMBL/Web/File/Utils/URL.pm
-rw-rw-r--. 1 weix weix 10984 Mar 24 10:47 /usr/local/ensembl-live/ensembl-webcode/modules/EnsEMBL/Web/File/Utils/URL.pm

Sharon

On Mar 24, 2015, at 12:12 PM, Anne Lyle <annelyle at ebi.ac.uk<mailto:annelyle at ebi.ac.uk><mailto:annelyle at ebi.ac.uk>> wrote:

Hi Sharon

Sorry, I misread your email. I did put in a code fix elsewhere to use a GET request instead of HEAD (e.g. in the file_exists method), but the file attachment interface is also using the get_filesize method, which only works if the server accepts a HEAD request.

I’ve made some changes to work around this situation, as we’re not responsible for the setup of the Atlas server (we just happen to be based at the same institute). Please update your code again and let me know if you have any further problems.

Cheers

Anne

On 24 Mar 2015, at 15:33, Wei, Xuehong <weix at cshl.edu<mailto:weix at cshl.edu><mailto:weix at cshl.edu>> wrote:

Hi Anne,

Thanks for the information. I just updated ensembl-webcode to the release/79. The problem is still there. And I take a look at the master code at https://github.com/Ensembl/ensembl-webcode/blob/release/79/modules/EnsEMBL/Web/File/Utils/URL.pm

The function get_headers still use my $response = $http->request('HEAD', $url);

sub get_headers {
      ### Get one or all headers from a remote file
      ### @param url - URL of file
      ### @param Args Hashref
      ### header (optional) String - name of header
      ### hub EnsEMBL::Web::Hub
      ### nice (optional) Boolean - see introduction
      ### compression String (optional) - compression type
      ### @return Hashref containing results (single header or hashref of headers) or errors (ArrayRef)
      my ($file, $args) = @_;
      my $url = ref($file) ? $file->absolute_read_path : $file;
      my ($all_headers, $result, $error);

      if ($url =~ /^ftp/) {
      ## TODO - support FTP properly!
      return {'Content-Type' => 1};
      }
      else {
      my %params = ('timeout' => 10);
      if ($args->{'hub'}->species_defs->ENSEMBL_WWW_PROXY) {
      $params{'http_proxy'} = $args->{'hub'}->species_defs->ENSEMBL_WWW_PROXY;
      $params{'https_proxy'} = $args->{'hub'}->species_defs->ENSEMBL_WWW_PROXY;
      }
      my $http = HTTP::Tiny->new(%params);

      my $response = $http->request('HEAD', $url);
      if ($response->{'success'}) {
      $all_headers = $response->{'headers'};
      }
      else {
      $error = _get_http_tiny_error($response);
      }
      }

      $result = $args->{'header'} ? $all_headers->{$args->{'header'}} : $all_headers;

      if ($args->{'nice'}) {
      return $error ? {'error' => [$error]} : {'headers' => $result};
      }
      else {
      if ($error) {
      throw exception('URLException', "Could not get headers.") unless $args->{'no_exception'};
      return 0;
      }
      else {
      return $result;
      }
      }
      }

Sharon

On Mar 24, 2015, at 5:14 AM, Anne Lyle <annelyle at ebi.ac.uk<mailto:annelyle at ebi.ac.uk><mailto:annelyle at ebi.ac.uk><mailto:annelyle at ebi.ac.uk>> wrote:

Hi

We’re aware of this issue, and recently pushed out a fix which sends a GET request instead of HEAD. Have you updated your ensembl-webcode repository in the past few days?

Cheers

Anne

On 23 Mar 2015, at 17:37, Wei, Xuehong <weix at cshl.edu<mailto:weix at cshl.edu><mailto:weix at cshl.edu><mailto:weix at cshl.edu>> wrote:

Hi, Ensembl Developers,

On Gramene genome browser which is build on top of ensemble web infrastructure, users cannot succeed in adding Atlas tracks by url. In the error log, we see error messages like

"Uncaught exception 'URLException' with message 'Could not get headers.'\n  Thrown by EnsEMBL::Web::File::Utils::URL::get_headers in module EnsEMBL::Web::File::Utils::URL at /usr/local/ensembl-live/ensembl-webcode/modules/EnsEMBL/Web/File/Utils/URL.pm on line 302…”

I further trace the source of the error to the following code in function get_header

my $http = HTTP::Tiny->new(%params);
my $response = $http->request('HEAD', $url); ####FAILED

I wrote a test script to test this $http->request and found while ‘HEAD’ returned error, ‘GET’ would succeed (See below). Obviously ensemble genome browser has no problem using ‘HEAD’. So I am wondering whether this is a permission setting on wwwdev.ebi.ac.uk<http://wwwdev.ebi.ac.uk/><http://wwwdev.ebi.ac.uk/><http://wwwdev.ebi.ac.uk<http://wwwdev.ebi.ac.uk/>><http://wwwdev.ebi.ac.uk<http://wwwdev.ebi.ac.uk/>> that restricts the methods from outside request to ‘GET’. If it is, could you grant outside request ‘HEAD’ method too?

Thanks,

#!/bin/env perl

use HTTP::Tiny;
use Data::Dumper;

my $url='http://wwwdev.ebi.ac.uk/gxa/experiments/E-ATMX-20/tracks/E-ATMX-20.g1_g2.genes.log2foldchange.bedGraph';

my $http = HTTP::Tiny->new;

my $response = $http->request('HEAD', $url);

warn (Dumper($response));

perl test.pl

$VAR1 = {
      'success' => '',
      'protocol' => 'HTTP/1.1',
      'headers' => {
                     'content-type' => 'text/html;charset=UTF-8',
                     'set-cookie' => 'JSESSIONID=C3301F4233490280BC8F2658EF225186; Path=/gxa/; HttpOnly',
                     'transfer-encoding' => 'chunked',
                     'date' => 'Sun, 22 Mar 2015 19:13:36 GMT',
                     'vary' => 'Accept-Encoding',
                     'allow' => 'GET',
                     'server' => 'Apache-Coyote/1.1'
                   },
      'status' => '405',
      'url' => 'http://wwwdev.ebi.ac.uk/gxa/experiments/E-ATMX-20/tracks/E-ATMX-20.g1_g2.genes.log2foldchange.bedGraph',
      'reason' => 'Method Not Allowed'
    };

Sharon

_______________________________________________
Dev mailing list    Dev at ensembl.org<mailto:Dev at ensembl.org><mailto:Dev at ensembl.org><mailto:Dev at ensembl.org>
Posting guidelines and subscribe/unsubscribe info: http://lists.ensembl.org/mailman/listinfo/dev
Ensembl Blog: http://www.ensembl.info/

_______________________________________________
Dev mailing list    Dev at ensembl.org<mailto:Dev at ensembl.org><mailto:Dev at ensembl.org><mailto:Dev at ensembl.org>
Posting guidelines and subscribe/unsubscribe info: http://lists.ensembl.org/mailman/listinfo/dev
Ensembl Blog: http://www.ensembl.info/

_______________________________________________
Dev mailing list    Dev at ensembl.org<mailto:Dev at ensembl.org><mailto:Dev at ensembl.org>
Posting guidelines and subscribe/unsubscribe info: http://lists.ensembl.org/mailman/listinfo/dev
Ensembl Blog: http://www.ensembl.info/

_______________________________________________
Dev mailing list    Dev at ensembl.org<mailto:Dev at ensembl.org><mailto:Dev at ensembl.org>
Posting guidelines and subscribe/unsubscribe info: http://lists.ensembl.org/mailman/listinfo/dev
Ensembl Blog: http://www.ensembl.info/

_______________________________________________
Dev mailing list    Dev at ensembl.org<mailto:Dev at ensembl.org>
Posting guidelines and subscribe/unsubscribe info: http://lists.ensembl.org/mailman/listinfo/dev
Ensembl Blog: http://www.ensembl.info/

_______________________________________________
Dev mailing list    Dev at ensembl.org<mailto:Dev at ensembl.org>
Posting guidelines and subscribe/unsubscribe info: http://lists.ensembl.org/mailman/listinfo/dev
Ensembl Blog: http://www.ensembl.info/