ARCHIVE SCIENCE PORTAL

With the new, interactive Archive Science Portal it is possible to access the ESO science archive, the collection and distribution point of the data from the La Silla Paranal Observatory. 


Leave us your feedback on the graphical appearance of the Search Interface and help us in improving its layout and usability.  Ask us questions or give us comments on the search/visualization/retrieval of Phase3 archive data. 

+4
Completed

Search by target - Name resolver

Nausicaa Delmotte 6 years ago in Search for data updated by Archive Science Group 6 years ago 1

I'm starting a new topic. What is the name resolver being used by the graphical search interface?

Is it Sesame at http://cds.u-strasbg.fr/cgi-bin/Sesame

Answer
Archive Science Group 6 years ago

Dear Nausicaa,
yes, indeed the name resolved used by the ESO graphical search interface is Sesame!
Cheers,

Chiara

+3
Completed

Exploring the Follow up of GW events

Archive Science Group 4 years ago in Search for data updated 4 years ago 1

Image 65


We would like to stress the power of the new version of the archive science portal for detecting electromagnetic counterpart of gravitational wave detections. For instance, you can easily search for all data present in the archive associated to the irregular 50% confidence contours for the location of the GW superevent s191205ah: Polygon-1, Polygon-2, and Polygon-3.

    The procedure followed to obtain the results above is described in our Jupyter notebook. Where you can also find some useful tools to explore the archive science portal and to unleash the full power of the programmatic access. You can find more notebooks here.

    Feedbacks
    +3
    Completed

    New version of the Archive Science Portal -> V2.0

    Archive Science Group 4 years ago in Layout, Appearance and color scheme updated 4 years ago 1

    Dear users,

    we are excited to announce that ESO Archive Science Portal has been substantially upgraded. The full list of improvement present in the new version 2.0 has been announced to the scientific community here.



    Following users feedback, the updated Archive Science Portal provides (among others) the following improvements:

    • A single interface allows users to access data from La Silla Paranal, APEX ,and -now- from ALMA as well:
      Whale Galaxy
      See for instance here the ALMA footprint over the WHALE galaxy.
      These ALMA data can be retrieved also with this programmatic query:
      SELECT
      s_ra, s_dec, dp_id, instrument_name, obstech, proposal_id, obs_release_date
      FROM
      ivoa.ObsCore
      WHERE
      intersects(s_region, circle('',3.72333,-39.19667,0.3))=1 AND instrument_name='ALMA'


    • We now provide previews for datacubes, including interactive spectrum extraction:
      Muse Spectrum
      Users can now extract a quick look MUSE spectrum directly from the Archive Interface

    • Search within user defined polygons:
      Jewel Box
      For instance an user can download all the spectra of the stars in the Jewel Box cluster within a customized region. Naturally this is possible also programmatically:
      SELECT
      s_ra, s_dec, dp_id, instrument_name, obstech, proposal_id, obs_release_date
      FROM
      ivoa.ObsCore
      WHERE
      intersects(s_region, POLYGON( 'J2000', 193.15859,-60.21354,
      193.40507, -60.15885,
      193.80660, -60.21094,
      193.85447, -60.38943,
      193.61956, -60.49661,
      193.21774, -60.49711,
      193.00831, -60.43487,
      193.04188, -60.33450,
      193.03932, -60.26952) )=1 AND obstech NOT LIKE 'IMAGE%'


    Feedbacks
    +2
    Completed

    New version of the Archive Science Portal

    Archive Science Group 5 years ago in Layout, Appearance and color scheme updated 5 years ago 1

    Dear users,
    we are happy to announce that a new version of the Archive Science Portal has been released and announced to the scientific community here

    Based on user feedback a new view mode has been implemented as a central new feature in addition to the already existing sky view (left-hand panel of figure above). It allows users to inspect, at a glance, the multi-dimensional query space in up to seventeen 1D distributions (right-hand panel of figure). They include physical parameters like sensitivity and signal-to-noise ratio, alongside more traditional ones such as Programme ID or instrumental setup. Users can easily interact with any list or histogram to add or remove search constraints and to build up rather complex queries in a progressive fashion.  

    Beyond that, the updated Archive Science Portal provides the following improvements:

    • the capability of bookmarking and sharing queries;
    • customizable set of query parameters and management of query constraints;
    • improved layout and support for mobile and touch devices;
    • several bug fixes.

    Image 15

    0
    Completed

    How to find data that is not on the archive, but should be.

    Mateus 6 months ago in Search for data updated by Archive Science Group 6 months ago 2

    Hello.

    I am searching for some data of symbiotic systems on LMC and there are papers on literature that used ESO to get their data, but It doesn't show here on archive, like the case of https://articles.adsabs.harvard.edu/pdf/1996A%26A...307..516M. In there, they observed a sample of SySt on 1993, but this data does not exist here 

    Comments Feedbacks Questions
    Answer
    Archive Science Group 6 months ago

    Dear Mateus,

    Unfortunately these data are not available. Back then data from 3.6m (or 1.54m) was not ingested yet in the ESO Archive.

    Sorry we cannot help you.

    0
    Started

    cutouts

    Anonymous 2 years ago in Search for data updated by Archive Science Group 2 years ago 1

    how can I download cutouts of a given size from the Science Portal? I have successfully uploaded a list of target, and they are shown in the bottom of the page. However, from the download page, I cannot select "cutout" as it is grey. 

    Answer
    Archive Science Group 2 years ago

    Dear user,

    There are two different options to upload the list of targets: 

    • You can upload your list of targets here to ensure that the cutout option will be already pre-selected in the Download Portal when downloading the data. (option number one in screenshot)

    • If you upload your target list from the main window, the functionality is the same except for the need to select the cutout option in the Download Portal explicitly. Data volumes displayed in the download portal reflect the data reduction thanks to cutouts in an approximate fashion.

    You can get more information clicking on on the scissors as shown in the screenshot below (option number 2), and more extended explanation if you click on "see dedicated help topic" (option number 3) on the displayed window.

    Once you are in the download portal, regardless of targetlist or single target, one reason for the problem you're describing comes as a result from any search of (1d) spectra by position (+radius) because there is nothing to cutout from the 1-d spectrum based on the given positional constraint. Therefore the cutout service is disabled (=greyed out).

    I hope this answers your question.

    Best regards,

    Archive Science Group

    0
    Started

    Retrieve calibration files for HARPS programatically

    Andrew Jolly 2 years ago in Search for data updated by Archive Science Group 2 years ago 1

    Has anyone had any experience with accessing the ancillary files for a HARPS observation programatically?

    I found a filename in the header of the science file under the key ASSON1, but this filename does not match the files that are downloaded manually from the data archive, and when unpacked are not the same.

    I have found a series of tutorials related to calibration files but I haven't been able to get any to work. If someone has a script that they could share that would be really welcome, thanks!

    Answer
    Archive Science Group 2 years ago

    Dear Andrew,
    The information that you should use the datalink service to get to the ancillary files is provided in the programmatic overview page: http://archive.eso.org/cms/eso-data/programmatic-access.html but I fully agree that that is not enough. I will publish a script for that.
    Thanks for pointing this out. In the meantime...

    Here I provide you preliminary snippets of the code necessary to do that, with explanations.
    As said, it is the datalink service that allows you to find all kinds of files related to the input one.The datalink response for a HARPS calibrated spectrum contains, among others, the ancillary file you want to download (the tar ball).
    As an example, try:

    http://archive.eso.org/datalink/links?ID=ivo://eso.org/ID?ADP.2014-09-16T11:03:30.940

    The ancillary files can be identified by the "semantics" field which must be set to "#auxiliary".
    In python:
    import eso_programmatic.py as eso

    # The eso_programmatic.py
    # is published here: http://archive.eso.org/programmatic/HOWTO/jupyter/authentication_and_authorisation/eso_programmatic.py

    # Let's get the access_url of 3 HARPS products:
    query = """SELECT top 3 access_url from ivoa.ObsCore where obs_collection='HARPS'"""
    res = tap.search(query)
    print(res)

                                          access_url                                     
                                            object                                       
    -------------------------------------------------------------------------------------
    http://archive.eso.org/datalink/links?ID=ivo://eso.org/ID?ADP.2014-09-16T11:03:30.940
    http://archive.eso.org/datalink/links?ID=ivo://eso.org/ID?ADP.2014-09-16T11:03:30.947
    http://archive.eso.org/datalink/links?ID=ivo://eso.org/ID?ADP.2014-09-16T11:03:30.973
    

    # Let's loop through those 3, and for each of them loop through its #auxiliary entries (the tar ball is the only #auxiliary for an HARPS product anyway):
    for rec in (res):
    datalink = vo.dal.adhoc.DatalinkResults.from_result_url(rec['access_url'], session=session)
    ancillaries = datalink.bysemantics('#auxiliary')
    for anc in ancillaries: # for each ancillary, get its access_url, and use it to download the file
    # other useful info available: print(anc['eso_category'], anc['eso_origfile'], anc['content_length'], anc['access_url'])
    status_code, filepath = eso.downloadURL(anc['access_url'], session=session)
    if status_code == 200:
    print("File {0} downloaded as {1}".format(anc['eso_origfile'], filepath))

    The result is:

    File HARPS.2006-08-09T05:48:52.136_DRS_HARPS_3.5.tar downloaded as ./ADP.2014-09-16T11:08:02.037.tar
    File HARPS.2006-01-30T08:42:04.135_DRS_HARPS_3.5.tar downloaded as ./ADP.2014-09-16T11:04:44.533.tar
    File HARPS.2006-07-30T07:45:53.333_DRS_HARPS_3.5.tar downloaded as ./ADP.2014-09-16T11:04:48.567.tar

    If you prefer to download the tar ball with its original name, then add "filename=anc['eso_origfile']" as in:
    status_code, filepath = eso.downloadURL(anc['access_url'], filename=anc['eso_origfile'], session=session)
    and you'll obtain:

    File HARPS.2006-08-09T05:48:52.136_DRS_HARPS_3.5.tar downloaded as ./HARPS.2006-08-09T05:48:52.136_DRS_HARPS_3.5.tar
    File HARPS.2006-01-30T08:42:04.135_DRS_HARPS_3.5.tar downloaded as ./HARPS.2006-01-30T08:42:04.135_DRS_HARPS_3.5.tar
    File HARPS.2006-07-30T07:45:53.333_DRS_HARPS_3.5.tar downloaded as ./HARPS.2006-07-30T07:45:53.333_DRS_HARPS_3.5.tar


    Thanks a lot for reporting the absence of examples on this!

    Alberto Micol

    ESO Archive Science Group


    0
    Started

    Absolute flux calibrated spectra

    Enrique Solano 3 years ago in Search for data updated by Archive Science Group 3 years ago 1

    Dear Helpdesk,

    I would need to find flux callibrated spectra in absolute units but it is not clear to me how to query the archive by this condition.


    Thank you very much in advance for your help.

    0
    Completed

    Extracting reduced data of M dwarfs

    Anonymous 4 years ago in Search for data updated by Archive Science Group 3 years ago 1

    Hi,

    I'm working on the X-Shooter library and M dwarfs. I'm trying to extract the reduced data of M dwarfs resolved by X-Shooter using a list of M dwarfs obtained from SIMBAD. The stars range from M0V-M9.5V. Is there an easy way of doing this programmatically? The list contains many stars that are not resolved by X-Shooter too so it will have to only extract the ones that are present in the ESO archive. I have been downloading the files manually from the website but it has an upper limit of 1000 items per list which has prevented me from using lists containing more than 1000 stars. I would also like to know if there is a way to go around this limit programmatically as well as on the website.

    Many thanks!

    Questions
    Answer
    Archive Science Group 4 years ago

    Dear user,

       many thanks for using the ESO archive system!

    To access to XSHOOTER data programmatically you can use a query similar to this one:

    
    SELECT
        target_name, dp_id, s_ra, s_dec, t_exptime, em_min, em_max, 
        dataproduct_type, instrument_name, obstech, abmaglim,
        proposal_id, obs_collection
    FROM
        ivoa.ObsCore
    WHERE
        (INTERSECTS(CIRCLE('ICRS',109.66824871,-24.55869773,5./3600), s_region)=1 OR
         INTERSECTS(CIRCLE('ICRS',279.729582,6.27016,5./3600), s_region)=1
        )
    AND
        instrument_name LIKE 'XSHOO%'
    
    

    where you can include a list of INTERSECTS conditions connected with the OR logic (see these HOWTOs for further details). With this, however, you will still hit a limit in the number of objects to be queried. This is due to the maximum length of the url that could be sent to the TAP service.

    It is not possible to overcome the 1000 object limit in the Archive Science Portal. If splitting your list in groups containing <1000 items each is not an option for you, an alternative can be found in this forum page. Here with provide a walk-through on how to get data given a list of star names with python. This requires the installation of the unofficial ESOAsg python package.

    Enjoy your data!

    0
    Completed

    How can I search for data from specific ESO programme ID using ESO Science Portal interface?

    Anonymous 4 years ago in Search for data updated by Archive Science Group 3 years ago 1

    I know that specific dataset has been generated from a given Programme ID. I see that Program Id is a selectable item, but the list of available options for X-SHOOTER is way too long to find the right one. Is there a way to type it somewhere in the interface?

    Questions
    Answer
    Archive Science Group 4 years ago

    Archive

    Dear user,
        many thanks for using the Archive Science Portal! There are 3 ways to solve your problem:

    • You can expand the Query Parameter column in the Archive Science Portal (see figure on the right). From this you will be able to perform a more refined search for a given parameter. This is done by clicking on the little show input form arrow below each panel.
    • Alternatively, you can insert the ProgId you are interested in into the ESO XSHOOTER instrument specific query form.
    • Finally, you can also run a TAP query where you select instrument_name='XSHOOTER' and your favourite proposal_id:
      SELECT
          target_name, dp_id, s_ra, s_dec, 
          t_exptime, em_min, em_max, 
          dataproduct_type, instrument_name, 
          obstech, proposal_id, obs_collection, 
          access_url
      FROM
          ivoa.ObsCore
      WHERE
          proposal_id LIKE '084.B-0869%' 
      AND
          instrument_name='XSHOOTER'

    Please let us know if this answer your question! and enjoy your data!