ARCHIVE SCIENCE PORTAL

With the new, interactive Archive Science Portal it is possible to access the ESO science archive, the collection and distribution point of the data from the La Silla Paranal Observatory. 


Leave us your feedback on the graphical appearance of the Search Interface and help us in improving its layout and usability.  Ask us questions or give us comments on the search/visualization/retrieval of Phase3 archive data. 

+4
Completed

Search by target - Name resolver

Nausicaa Delmotte 4 years ago in Search for data updated by Archive Science Group 4 years ago 1

I'm starting a new topic. What is the name resolver being used by the graphical search interface?

Is it Sesame at http://cds.u-strasbg.fr/cgi-bin/Sesame

Answer
Archive Science Group 4 years ago

Dear Nausicaa,
yes, indeed the name resolved used by the ESO graphical search interface is Sesame!
Cheers,

Chiara

+2
Completed

Exploring the Follow up of GW events

Archive Science Group 2 years ago in Search for data updated 2 years ago 1


We would like to stress the power of the new version of the archive science portal for detecting electromagnetic counterpart of gravitational wave detections. For instance, you can easily search for all data present in the archive associated to the irregular 50% confidence contours for the location of the GW superevent s191205ah: Polygon-1, Polygon-2, and Polygon-3.

    The procedure followed to obtain the results above is described in our Jupyter notebook. Where you can also find some useful tools to explore the archive science portal and to unleash the full power of the programmatic access. You can find more notebooks here.

    Feedbacks
    0
    Started

    cutouts

    Anonymous 8 months ago in Search for data updated by Archive Science Group 8 months ago 1

    how can I download cutouts of a given size from the Science Portal? I have successfully uploaded a list of target, and they are shown in the bottom of the page. However, from the download page, I cannot select "cutout" as it is grey. 

    Answer
    Archive Science Group 8 months ago

    Dear user,

    There are two different options to upload the list of targets: 

    • You can upload your list of targets here to ensure that the cutout option will be already pre-selected in the Download Portal when downloading the data. (option number one in screenshot)

    • If you upload your target list from the main window, the functionality is the same except for the need to select the cutout option in the Download Portal explicitly. Data volumes displayed in the download portal reflect the data reduction thanks to cutouts in an approximate fashion.

    You can get more information clicking on on the scissors as shown in the screenshot below (option number 2), and more extended explanation if you click on "see dedicated help topic" (option number 3) on the displayed window.

    Once you are in the download portal, regardless of targetlist or single target, one reason for the problem you're describing comes as a result from any search of (1d) spectra by position (+radius) because there is nothing to cutout from the 1-d spectrum based on the given positional constraint. Therefore the cutout service is disabled (=greyed out).

    I hope this answers your question.

    Best regards,

    Archive Science Group

    0
    Started

    Retrieve calibration files for HARPS programatically

    Andrew Jolly 9 months ago in Search for data updated by Archive Science Group 8 months ago 1

    Has anyone had any experience with accessing the ancillary files for a HARPS observation programatically?

    I found a filename in the header of the science file under the key ASSON1, but this filename does not match the files that are downloaded manually from the data archive, and when unpacked are not the same.

    I have found a series of tutorials related to calibration files but I haven't been able to get any to work. If someone has a script that they could share that would be really welcome, thanks!

    Answer
    Archive Science Group 8 months ago

    Dear Andrew,
    The information that you should use the datalink service to get to the ancillary files is provided in the programmatic overview page: http://archive.eso.org/cms/eso-data/programmatic-access.html but I fully agree that that is not enough. I will publish a script for that.
    Thanks for pointing this out. In the meantime...

    Here I provide you preliminary snippets of the code necessary to do that, with explanations.
    As said, it is the datalink service that allows you to find all kinds of files related to the input one.The datalink response for a HARPS calibrated spectrum contains, among others, the ancillary file you want to download (the tar ball).
    As an example, try:

    http://archive.eso.org/datalink/links?ID=ivo://eso.org/ID?ADP.2014-09-16T11:03:30.940

    The ancillary files can be identified by the "semantics" field which must be set to "#auxiliary".
    In python:
    import eso_programmatic.py as eso

    # The eso_programmatic.py
    # is published here: http://archive.eso.org/programmatic/HOWTO/jupyter/authentication_and_authorisation/eso_programmatic.py

    # Let's get the access_url of 3 HARPS products:
    query = """SELECT top 3 access_url from ivoa.ObsCore where obs_collection='HARPS'"""
    res = tap.search(query)
    print(res)

                                          access_url                                     
                                            object                                       
    -------------------------------------------------------------------------------------
    http://archive.eso.org/datalink/links?ID=ivo://eso.org/ID?ADP.2014-09-16T11:03:30.940
    http://archive.eso.org/datalink/links?ID=ivo://eso.org/ID?ADP.2014-09-16T11:03:30.947
    http://archive.eso.org/datalink/links?ID=ivo://eso.org/ID?ADP.2014-09-16T11:03:30.973
    

    # Let's loop through those 3, and for each of them loop through its #auxiliary entries (the tar ball is the only #auxiliary for an HARPS product anyway):
    for rec in (res):
    datalink = vo.dal.adhoc.DatalinkResults.from_result_url(rec['access_url'], session=session)
    ancillaries = datalink.bysemantics('#auxiliary')
    for anc in ancillaries: # for each ancillary, get its access_url, and use it to download the file
    # other useful info available: print(anc['eso_category'], anc['eso_origfile'], anc['content_length'], anc['access_url'])
    status_code, filepath = eso.downloadURL(anc['access_url'], session=session)
    if status_code == 200:
    print("File {0} downloaded as {1}".format(anc['eso_origfile'], filepath))

    The result is:

    File HARPS.2006-08-09T05:48:52.136_DRS_HARPS_3.5.tar downloaded as ./ADP.2014-09-16T11:08:02.037.tar
    File HARPS.2006-01-30T08:42:04.135_DRS_HARPS_3.5.tar downloaded as ./ADP.2014-09-16T11:04:44.533.tar
    File HARPS.2006-07-30T07:45:53.333_DRS_HARPS_3.5.tar downloaded as ./ADP.2014-09-16T11:04:48.567.tar

    If you prefer to download the tar ball with its original name, then add "filename=anc['eso_origfile']" as in:
    status_code, filepath = eso.downloadURL(anc['access_url'], filename=anc['eso_origfile'], session=session)
    and you'll obtain:

    File HARPS.2006-08-09T05:48:52.136_DRS_HARPS_3.5.tar downloaded as ./HARPS.2006-08-09T05:48:52.136_DRS_HARPS_3.5.tar
    File HARPS.2006-01-30T08:42:04.135_DRS_HARPS_3.5.tar downloaded as ./HARPS.2006-01-30T08:42:04.135_DRS_HARPS_3.5.tar
    File HARPS.2006-07-30T07:45:53.333_DRS_HARPS_3.5.tar downloaded as ./HARPS.2006-07-30T07:45:53.333_DRS_HARPS_3.5.tar


    Thanks a lot for reporting the absence of examples on this!

    Alberto Micol

    ESO Archive Science Group


    0
    Started

    Absolute flux calibrated spectra

    Enrique Solano 1 year ago in Search for data updated by Archive Science Group 1 year ago 1

    Dear Helpdesk,

    I would need to find flux callibrated spectra in absolute units but it is not clear to me how to query the archive by this condition.


    Thank you very much in advance for your help.

    0
    Completed

    Extracting reduced data of M dwarfs

    Anonymous 2 years ago in Search for data updated by Archive Science Group 2 years ago 1

    Hi,

    I'm working on the X-Shooter library and M dwarfs. I'm trying to extract the reduced data of M dwarfs resolved by X-Shooter using a list of M dwarfs obtained from SIMBAD. The stars range from M0V-M9.5V. Is there an easy way of doing this programmatically? The list contains many stars that are not resolved by X-Shooter too so it will have to only extract the ones that are present in the ESO archive. I have been downloading the files manually from the website but it has an upper limit of 1000 items per list which has prevented me from using lists containing more than 1000 stars. I would also like to know if there is a way to go around this limit programmatically as well as on the website.

    Many thanks!

    Questions
    Answer
    Archive Science Group 2 years ago

    Dear user,

       many thanks for using the ESO archive system!

    To access to XSHOOTER data programmatically you can use a query similar to this one:

    
    SELECT
        target_name, dp_id, s_ra, s_dec, t_exptime, em_min, em_max, 
        dataproduct_type, instrument_name, obstech, abmaglim,
        proposal_id, obs_collection
    FROM
        ivoa.ObsCore
    WHERE
        (INTERSECTS(CIRCLE('ICRS',109.66824871,-24.55869773,5./3600), s_region)=1 OR
         INTERSECTS(CIRCLE('ICRS',279.729582,6.27016,5./3600), s_region)=1
        )
    AND
        instrument_name LIKE 'XSHOO%'
    
    

    where you can include a list of INTERSECTS conditions connected with the OR logic (see these HOWTOs for further details). With this, however, you will still hit a limit in the number of objects to be queried. This is due to the maximum length of the url that could be sent to the TAP service.

    It is not possible to overcome the 1000 object limit in the Archive Science Portal. If splitting your list in groups containing <1000 items each is not an option for you, an alternative can be found in this forum page. Here with provide a walk-through on how to get data given a list of star names with python. This requires the installation of the unofficial ESOAsg python package.

    Enjoy your data!

    0
    Completed

    How can I search for data from specific ESO programme ID using ESO Science Portal interface?

    Anonymous 2 years ago in Search for data updated by Archive Science Group 2 years ago 1

    I know that specific dataset has been generated from a given Programme ID. I see that Program Id is a selectable item, but the list of available options for X-SHOOTER is way too long to find the right one. Is there a way to type it somewhere in the interface?

    Questions
    Answer
    Archive Science Group 2 years ago

    Archive

    Dear user,
        many thanks for using the Archive Science Portal! There are 3 ways to solve your problem:

    • You can expand the Query Parameter column in the Archive Science Portal (see figure on the right). From this you will be able to perform a more refined search for a given parameter. This is done by clicking on the little show input form arrow below each panel.
    • Alternatively, you can insert the ProgId you are interested in into the ESO XSHOOTER instrument specific query form.
    • Finally, you can also run a TAP query where you select instrument_name='XSHOOTER' and your favourite proposal_id:
      SELECT
          target_name, dp_id, s_ra, s_dec, 
          t_exptime, em_min, em_max, 
          dataproduct_type, instrument_name, 
          obstech, proposal_id, obs_collection, 
          access_url
      FROM
          ivoa.ObsCore
      WHERE
          proposal_id LIKE '084.B-0869%' 
      AND
          instrument_name='XSHOOTER'

    Please let us know if this answer your question! and enjoy your data!

    0
    Completed

    Metallicity data

    Anonymous 2 years ago in Search for data updated 1 year ago 2

    Hi,

    first of all thank you very much for your awesome work.

    I'm new to this data retrieval system and I feel kind of lost in front of it. I'm looking for metallicity data (in particular the [Fe/H] ratio) in the Milky Way bulge: how can I query the various databases for such data?

    Thank you very much again!

    Questions
    Answer
    Archive Science Group 2 years ago

    Dear user,

       many thanks for your question.

      We suggest you to explore our catalogue facility: https://www.eso.org/qi/ where the content of all catalogues currently in the ESO archive is described. Reading your question, it seems that you may be interested in GAIAESO and AMBRE (but please check thoroughly that this is indeed the case and if there are other surveys that may cover your scientific interest).

      To access the content of the catalogue and to download the corresponding data, we provide a variety of options. For instance for GAIAESO you can:

    • query the Archive Science Portal for GAIAESO here;
    • directly explore the different columns of the catalogue here;
    • have access the data programmatically here;
    • use the phase3 page dedicated to the collection here.

    Please let us know if this satisfy your request and/or if you would like to have further assistance.

    Best regards from the Archive Science Group!

    0
    Completed

    Search for targets with target name

    Anonymous 2 years ago in Search for data updated by Archive Science Group 2 years ago 1

    it is not much clear how to search for a target. I tried to look for Sedna spectra, I tried typing Sedna, SEDNA, 90377 Sedna, 90377 SEDNA, and all the possible combinations on upper case, lower case, numbers, with spaces, without spaces, ecc, but I can't get any result. What am i doing wrong? I wasn't neither able to find an instruction page.


    Thanks

    Questions
    Answer
    Archive Science Group 2 years ago

    Dear anonymous user,

    thanks for your message and for using the Forum :)


    The problem seems to be that there are no reduced products available for Sedna. In fact, also the search for coordinates (03:52:37 +07:58:18) produces no result.

    There are a few FORS2/ISAAC observations of SEDNA in the raw data query form (pull-down menu selection in the green area=> "search by OBJECT FITS keyword", or input "sedna" in the "title" field of the pink area)

    Finally, there are also external services like the "Solar System Object Image Search" at CADC that is scanning, among many others, some of the ESO image data collections (*) and returning lists of datasets containing potential observations of the SSOs. This service even provides links to the ESO archive for data download. I found 302 matching images (WFI, EFOSC, ISAAC, FORS).

    I hope this helps, but please do not hesitate to write us if you still have doubts or problems.

    Cheers,

    Chiara (on behalf of ASG)


    PS In the Archive Portal, if you put your mouse arrow on the target window, a small facet with instructions will appear automatically and explain what kind of input is accepted and in which format.

    (*) ESO-LaSilla_2.2m/WFI ESO-NTT/EFOSC ESO-NTT/EMMI ESO-NTT/SOFI ESO-NTT/SUSI ESO-NTT/SUSI2 ESO-VISTA/VIRCAM ESO-VLT/FORS1 ESO-VLT/FORS2 ESO-VLT/HAWKI ESO-VLT/ISAAC ESO-VLT/NAOS-CONICA ESO-VLT/VIMOS ESO-VST/OMEGACAM

    0

    The 'create a link to this page' functionality

    Dear Archive Science Portal users,

    thanks to an anonymous user, we just discovered that there is a small bug with the 'create a link to this page' button. 

    In fact, when one uses this functionality to create a link to a search around given coordinates (with any other constraints),
    the link erroneously points to null coordinates instead. 

    We are already working to solve this. Meanwhile, a smart and quick way around that allows one to get back the right query via the generated link is to click on the red, curved arrow below the coordinate windows and on the right side of the FoV (see attached screenshot).

    After clicking, you will see that your research will point to the right coordinates (and have all the other constraints).


    Many thanks to the anonymous user who spotted this bug.

    We are looking into the problem already and will report here any updates.


    Cheers,

    ASG

     

    Comments