ARCHIVE SCIENCE PORTAL
With the new, interactive Archive Science Portal it is possible to access the ESO science archive, the collection and distribution point of the data from the La Silla Paranal Observatory.
Leave us your feedback on the graphical appearance of the Search Interface and help us in improving its layout and usability. Ask us questions or give us comments on the search/visualization/retrieval of Phase3 archive data.
Search by target - Name resolver
I'm starting a new topic. What is the name resolver being used by the graphical search interface?
Is it Sesame at http://cds.u-strasbg.fr/cgi-bin/Sesame
Dear Nausicaa,
yes, indeed the name resolved used by the ESO graphical search interface is Sesame!
Cheers,
Chiara
Exploring the Follow up of GW events
We would like to stress the power of the new version of the archive science portal for detecting electromagnetic counterpart of gravitational wave detections. For instance, you can easily search for all data present in the archive associated to the irregular 50% confidence contours for the location of the GW superevent s191205ah: Polygon-1, Polygon-2, and Polygon-3.
The procedure followed to obtain the results above is described in our Jupyter notebook. Where you can also find some useful tools to explore the archive science portal and to unleash the full power of the programmatic access. You can find more notebooks here.
How to find data that is not on the archive, but should be.
Hello.
I am searching for some data of symbiotic systems on LMC and there are papers on literature that used ESO to get their data, but It doesn't show here on archive, like the case of https://articles.adsabs.harvard.edu/pdf/1996A%26A...307..516M. In there, they observed a sample of SySt on 1993, but this data does not exist here
Dear Mateus,
Unfortunately these data are not available. Back then data from 3.6m (or 1.54m) was not ingested yet in the ESO Archive.
Sorry we cannot help you.
cutouts
how can I download cutouts of a given size from the Science Portal? I have successfully uploaded a list of target, and they are shown in the bottom of the page. However, from the download page, I cannot select "cutout" as it is grey.
Dear user,
There are two different options to upload the list of targets:
- You can upload your list of targets here to ensure that the cutout option will be already pre-selected in the Download Portal when downloading the data. (option number one in screenshot)
- If you upload your target list from the main window, the functionality is the same except for the need to select the cutout option in the Download Portal explicitly. Data volumes displayed in the download portal reflect the data reduction thanks to cutouts in an approximate fashion.
You can get more information clicking on on the scissors as shown in the screenshot below (option number 2), and more extended explanation if you click on "see dedicated help topic" (option number 3) on the displayed window.
Once you are in the download portal, regardless of targetlist or single target, one reason for the problem you're describing comes as a result from any search of (1d) spectra by position (+radius) because there is nothing to cutout from the 1-d spectrum based on the given positional constraint. Therefore the cutout service is disabled (=greyed out).
I hope this answers your question.
Best regards,
Archive Science Group
Retrieve calibration files for HARPS programatically
Has anyone had any experience with accessing the ancillary files for a HARPS observation programatically?
I found a filename in the header of the science file under the key ASSON1, but this filename does not match the files that are downloaded manually from the data archive, and when unpacked are not the same.
I have found a series of tutorials related to calibration files but I haven't been able to get any to work. If someone has a script that they could share that would be really welcome, thanks!
Dear Andrew,
The information that you should use the datalink service to get to the ancillary files is provided in the programmatic overview page: http://archive.eso.org/cms/eso-data/programmatic-access.html but I fully agree that that is not enough. I will publish a script for that.
Thanks for pointing this out. In the meantime...
Here I provide you preliminary snippets of the code necessary to do that, with explanations.
As said, it is the datalink service that allows you to find all kinds of files related to the input one.The datalink response for a HARPS calibrated spectrum contains, among others, the ancillary file you want to download (the tar ball).
As an example, try:
http://archive.eso.org/datalink/links?ID=ivo://eso.org/ID?ADP.2014-09-16T11:03:30.940
The ancillary files can be identified by the "semantics" field which must be set to "#auxiliary".
In python:
import eso_programmatic.py as eso
# The eso_programmatic.py
# is published here: http://archive.eso.org/programmatic/HOWTO/jupyter/authentication_and_authorisation/eso_programmatic.py
# Let's get the access_url of 3 HARPS products:
query = """SELECT top 3 access_url from ivoa.ObsCore where obs_collection='HARPS'"""
res = tap.search(query)
print(res)
access_url object ------------------------------------------------------------------------------------- http://archive.eso.org/datalink/links?ID=ivo://eso.org/ID?ADP.2014-09-16T11:03:30.940 http://archive.eso.org/datalink/links?ID=ivo://eso.org/ID?ADP.2014-09-16T11:03:30.947 http://archive.eso.org/datalink/links?ID=ivo://eso.org/ID?ADP.2014-09-16T11:03:30.973
# Let's loop through those 3, and for each of them loop through its #auxiliary entries (the tar ball is the only #auxiliary for an HARPS product anyway):
for rec in (res):
datalink = vo.dal.adhoc.DatalinkResults.from_result_url(rec['access_url'], session=session)
ancillaries = datalink.bysemantics('#auxiliary')
for anc in ancillaries: # for each ancillary, get its access_url, and use it to download the file
# other useful info available: print(anc['eso_category'], anc['eso_origfile'], anc['content_length'], anc['access_url'])
status_code, filepath = eso.downloadURL(anc['access_url'], session=session)
if status_code == 200:
print("File {0} downloaded as {1}".format(anc['eso_origfile'], filepath))
The result is:
File HARPS.2006-08-09T05:48:52.136_DRS_HARPS_3.5.tar downloaded as ./ADP.2014-09-16T11:08:02.037.tar File HARPS.2006-01-30T08:42:04.135_DRS_HARPS_3.5.tar downloaded as ./ADP.2014-09-16T11:04:44.533.tar File HARPS.2006-07-30T07:45:53.333_DRS_HARPS_3.5.tar downloaded as ./ADP.2014-09-16T11:04:48.567.tar
If you prefer to download the tar ball with its original name, then add "filename=anc['eso_origfile']" as in:
status_code, filepath = eso.downloadURL(anc['access_url'], filename=anc['eso_origfile'], session=session)
and you'll obtain:
File HARPS.2006-08-09T05:48:52.136_DRS_HARPS_3.5.tar downloaded as ./HARPS.2006-08-09T05:48:52.136_DRS_HARPS_3.5.tar File HARPS.2006-01-30T08:42:04.135_DRS_HARPS_3.5.tar downloaded as ./HARPS.2006-01-30T08:42:04.135_DRS_HARPS_3.5.tar File HARPS.2006-07-30T07:45:53.333_DRS_HARPS_3.5.tar downloaded as ./HARPS.2006-07-30T07:45:53.333_DRS_HARPS_3.5.tar
Thanks a lot for reporting the absence of examples on this!
Alberto Micol
Absolute flux calibrated spectra
Dear Helpdesk,
I would need to find flux callibrated spectra in absolute units but it is not clear to me how to query the archive by this condition.
Thank you very much in advance for your help.
Extracting reduced data of M dwarfs
Hi,
I'm working on the X-Shooter library and M dwarfs. I'm trying to extract the reduced data of M dwarfs resolved by X-Shooter using a list of M dwarfs obtained from SIMBAD. The stars range from M0V-M9.5V. Is there an easy way of doing this programmatically? The list contains many stars that are not resolved by X-Shooter too so it will have to only extract the ones that are present in the ESO archive. I have been downloading the files manually from the website but it has an upper limit of 1000 items per list which has prevented me from using lists containing more than 1000 stars. I would also like to know if there is a way to go around this limit programmatically as well as on the website.
Many thanks!
Dear user,
many thanks for using the ESO archive system!
To access to XSHOOTER data programmatically you can use a query similar to this one:
SELECT
target_name, dp_id, s_ra, s_dec, t_exptime, em_min, em_max,
dataproduct_type, instrument_name, obstech, abmaglim,
proposal_id, obs_collection
FROM
ivoa.ObsCore
WHERE
(INTERSECTS(CIRCLE('ICRS',109.66824871,-24.55869773,5./3600), s_region)=1 OR
INTERSECTS(CIRCLE('ICRS',279.729582,6.27016,5./3600), s_region)=1
)
AND
instrument_name LIKE 'XSHOO%'
where you can include a list of INTERSECTS conditions connected with the OR logic (see these HOWTOs for further details). With this, however, you will still hit a limit in the number of objects to be queried. This is due to the maximum length of the url that could be sent to the TAP service.
It is not possible to overcome the 1000 object limit in the Archive Science Portal. If splitting your list in groups containing <1000 items each is not an option for you, an alternative can be found in this forum page. Here with provide a walk-through on how to get data given a list of star names with python. This requires the installation of the unofficial ESOAsg python package.
Enjoy your data!
How can I search for data from specific ESO programme ID using ESO Science Portal interface?
I know that specific dataset has been generated from a given Programme ID. I see that Program Id is a selectable item, but the list of available options for X-SHOOTER is way too long to find the right one. Is there a way to type it somewhere in the interface?
Dear user,
many thanks for using the Archive Science Portal! There are 3 ways to solve your problem:
- You can expand the Query Parameter column in the Archive Science Portal (see figure on the right). From this you will be able to perform a more refined search for a given parameter. This is done by clicking on the little show input form arrow below each panel.
- Alternatively, you can insert the ProgId you are interested in into the ESO XSHOOTER instrument specific query form.
- Finally, you can also run a TAP query where you select
instrument_name='XSHOOTER'
and your favouriteproposal_id
:SELECT target_name, dp_id, s_ra, s_dec, t_exptime, em_min, em_max, dataproduct_type, instrument_name, obstech, proposal_id, obs_collection, access_url FROM ivoa.ObsCore WHERE proposal_id LIKE '084.B-0869%' AND instrument_name='XSHOOTER'
Please let us know if this answer your question! and enjoy your data!
Metallicity data
Hi,
first of all thank you very much for your awesome work.
I'm new to this data retrieval system and I feel kind of lost in front of it. I'm looking for metallicity data (in particular the [Fe/H] ratio) in the Milky Way bulge: how can I query the various databases for such data?
Thank you very much again!
Dear user,
many thanks for your question.
We suggest you to explore our catalogue facility: https://www.eso.org/qi/ where the content of all catalogues currently in the ESO archive is described. Reading your question, it seems that you may be interested in GAIAESO and AMBRE (but please check thoroughly that this is indeed the case and if there are other surveys that may cover your scientific interest).
To access the content of the catalogue and to download the corresponding data, we provide a variety of options. For instance for GAIAESO you can:
- query the Archive Science Portal for GAIAESO here;
- directly explore the different columns of the catalogue here;
- have access the data programmatically here;
- use the phase3 page dedicated to the collection here.
Please let us know if this satisfy your request and/or if you would like to have further assistance.
Best regards from the Archive Science Group!
Search for targets with target name
it is not much clear how to search for a target. I tried to look for Sedna spectra, I tried typing Sedna, SEDNA, 90377 Sedna, 90377 SEDNA, and all the possible combinations on upper case, lower case, numbers, with spaces, without spaces, ecc, but I can't get any result. What am i doing wrong? I wasn't neither able to find an instruction page.
Thanks
Dear anonymous user,
thanks for your message and for using the Forum :)
The problem seems to be that there are no reduced products available for Sedna. In fact, also the search for coordinates (03:52:37 +07:58:18) produces no result.
There are a few FORS2/ISAAC observations of SEDNA in the raw data query form (pull-down menu selection in the green area=> "search by OBJECT FITS keyword", or input "sedna" in the "title" field of the pink area)
Finally, there are also external services like the "Solar System Object Image Search" at CADC that is scanning, among many others, some of the ESO image data collections (*) and returning lists of datasets containing potential observations of the SSOs. This service even provides links to the ESO archive for data download. I found 302 matching images (WFI, EFOSC, ISAAC, FORS).
I hope this helps, but please do not hesitate to write us if you still have doubts or problems.
Cheers,
Chiara (on behalf of ASG)
PS In the Archive Portal, if you put your mouse arrow on the target window, a small facet with instructions will appear automatically and explain what kind of input is accepted and in which format.
(*) ESO-LaSilla_2.2m/WFI ESO-NTT/EFOSC ESO-NTT/EMMI ESO-NTT/SOFI ESO-NTT/SUSI ESO-NTT/SUSI2 ESO-VISTA/VIRCAM ESO-VLT/FORS1 ESO-VLT/FORS2 ESO-VLT/HAWKI ESO-VLT/ISAAC ESO-VLT/NAOS-CONICA ESO-VLT/VIMOS ESO-VST/OMEGACAM
Customer support service by UserEcho