All notable changes to
sentinelsat will be listed here.
[1.1.1] – 2021-11-23¶
- Show help when no CLI arguments are provided.
- Tests are no longer included in the Python source package to significantly reduce its size.
- Fixed handling of missing Online OData attribute in older DHuS versions, such as https://coda.eumetsat.int. (#535 @valgur)
- Fix the
filter_nodefunction generated by
make_path_filterso that node paths are no longer converted to lowercase before pattern matching. (#541 @avalentino)
download_all(), the initial check on files already existing now ignores server errors when determining the file name. Any existing files are still skipped when initiating the download. (#543 @joooeey)
[1.1.0] – 2021-08-19¶
download_all() has been completely rewritten to correctly handle server-side concurrency limits and bugs.
- New CLI option
--fmtto specify the format of the listed products. (#526 @avalentino)
- More detailed progress info during downloads. Added progressbars for overall download and LTA retrieval progress. Product names are now included in individual download progress bars. (#532 @valgur)
- Progressbars no longer conflict with logging output. (#531 @valgur)
--fail-fastoption for downloading. (#532)
SentinelAPIwhich are used to ensure that the server-side limit for concurrent downloads (4 for scihub.copernicus.eu) and the maximum number of concurrent LTA retrievals (set to 10 by default) are respected everywhere. (#532)
- Added an optional
SentinelAPI, which sets the maximum time to wait for products to be retrieved from the LTA. Defaults to unlimited. (#532)
SentinelAPI, which defaults to 10 seconds and limits the rate of download retries to give the server some time to recover. (#532)
- Added support for SHA3-256 checksums used by some newer Sentinel-2 products. (#523 @valgur)
- For more advanced use-cases,
Downloader.download_all(), which returns a detailed
DownloadStatusenum for each product, can now be used. (#532)
- GeoJSON geometries consisting of multiple features are now supported. (#530 @mackland)
download_all()now waits for all LTA retrievals to finish before exiting. (#532)
download_all()increased from 2 to 4. (#532)
- Default LTA retry period was reduced from 600 to 60 seconds. (#532)
SentinelProductsAPIhas been merged into
SentinelAPIand will be removed in a future release. (#532)
- Suppressed the unnecessary server stacktrace printing for
QueryLengthError. (#524 @valgur)
- Changed logic for
_get_filenamewhen Content-Disposition header not present (#494 @rbrishabh)
download_all()strictly tracks and limits the number of concurrent GET requests made (which includes LTA triggering) to respect server-side limits. (#532, #508 @avalentino).
- Triggering is now much more robust against server-side bugs (such as the frequent HTTP 500: NullPonterException, Postgres bugs, etc.). (#532)
- Better handling of HTTP 403 error in
- Downloading with
download_all()can now be cleanly interrupted without the downloads remaining active in the background. (#481, #532 @valgur)
[1.0.1] – 2021-05-25¶
[1.0.0] – 2021-05-19¶
- Dropped support for Python 2.7. Now setuptools requires Python >= 3.6.
--endCLI arguments no longer use the default values of
NOW. (#459 @valgur)
--footprintsnow expects a path for the output file to be specified instead of using the previous default
<--path>/search_footprints.geojson. (#462 @valgur, #407 @IpsumCapra)
--querymust now be repeated instead of passing a single comma-separated string. Repeated keywords get treated as an “or” expression. (#463 @valgur)
- Display DHuS server version with CLI flag
- Added searching by placenames with the CLI flag
- Added CLI support for
--geometryinput as a WKT or GeoJSON string (#381 @thomasyoung-audet)
- Download quicklooks directly with the CLI flag
- Got rid of the special handling of
--nameCLI arguments. The product IDs are now simply passed to
api.query()as a set. As a result they no longer ignore the date range arguments (fixes #387). (#390 @valgur)
--timeoutoption with a default value of 60 seconds to avoid waiting indefinitely for a response. (#475 @valgur)
--gnssCLI flag to use the GNSS end-point for orbit products query and download (#441 @avalentino).
sentinelsat/products.pymodule providing a “product nodes” API that allows to filter and download only selected files of the requested products (#414 @avalentino)
trigger_offline_retrieval(). (#476 @valgur)
SentinelAPIErrorexceptions with more specific types:
SentinelAPIError– the parent, catch-all exception. Only used when no other more specific exception can be applied.
ServerError– raised when the server responded in an unexpected manner, typically due to undergoing maintenance.
UnauthorizedError– raised when attempting to retrieve a product with incorrect credentials.
QuerySyntaxError– raised when the query string could not be parsed on the server side.
QueryLengthError– raised when the query string length was excessively long.
InvalidKeyError– raised when product with given key was not found on the server.
InvalidChecksumError– MD5 checksum of a local file does not match the one from the server.
LTAError– raised when triggering a retrieval from the Long Term Archive failed.
LTATriggered– raised in some cases when the product is offline and retrieval was triggered successfully.
Use the HTTP status instead of OData metadata to determine the online status of a product when downloading. This is a workaround for the rare server-side bug of the OData info for the online status being incorrect (#467). (#469 @valgur)
LTAErrorif the requested product is offline. (#476 @valgur) This is a workaround for the rare server-side bug of the OData info for the online status being incorrect (#467). (#469 @valgur)
Search queries now use GET instead of POST requests because the Sentinel-5P Pre-Operations Data Hub does not support POST (#380). As a consequence, the search query contents are now limited to latin1 encoding instead of UTF-8. (#473 @valgur)
query_raw()that was deprecated in v0.12.
- Fix location information for Nominatim bounding box queries (#384)
- Get file name extension more reliably from either the HTTP header or an OData attribute. (#378, #472 @valgur)
- Updated the API Hub URL to https://apihub.copernicus.eu/apihub/.
- Server-side error info has become much more detailed and the client code has been updated to correctly handle that.
check_existing()now determines the filename correctly for Sentinel-5 products. (@valgur #452)
- Fix accidental downloading of the whole product in memory when the product is actually available despite being marked as offline in its metadata. (#386, #421, #454 @lucadelu)
- Fixed timeout not being used in some queries. (#454 @valgur)
[0.14] – 2020-06-12¶
- discontinued support for Python <=3.4
- code formatting with black checked by Travis-CI (#352)
- reorganize unit tests into small groups with their own files (#287)
- reduced code duplication in unit tests by making greater use of pytest fixtures. (#287)
- force unit tests to include one of the markers ‘fast’, ‘scihub’ or ‘mock_api’ (#287)
- automatic return code checking of CLI tests (#287)
- Replaced direct
vcrpyusage in unit tests with
pytestcommand line options changed from
--vcr-record [once|record_new|all. See vcrpy docs for details. (#283)
[0.13] – 2019-04-05¶
- Query keywords with interval ranges now also support single-sided ranges by using
'*'to denote no bound, for example
query(date=(None, 'NOW-1YEAR')). If both bounds are set to unlimited, the keyword will be removed from the query. (#210)
- Raise an exception in case of duplicate keywords present in a query. Case is ignored to match the server-side behavior. (#210)
- Support for Python 3.7
- Support for GeoJSON files with a single
- Added support for Unicode symbols in search queries. (#230)
- Raise ValueError exception if longitude is outside [-180, 180] or latitude is outside [-90, 90] (#236, #218 @Andrey-Raspopov)
timeoutattribute to avoid indefinite wait on response from the server (#256, @viktorbahr)
- Parsing the
IngestionDatefields of an OData response
- Trying to download an offline product from the Copernicus Open Access Hub triggers its retrieval from the long term archive. Downloading of the product is not scheduled.
- Added support for downloading Sentinel 5P data in the CLI via the ‘–sentinel 5’ flag
- Add support in the CLI for reading credentials from ~/.netrc and document existing functionality in the API (#90)
- Spaces in query parameter values are now handled correctly be escaping them with a backslash, where appropriate. (#169, #211)
- Fixed some CLI errors not returning a non-zero exit code. (#209)
- Fixed typo for
area_relationquery parameter documentation from
'Intersects'. (#225 @scottstanie)
check_query_length()logic to match the changed server-side behavior. (#230)
- Clarify usage of GeoJSON files with CLI in docs (#229 @psal93)
to_geopandas()now returns an empty GeoDataFrame for an empty product list input.
[docs]with a single
[dev]installation extras target. (#208)
- Adapted .travis.yml to build fiona and pyproj from source for Python 3.7.
- Minimum pytest version
pytest >= 3.6.3required by
- The existing practice of not accessing the network from unit tests, unless running with
--vcr reset, is now enforced by throwing a
SocketBlockedErrorin such cases. (#207)
[0.12.2] – 2018-06-20¶
- environment variables
SENTINEL_PASSWORDare superceded by
- Updated handling of invalid queries. An exception is raised in such cases. #168
order_byparameter being ignored in queries that require multiple subqueries (that is, queries that return more than 100 products) (#200)
- Special handling of quote symbols in query strings due to a server-side error is no longer necessary and has been removed. #168
- Updated effective query length calculation in
check_query_length()to reflect server-side changes.
- skip failing tests on optional dependency Pandas for Python 3.3 and 3.4
- Unit tests work irrespective of the directory they are run from.
[0.12.1] – 2017-10-24¶
requestsencoding to UTF8
- fixed a backwards incompatible change in the
- inconsistent documentation on the use of range parameters such as
[0.12.0] – 2017-08-10¶
- Option to change the type of spatial relation for the AOI in
query(). The choices are ‘Interesects’, ‘Contains’ and ‘IsWithin’.
query()which controls the fields by which the products are sorted on the server side before being returned.
-o/--order-byon the CLI.
limitthe number of products returned by
query()and to set the number of products to skip via
-l/--limiton the CLI.
query()to append any additional raw query string to the query.
- Query parameters that take intervals as values can now be passed a tuple of the interval range values.
- Date validation and parsing has been extended to all date-type parameters in queries, such as ‘ingestiondate’.
count()which quickly returns the number of products matching a query on the server without retrieving the full response.
check_query_lengthto check if a query will fail because of being excessively long.
- Option to adjust the number of decimal figures in the coordinates of the WKT string returned by
- CLI option to query by UUID (
--uuid) or filename (
- A more informative error message is shown if a too long query string was likely the cause of the query failing on the server side. This can be useful if the WKT string length would cause the query to fail otherwise.
- Progressbars can be disabled by setting
False. Progressbars may be customized by overriding the
- Contribution guidelines.
- Tests for validity of documentation and RST files.
- Merged CLI subcommands
- CLI uses keywords instead of positional arguments, i.e.
query()have been replaced with a single
dateparameter that takes a tuple of start and end dates as input.
- Files being downloaded now include an ‘.incomplete’ suffix in their name until the download is finished.
download_all(). Similar functionality has been provided in the new
format_query_datehas been changed into a public function.
- Added a progressbar to long-running queries.
- Tests can now be run from any directory rather than the repository root.
- Made the query string slightly more compact by getting rid of unnecessary ‘AND’ operators, spaces and parentheses.
- Reduced the size of the VCR.py cassettes used in unit tests.
- changed license from AGPLv3 to GPLv3+
query_raw()has been merged with
query()and is deprecated. Use
[0.11] – 2017-06-01¶
requests. This makes installation significantly easier. (#117)
- An exception is raised in
download_all()if all downloads failed.
- Change ‘Sentinels Scientific Datahub’ to ‘Copernicus Open Access Hub’ (#100)
--vcr resetto better reflect its true behavior.
[0.10] – 2017-05-30¶
- GeoJSON footprints are allowed to contain just a single geometry instead of a feature collection. Any geometry type that has a WKT equivalent is supported (rather than only Polygons).
get_product_odata()can be used to get the full metadata information available for a product if
query_raw()that takes full text search string as input and returns a parsed dictionary just like the updated
--sentinel=<int>option to select satellite (constellation)
SentinelAPI, etc. can be directly imported from
areaargument expects a WKT string as input instead of a coordinate string. (Issue #101)
- Date arguments can be disabled by setting them to
Noneand their values are validated on the client side. (Issue #101)
- The return value has been changed to a dict of dicts of parsed metadata values. One entry per product with the product ID as the key.
download_all()expects a list of product IDs as input. This is compatible with the output of
get_coordinates()has been replaced with functions
geojson_to_wkt(). (Issue #101)
- Use more compact and descriptive error messages from the response headers, if available.
--sentinel2will be removed with the next major release
to_dict()has been removed since it is no longer required.
load_query()has been made private (renamed to
[0.9.1] – 2017-03-06¶
--versionoption to command line utilities
- install requirements for building the documentation
- documentation of sorting with
[0.9] – 2017-02-26¶
to_geodataframewhich convert the response content to respective types. The pandas, geopandas and shapely dependencies are not installed by default.
--footprintsnow includes all returned product properties in the output.
KeyError('No results returned.')is no longer returned for zero returned products in a response.
- Added underscore to methods and functions that are not expected to be used outside the package.
- Instance variables
contenthave been removed,
last_status_codehave been made private.
[0.8.1] – 2017-02-05¶
- added a changelog
- use logging instead of print
- docs represent new
[0.8] – 2017-01-27¶
- options to create new, reset or ignore vcr cassettes for testing
querynow returns a list of search results
download_allrequires the list of search results as an argument
SentinelAPIdoes not save query results as class attributes
[0.7.3] – 2016-12-09¶
page_sizeto better reflect pagination
- tests use
- support GeoJSON polygons with optional (third) z-coordinate
[0.7.1] – 2016-10-28¶
- pagination support for query results
- number of query results per page set to 100
[0.6.5] – 2016-06-22¶
- support for large queries
- Removed redundant information from Readme that is also present on Readthedocs
[0.6.1] – 2016-04-22¶
- Sphinx documentation setup with autodoc and numpydoc
- Redthedocs.org integration
[0.4.3] – 2015-11-23¶
- option to select a different dhus api
https://scihub.esa.int/apihub/as standard url
[0.4] – 2015-09-28¶
- method to manually select the CA certificate bundle
- function to return footprints of the queried Sentinel scenes
- CA-certificate SSL errors
[0.1] – 2015-06-05¶
- first release