Signed-off-by: Daniel Scheffler <danschef@gfz-potsdam.de>

Recreated test dataset for Test_Landsat8_PreCollectionData. Replaced two os.system calls with subcall_with_output.

Signed-off-by: Daniel Scheffler <danschef@gfz-potsdam.de>

Fixed copy/paste error in license headers. Added .h5 files to GitLFS. Fixed missing bracket. Updated HISTORY.rst and version info.

Signed-off-by: Daniel Scheffler <danschef@gfz-potsdam.de>

Signed-off-by: Daniel Scheffler <danschef@gfz-potsdam.de>

Signed-off-by: Daniel Scheffler <danschef@gfz-potsdam.de>

Fixed deadlock within MemoryReserver. Added number of waiting processes to redis. Fixed linting. Removed python-redis-lock from dependencies.

Daniel Scheffler authored Mar 20, 2018 and Mathias Peters committed Mar 20, 2018

Pipeline 'run_complete_preprocessing' now returns processing reports only (may fix deadlock after large reference jobs).

Updated version info (0.14.1).

Added some options to options_default.json. Bugfix database_tools.record_stats_memusage. Fixed missing lib within CI config.

Further developed recording of memory usage statistics. Deleted deprecated classes within 'locks' module. Added system overload blocking. Revised default resource limits. Bugfix for not raising exceptions during GMS_pipeline.

Added recording of memory usage via new database table 'stats_mem_usage_homo'. Allows to intelligently estimation of memory usage.

Added MultiSlotLock inheriting from redis_semaphore.Semaphore.

Added option 'write_ENVIclassif_cloudmask' (fixes issue #72).

Converted GMS_object.from_sensor_subsystems, GMS_object.from_tiles and GMS_object.from_disk to class methods.

Fixed issue #68 (Cloud mask is not applied equally to all bands when filling clouds with no data values).

Added test_locks.py.

Implemented process locks to avoid CPU/RAM overload on case multiple GMS jobs are running on the same host.

Former-commit-id: a9a082a7
Former-commit-id: 81b03365

Pipeline now returns processed GMS_objects without array data. Multiprocessing now uses pool.imap_unordered. Fixed job summaries.
Former-commit-id: 9747ba76
Former-commit-id: 16b356e9

Former-commit-id: 901266f5
Former-commit-id: b05184f5

Fixed issue #50 (Invalid job progress statistics in case a subsystem fails after another one of the same scene ID already succeeded in the same mapper) by adding class attribute GMS_object.proc_status_all_GMSobjs and GMS_object.proc_status() property. Added gms_object.update_proc_status() decorator.
Revised exception handler. Improved test_exception_handler module.
Fixed a severe bug that copied the same dataset list to all subsequent process controllers.
Former-commit-id: cd572222
Former-commit-id: 0cce191d

Former-commit-id: bc76a722
Former-commit-id: 5f1dbbb4

Former-commit-id: a3369bcd
Former-commit-id: c2bae567

Former-commit-id: 389155a6
Former-commit-id: 21d574b6

Former-commit-id: 6680c7b2
Former-commit-id: d6e81881

Former-commit-id: cfe6c223
Former-commit-id: 48007c69

Added options: spatial_ref_min_overlap, spatial_ref_min_cloudcov, spatial_ref_max_cloudcov, spatial_ref_plusminus_days, spatial_ref_plusminus_years, band_wavelength_for_matching, spatial_resamp_alg, clip_to_extent, mgrs_pixel_buffer, output_data_compression.
Former-commit-id: 7051ba14
Former-commit-id: a9b856c5

Renamed option conversion_type_optical to target_radunit_optical and conversion_type_thermal to target_radunit_thermal. Keys in provided JSON file are now optional. Bugfixes.
Former-commit-id: 981de81d
Former-commit-id: 68ff63b7