Skip to content

In sicor_ac if a crash happens in ac function the output directory is not created leading to an error when we attemp to write METADATA in the finally clause.

The code between sicor_ac.py:457 and sicor_ac.py:471:

        if all_output_exists is False:  # do AC
            # load image
            s2img = load_product(options=options, logger=logger)
            # do ac
            s2img = ac_gms(s2img, options=options, logger=logger, script=False)

            # save some memory -> discard the band that are not needed anymore
            needed_l1c_bands = set(sum([oo[kk] for kk in ("rgb_bands",) for oo in options["output"] if kk in oo], []))
            obsolete_bands = [band for band in s2img.data.keys() if band not in needed_l1c_bands]
            for band in obsolete_bands:
                del s2img.data[band]  # remove L1C data from memory

            tbf = time()
            IO.write_results(s2img=s2img, options=options, logger=logger)
            del s2img  # free memory

can crash for 2 reasons:

  • The auxiliary data is missing, such as could mask.
  • The issue #50 (closed)

In the clause finally the ac function will attempt to write metadata, sicor_ac.py:570

            for opts in options["output"]:
                if opts["type"] == "metadata":
                    opts["fn"] = opts["fn"].replace(options["run_suffix"], "")
                    IO.write_metadata(opts, options, logger=logger)

but the output directory does not exist yet because an exception occurred as explained above. Since the output directory does not exist, this leads to the following error message:

Traceback (most recent call last):
  File "/usr/local/python/.pyenv/versions/gts2_python_3.6.9/lib/python3.6/site-packages/rq/worker.py", line 883, in perform_job
    rv = job.perform()
  File "/usr/local/python/.pyenv/versions/gts2_python_3.6.9/lib/python3.6/site-packages/rq/job.py", line 645, in perform
    self._result = self._execute()
  File "/usr/local/python/.pyenv/versions/gts2_python_3.6.9/lib/python3.6/site-packages/rq/job.py", line 651, in _execute
    return self.func(*self.args, **self.kwargs)
  File "/usr/local/python/.pyenv/versions/gts2_python_3.6.9/lib/python3.6/site-packages/sicor-0.14.2-py3.6.egg/sicor/sicor_ac.py", line 570, in ac
    IO.write_metadata(opts, options, logger=logger)
  File "/usr/local/python/.pyenv/versions/gts2_python_3.6.9/lib/python3.6/site-packages/sicor-0.14.2-py3.6.egg/sicor/sicor.py", line 741, in write_metadata
    with open(opts["fn"], "w") as fl:
FileNotFoundError: [Errno 2] No such file or directory: '/home/gts2/data/S2B_L2A_v0.14/33U/VU/2020/S2B_MSIL2A_20200804T101559_N0209_R065_T33UVU_20200804T130113.SAFE/GRANULE/L2A_T33UVU_A017823_20200804T101730/META_T33UVU_A017823_20200804T101730.json'

If we do a print(err) and an assertion in the exception clause, it is possible to see the reason (in this case the issue #50 (closed) ):

Traceback (most recent call last):
  File "/usr/local/python/.pyenv/versions/gts2_python_3.6.9/lib/python3.6/site-packages/numpy/core/function_base.py", line 117, in linspace
    num = operator.index(num)
TypeError: 'float' object cannot be interpreted as an integer

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/python/.pyenv/versions/gts2_python_3.6.9/lib/python3.6/site-packages/sicor-0.14.2-py3.6.egg/sicor/sicor_ac.py", line 470, in ac
    IO.write_results(s2img=s2img, options=options, logger=logger)
  File "/usr/local/python/.pyenv/versions/gts2_python_3.6.9/lib/python3.6/site-packages/sicor-0.14.2-py3.6.egg/sicor/sicor.py", line 680, in write_results
    IO.write_rgb_jpeg(s2img=s2img, opts=opts, logger=logger)
  File "/usr/local/python/.pyenv/versions/gts2_python_3.6.9/lib/python3.6/site-packages/sicor-0.14.2-py3.6.egg/sicor/sicor.py", line 714, in write_rgb_jpeg
    hist_chop_off_fraction=opts["hist_chop_off_fraction"])
  File "/usr/local/python/.pyenv/versions/gts2_python_3.6.9/lib/python3.6/site-packages/sicor-0.14.2-py3.6.egg/sicor/sensors/RSImage.py", line 182, in image_to_rgb
    bins = np.linspace(0.0, 1.0, 100 / 2.0)
  File "<__array_function__ internals>", line 6, in linspace
  File "/usr/local/python/.pyenv/versions/gts2_python_3.6.9/lib/python3.6/site-packages/numpy/core/function_base.py", line 121, in linspace
    .format(type(num)))
TypeError: object of type <class 'float'> cannot be safely interpreted as an integer.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/python/.pyenv/versions/gts2_python_3.6.9/lib/python3.6/site-packages/sicor-0.14.2-py3.6.egg/sicor/sicor_ac.py", line 522, in ac
    assert 0
AssertionError

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/usr/local/python/.pyenv/versions/gts2_python_3.6.9/lib/python3.6/site-packages/rq/worker.py", line 883, in perform_job
    rv = job.perform()
  File "/usr/local/python/.pyenv/versions/gts2_python_3.6.9/lib/python3.6/site-packages/rq/job.py", line 645, in perform
    self._result = self._execute()
  File "/usr/local/python/.pyenv/versions/gts2_python_3.6.9/lib/python3.6/site-packages/rq/job.py", line 651, in _execute
    return self.func(*self.args, **self.kwargs)
  File "/usr/local/python/.pyenv/versions/gts2_python_3.6.9/lib/python3.6/site-packages/sicor-0.14.2-py3.6.egg/sicor/sicor_ac.py", line 572, in ac
    IO.write_metadata(opts, options, logger=logger)
  File "/usr/local/python/.pyenv/versions/gts2_python_3.6.9/lib/python3.6/site-packages/sicor-0.14.2-py3.6.egg/sicor/sicor.py", line 741, in write_metadata
    with open(opts["fn"], "w") as fl:
FileNotFoundError: [Errno 2] No such file or directory: '/home/gts2/data/S2B_L2A_v0.14/33U/VU/2020/S2B_MSIL2A_20200804T101559_N0209_R065_T33UVU_20200804T130113.SAFE/GRANULE/L2A_T33UVU_A017823_20200804T101730/META_T33UVU_A017823_20200804T101730.json'