Skip to content
Snippets Groups Projects
Commit bfdc0581 authored by Daniel Scheffler's avatar Daniel Scheffler
Browse files

256 is out of range for uint8, use 255 instead.

parent 0aadfb65
Branches
Tags
No related merge requests found
Pipeline #72931 passed
......@@ -40,7 +40,7 @@ def get_outFillZeroSaturated(dtype) -> tuple:
"get_outFillZeroSaturated: Unknown dType: '%s'." % dtype
dict_outFill = {'int8': -128, 'uint8': 0, 'int16': -9999, 'uint16': 9999, 'float32': -9999.}
dict_outZero = {'int8': 0, 'uint8': 1, 'int16': 0, 'uint16': 1, 'float32': 0.}
dict_outSaturated = {'int8': 127, 'uint8': 256, 'int16': 32767, 'uint16': 65535, 'float32': 65535.}
dict_outSaturated = {'int8': 127, 'uint8': 255, 'int16': 32767, 'uint16': 65535, 'float32': 65535.}
return dict_outFill[dtype], dict_outZero[dtype], dict_outSaturated[dtype]
......
0% Loading or .
You are about to add 0 people to the discussion. Proceed with caution.
Please register or to comment