Skip to content
GitLab
Menu
Projects
Groups
Snippets
Help
Help
Support
Community forum
Keyboard shortcuts
?
Submit feedback
Contribute to GitLab
Sign in
Toggle navigation
Menu
Open sidebar
geomultisens
gms_preprocessing
Commits
cf74b030
Commit
cf74b030
authored
Mar 31, 2020
by
Daniel Scheffler
Browse files
Removed 'maxtasksperchild=1' from Pool.
Signed-off-by:
Daniel Scheffler
<
danschef@gfz-potsdam.de
>
parent
2747586a
Pipeline
#8034
passed with stage
in 17 minutes and 50 seconds
Changes
1
Pipelines
1
Hide whitespace changes
Inline
Side-by-side
gms_preprocessing/processing/multiproc.py
View file @
cf74b030
...
...
@@ -50,7 +50,7 @@ def MAP(func, args, CPUs=None, flatten_output=False):
CPUs
=
CPUs
if
CPUs
<=
CFG
.
CPUs
else
CFG
.
CPUs
# treat CFG.CPUs as maximum number of CPUs
if
CPUs
and
CPUs
>
1
and
len
(
args
)
>
1
:
with
Pool
(
CPUs
,
maxtasksperchild
=
1
)
as
pool
:
with
Pool
(
CPUs
)
as
pool
:
results
=
pool
.
map
(
func
,
args
)
# always returns a list
else
:
results
=
[
func
(
argset
)
for
argset
in
args
]
# generator does not always work properly here
...
...
@@ -83,7 +83,7 @@ def imap_unordered(func, args, CPUs=None, flatten_output=False):
CPUs
=
CPUs
if
CPUs
<=
CFG
.
CPUs
else
CFG
.
CPUs
# treat CFG.CPUs as maximum number of CPUs
if
CPUs
and
CPUs
>
1
and
len
(
args
)
>
1
:
with
Pool
(
CPUs
,
maxtasksperchild
=
1
)
as
pool
:
with
Pool
(
CPUs
)
as
pool
:
results
=
list
(
pool
.
imap_unordered
(
func
,
args
))
# returns an iterator
else
:
results
=
[
func
(
argset
)
for
argset
in
args
]
# generator does not always work properly here
...
...
Write
Preview
Supports
Markdown
0%
Try again
or
attach a new file
.
Attach a file
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Cancel
Please
register
or
sign in
to comment