Skip to content

GitLab

  • Menu
Projects Groups Snippets
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • EnPT EnPT
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 20
    • Issues 20
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 2
    • Merge requests 2
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Monitor
    • Monitor
    • Incidents
  • Packages & Registries
    • Packages & Registries
    • Package Registry
    • Container Registry
    • Infrastructure Registry
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • EnMAP
  • GFZ_Tools_EnMAP_BOX
  • EnPTEnPT
  • Issues
  • #81

Closed
Open
Created Dec 15, 2021 by Daniel Scheffler@danschefOwner

Coverage raises warnings when running in multiprocessing

coverage data file: cannot unpack non-iterable NoneType object
Coverage.py warning: Data file '/home/gfz-fe/scheffler/python/findr_homo/.coverage.geoms.1279091.080330' doesn't seem to be a coverage data file: Couldn't use data file '/home/gfz-fe/scheffler/python/findr_homo/.coverage.geoms.1279091.080330': no such table: coverage_schema
Coverage.py warning: Couldn't use data file '/home/gfz-fe/scheffler/python/findr_homo/.coverage.geoms.1274268.063192': database disk image is malformed

For example, in this pipeline.

Solution: Properly close and join multiprocessing.Pool like this:

with multiprocessing.Pool() as pool:
    pool.map(func, args)
    pool.close()
    pool.join()

This has to be done in the EnPT code (!60 (6ffec51a)) AND ALL EXTERNAL LIBRARIES where multiprocessing.Pool is called, i.e., in arosics, spechomo, sicor, sensormapgeo, ...

Another way is to make coverage exclude external packages by adding this to the .coveragerc:

[run]
omit = */site-packages/*,*/tests/*,*/.eggs/*
Assignee
Assign to
Time tracking