Skip to content
GitLab
Projects Groups Snippets
  • /
  • Help
    • Help
    • Support
    • Community forum
    • Submit feedback
    • Contribute to GitLab
  • Sign in
  • G gde-exporter
  • Project information
    • Project information
    • Activity
    • Labels
    • Members
  • Repository
    • Repository
    • Files
    • Commits
    • Branches
    • Tags
    • Contributors
    • Graph
    • Compare
  • Issues 0
    • Issues 0
    • List
    • Boards
    • Service Desk
    • Milestones
  • Merge requests 0
    • Merge requests 0
  • CI/CD
    • CI/CD
    • Pipelines
    • Jobs
    • Schedules
  • Deployments
    • Deployments
    • Environments
    • Releases
  • Packages and registries
    • Packages and registries
    • Package Registry
    • Infrastructure Registry
  • Monitor
    • Monitor
    • Incidents
  • Analytics
    • Analytics
    • Value stream
    • CI/CD
    • Repository
  • Wiki
    • Wiki
  • Snippets
    • Snippets
  • Activity
  • Graph
  • Create a new issue
  • Jobs
  • Commits
  • Issue Boards
Collapse sidebar
  • Dynamic ExposureDynamic Exposure
  • Global Dynamic ExposureGlobal Dynamic Exposure
  • gde-exporter
  • Merge requests
  • !8

Parallelised retrieval of quadkeys to run

  • Review changes

  • Download
  • Email patches
  • Plain diff
Merged Cecilia Nievas requested to merge feature/parallel_keys into main Aug 10, 2022
  • Overview 0
  • Commits 1
  • Pipelines 1
  • Changes 3

The querying of quadkeys associated with a particular exposure entity or data unit ID is quite slow. I've parallelised it so that the quadkeys associated with different exposure entities or data unit IDs get retrieved by different cores.

With my test case of 52 data units processed using 4 cores, the time used to retrieve quadkeys dropped to 30% of the non-parallelised case (i.e. an improvement of 70%). All data units together amounted for 3097 quadkeys (they were small data units). The gain will always depend on the size of the data unit/exposure entity and the number of cores available.

Assignee
Assign to
Reviewers
Request review from
Time tracking
Source branch: feature/parallel_keys