Skip to content

Parallelised retrieval of quadkeys to run

Cecilia Nievas requested to merge feature/parallel_keys into main

The querying of quadkeys associated with a particular exposure entity or data unit ID is quite slow. I've parallelised it so that the quadkeys associated with different exposure entities or data unit IDs get retrieved by different cores.

With my test case of 52 data units processed using 4 cores, the time used to retrieve quadkeys dropped to 30% of the non-parallelised case (i.e. an improvement of 70%). All data units together amounted for 3097 quadkeys (they were small data units). The gain will always depend on the size of the data unit/exposure entity and the number of cores available.

Merge request reports

Loading