Skip to content

Feature/queue

Karsten Prehn requested to merge feature/queue into main

This contains changes to the parallel computation. Now, the existing function _tile_polygons_in_quadkey and a new function _tile_grid_in_quadkey that contains most of the previous computations from the main function are combined in a new function discretize. To discretize tiles is now a simple call to discretizer.discretize(quadkeys_gdf).

This MR also introduces a queue of quadkeys that is published to all workers in parallel computations. The former pre-calculation of chunks is removed, as every idle worker can now simply take an item from the queue until there are no more pending jobs.

Additionally, this MR gets rid of initializing the TileGrid class with a custom discretizer.feature object. Instead, the processed dictionary, that contains the processed data from every calculation step, is passed into every next-step function that needs that data.

\approve @ds @laurens

Edited by Karsten Prehn

Merge request reports