exposure-japan merge requestshttps://git.gfz-potsdam.de/globaldynamicexposure/exposure-japan/-/merge_requests2022-01-31T10:32:11+01:00https://git.gfz-potsdam.de/globaldynamicexposure/exposure-japan/-/merge_requests/33Renamed the district-import function for consistency in naming2022-01-31T10:32:11+01:00Danijel SchorlemmerRenamed the district-import function for consistency in namingAll import functions start with `import_` except for the `read_districts_and_boundaries`. This renaming increases the consistency in naming.
\approve @shinde @taraAll import functions start with `import_` except for the `read_districts_and_boundaries`. This renaming increases the consistency in naming.
\approve @shinde @taraDanijel SchorlemmerDanijel Schorlemmerhttps://git.gfz-potsdam.de/globaldynamicexposure/exposure-japan/-/merge_requests/32Changed the Shapely polygon creation from WKT to WKB to speed up the processing2022-01-28T13:11:38+01:00Danijel SchorlemmerChanged the Shapely polygon creation from WKT to WKB to speed up the processingThe change of WKT to WKB for Shapely polygon generation speeds up the process by 10-20%.The change of WKT to WKB for Shapely polygon generation speeds up the process by 10-20%.Danijel SchorlemmerDanijel Schorlemmerhttps://git.gfz-potsdam.de/globaldynamicexposure/exposure-japan/-/merge_requests/31Feature/create import building numbers2022-02-11T16:31:09+01:00Simantini ShindeFeature/create import building numbersCreated a function to import building numbers
\approve @ds @taraCreated a function to import building numbers
\approve @ds @tarahttps://git.gfz-potsdam.de/globaldynamicexposure/exposure-japan/-/merge_requests/30Feature/optimize sql2022-01-27T17:33:04+01:00Danijel SchorlemmerFeature/optimize sqlOptimized SQL query for processing speed.
\approve @shindeOptimized SQL query for processing speed.
\approve @shindeSimantini ShindeSimantini Shindehttps://git.gfz-potsdam.de/globaldynamicexposure/exposure-japan/-/merge_requests/29Removed calculate_district_area2022-01-27T16:33:31+01:00Simantini ShindeRemoved calculate_district_areaRemoved the calculate_district_area function.
\approve @dsRemoved the calculate_district_area function.
\approve @dsSimantini ShindeSimantini Shindehttps://git.gfz-potsdam.de/globaldynamicexposure/exposure-japan/-/merge_requests/28Simplified the SQL query copying geometries from the Geopackage file2022-01-27T10:51:00+01:00Danijel SchorlemmerSimplified the SQL query copying geometries from the Geopackage fileChanged the query to avoid geometry conversions.
\approve @shindeChanged the query to avoid geometry conversions.
\approve @shindeDanijel SchorlemmerDanijel Schorlemmerhttps://git.gfz-potsdam.de/globaldynamicexposure/exposure-japan/-/merge_requests/27Improved the district import SQL query2022-01-26T17:45:44+01:00Danijel SchorlemmerImproved the district import SQL queryChanged the SQL query to use binary geometries instead of WKT geometries.
\approve @shindeChanged the SQL query to use binary geometries instead of WKT geometries.
\approve @shindeSimantini ShindeSimantini Shindehttps://git.gfz-potsdam.de/globaldynamicexposure/exposure-japan/-/merge_requests/26Added separate function to import population data2022-01-27T15:08:42+01:00Simantini ShindeAdded separate function to import population dataThe population data is imported in a separate function in to the database.
\approve @dsThe population data is imported in a separate function in to the database.
\approve @dsSimantini ShindeSimantini Shindehttps://git.gfz-potsdam.de/globaldynamicexposure/exposure-japan/-/merge_requests/25Rewrote the read_districts_function2022-01-16T22:25:19+01:00Simantini ShindeRewrote the read_districts_functionRewrote the read_districts function to make it more efficient.
The rest of the code remains unchanged for now and will be changed depending to how we reformat the code to fit in the SQL functionality.
\approve @dsRewrote the read_districts function to make it more efficient.
The rest of the code remains unchanged for now and will be changed depending to how we reformat the code to fit in the SQL functionality.
\approve @dshttps://git.gfz-potsdam.de/globaldynamicexposure/exposure-japan/-/merge_requests/24Added household data for districts with no information2021-12-10T11:07:26+01:00Simantini ShindeAdded household data for districts with no informationDid a few things in this MR:
1. Imported the dwelling numbers to calculate household data for wooden and non-wooden from the dwelling numbers list created in the previous MR instead of the earlier from the database.
2. Added functionali...Did a few things in this MR:
1. Imported the dwelling numbers to calculate household data for wooden and non-wooden from the dwelling numbers list created in the previous MR instead of the earlier from the database.
2. Added functionality to calculate household data for districts with no information and add them to the database.
Related issue: https://git.gfz-potsdam.de/dynamicexposure/collaboration/-/issues/377
\fyi @cnievas \approve @ds @tarahttps://git.gfz-potsdam.de/globaldynamicexposure/exposure-japan/-/merge_requests/23Added dwelling numbers for districts with no information2021-11-26T09:53:39+01:00Simantini ShindeAdded dwelling numbers for districts with no informationThis MR adds the dwelling numbers for administrative units with no information using the building densities.
\fyi @cnievas
\approve @ds @taraThis MR adds the dwelling numbers for administrative units with no information using the building densities.
\fyi @cnievas
\approve @ds @tarahttps://git.gfz-potsdam.de/globaldynamicexposure/exposure-japan/-/merge_requests/22Modified building numbers no info functionality2021-11-24T10:08:22+01:00Simantini ShindeModified building numbers no info functionalityIn this MR I have modified the functionality of building numbers for administrative unit without building information to accommodate the calculation of dwelling numbers for administrative unit without dwelling information.
\fyi @cnieva...In this MR I have modified the functionality of building numbers for administrative unit without building information to accommodate the calculation of dwelling numbers for administrative unit without dwelling information.
\fyi @cnievas
\approve @ds @tarahttps://git.gfz-potsdam.de/globaldynamicexposure/exposure-japan/-/merge_requests/21Functionality to calculate building numbers for districts with no building in...2021-11-26T09:53:39+01:00Simantini ShindeFunctionality to calculate building numbers for districts with no building informationThe following things have been done in the MR:
1. The PopulationDistribution table is inserted first after which it is the BuildingNumber table in the database.
2. Added functionality to calculate the relationship between population de...The following things have been done in the MR:
1. The PopulationDistribution table is inserted first after which it is the BuildingNumber table in the database.
2. Added functionality to calculate the relationship between population density and building density for building numbers with information
3. With the above relationship and the population density in building numbers with no information, calculate its building numbers.
4. Finally add these numbers to the BuildingNumber table.
\fyi @cnievas
\rfc @ds @tara @prehn
\approve @tara @dsSimantini ShindeSimantini Shindehttps://git.gfz-potsdam.de/globaldynamicexposure/exposure-japan/-/merge_requests/20Fixed errors in frequency distribution household data code2021-11-05T15:11:36+01:00Simantini ShindeFixed errors in frequency distribution household data codeFixed some typos and errors in the calculate_household_frequency_distributions function.
\approve @ds @taraFixed some typos and errors in the calculate_household_frequency_distributions function.
\approve @ds @taraSimantini ShindeSimantini Shindehttps://git.gfz-potsdam.de/globaldynamicexposure/exposure-japan/-/merge_requests/19Frequency distribution calculations for household data2021-11-03T19:49:41+01:00Simantini ShindeFrequency distribution calculations for household dataThe frequency distribution code is the last in data manipulation and would be added in 3 parts. the first part here: https://git.gfz-potsdam.de/dynamicexposure/globaldynamicexposure/exposure-japan/-/merge_requests/17 and second part: htt...The frequency distribution code is the last in data manipulation and would be added in 3 parts. the first part here: https://git.gfz-potsdam.de/dynamicexposure/globaldynamicexposure/exposure-japan/-/merge_requests/17 and second part: https://git.gfz-potsdam.de/dynamicexposure/globaldynamicexposure/exposure-japan/-/merge_requests/18 is already merged. In this last part, using the frequency distribution calculations, the error for mismatch of numbers is fixed for household and household member numbers.
\fyi @cnievas @prehn
\approve @ds @tarahttps://git.gfz-potsdam.de/globaldynamicexposure/exposure-japan/-/merge_requests/18Frequency distribution calculations for building numbers2021-11-03T18:08:16+01:00Simantini ShindeFrequency distribution calculations for building numbersThe frequency distribution code is the last in data manipulation and would be added in 3 parts. the first part here: https://git.gfz-potsdam.de/dynamicexposure/globaldynamicexposure/exposure-japan/-/merge_requests/17 is already merged.
...The frequency distribution code is the last in data manipulation and would be added in 3 parts. the first part here: https://git.gfz-potsdam.de/dynamicexposure/globaldynamicexposure/exposure-japan/-/merge_requests/17 is already merged.
In this second part, using the frequency distribution calculations, the error for mismatch of numbers is fixed for building numbers.
\fyi @cnievas @prehn
\approve @ds @tarahttps://git.gfz-potsdam.de/globaldynamicexposure/exposure-japan/-/merge_requests/17Frequency Distribution for dwelling numbers2021-11-03T18:08:16+01:00Simantini ShindeFrequency Distribution for dwelling numbersThe frequency distribution code is the last in data manipulation and would be added in 3 parts. In this first part, using the frequency distribution calculations. The error for mismatch of numbers is fixed for dwelling numbers.
\fyi @c...The frequency distribution code is the last in data manipulation and would be added in 3 parts. In this first part, using the frequency distribution calculations. The error for mismatch of numbers is fixed for dwelling numbers.
\fyi @cnievas @prehn
\approve @ds @taraSimantini ShindeSimantini Shindehttps://git.gfz-potsdam.de/globaldynamicexposure/exposure-japan/-/merge_requests/16Distributes the household data for constuction material2021-10-27T12:35:37+02:00Simantini ShindeDistributes the household data for constuction materialThis code calculates the proportion of dwelling numbers for combination of building attributes (building type and construction material) and uses these proportions to calculate and update the household numbers and household members.
\f...This code calculates the proportion of dwelling numbers for combination of building attributes (building type and construction material) and uses these proportions to calculate and update the household numbers and household members.
\fyi @cnievas
\rfc @prehn
\approve @ds @taraSimantini ShindeSimantini Shindehttps://git.gfz-potsdam.de/globaldynamicexposure/exposure-japan/-/merge_requests/15Expanded rows to add construction_material_id in HouseholdData2021-10-15T16:58:20+02:00Simantini ShindeExpanded rows to add construction_material_id in HouseholdDataThis MR does the following:
1. Expands the HouseholdData table for total, wooden and non wooden construction_material_ids.
2. Copies the similar values for number_household, number_household_members in these additional rows otherwise i...This MR does the following:
1. Expands the HouseholdData table for total, wooden and non wooden construction_material_ids.
2. Copies the similar values for number_household, number_household_members in these additional rows otherwise in the remaining columns it keeps the values as 0. for e.g.:
| construction_material_id | number_dwelling | number_household | number_household_members |
|--------------------------|-----------------|------------------|--------------------------|
| 0 | 2416700 | 2433500 | 5130900 |
| 1 | 0 | 2433500 | 5130900 |
| 4 | 0 | 2433500 | 5130900 |
I tried keeping the rows (for number_dwelling, etc) empty but it doesn't satisfy the constraint for the column number_dwelling and others to be integer or float (if you have any other suggestions please do tell me).
The reason for expanding the HouseholdData table for construction material total, wooden and non wooden are as follows:
1. The HouseholdData from the census doesn't include this expansion (wooden and non wooden).
2. Since there is a number_dwelling present in the HouseholdData table, this expansion is simpler. It would be done in the codes later by calculating the proportions of dwellings for wooden and non wooden construction and using these proportions to find the wooden and non wooden households.
\fyi @cnievas
\approve @ds @tara @prehnhttps://git.gfz-potsdam.de/globaldynamicexposure/exposure-japan/-/merge_requests/14Population table added2021-10-01T13:05:11+02:00Simantini ShindePopulation table addedAdded Population to the database. Linked to issue: https://git.gfz-potsdam.de/dynamicexposure/collaboration/-/issues/308
\approve @ds @tara @prehnAdded Population to the database. Linked to issue: https://git.gfz-potsdam.de/dynamicexposure/collaboration/-/issues/308
\approve @ds @tara @prehnSimantini ShindeSimantini Shinde