{"id":6741,"date":"2023-11-01T15:01:53","date_gmt":"2023-11-01T15:01:53","guid":{"rendered":"https:\/\/wetransform.to\/?post_type=news-and-events&p=6741"},"modified":"2023-11-16T15:12:04","modified_gmt":"2023-11-16T15:12:04","slug":"go-peg-generating-high-value-data-sets","status":"publish","type":"news-and-events","link":"https:\/\/wetransform.to\/de\/neuigkeiten-veranstaltungen\/go-peg-generating-high-value-data-sets\/","title":{"rendered":"3\u00bd Years of GO-PEG: Our Experience in Generating Cross-border, High Value Data Sets"},"content":{"rendered":"

In late 2019, a consortium of six partners from five countries set out to create harmonised open datasets and their corresponding metadata in areas such as environment, emergency and disaster management that would span at least five EU countries. In the GO-PEG project<\/a> (Generation of cross border Pan European Geospatial Datasets and Services), co-funded by the European Health and Digital Executive Agency (HaDEA),<\/a> we wanted to improve existing data harmonisation methodologies and tools and apply them to create the before-mentioned cross border data sets.<\/p>\n

This post summarizes the project and our key takeaways now that the project has ended.<\/p>\n

In the beginning: A challenge!<\/h2>\n

What started this project was that, even 12 years after the INSPIRE directive,<\/a> it was still very hard to create re-useable, fully harmonised, cross-border data sets. The European Commission recognised this and made a call for consortia to create such data sets as well as the tools required for that, and to inform future processes such as the implementation of the High Value Datasets Act.<\/a><\/p>\n

Thus, we set out at the beginning of the project with three main goals:<\/p>\n

    \n
  1. Identify high-value data sets that we should create through selection of key use cases and their data requirements<\/li>\n
  2. Identify current barriers to the creation of fully harmonised, cross-border data sets, and find technical and methodological solutions for these problems<\/li>\n
  3. Create the actual data sets and publish them so that they are visible in the European data portal<\/li>\n<\/ol>\n

    To do this, we applied an iterative approach, measuring the successes and gathering info on new issues in each iteration. As an example, we involved stakeholders to see if they could really use the data sets, or whether they were still missing valuable information.<\/p>\n

    Use Cases and High Value Data Sets<\/h2>\n

    In the project we evaluated more than 15 use cases, ranging from statistical applications to geologic risk assessment and COVID tracking.<\/p>\n

    \"diagram<\/p>\n

    More information on these is available on the project website,<\/a> so we are going to focus on three here in this article:<\/p>\n

    Flexible UAV no-fly zones with fAIRport<\/h3>\n

    In 2020, we started the fAIRport project,<\/a> with the goal of creating a detailed map of no-fly zones for Unmanned Aerial Vehicles (UAVs, commonly known as drones) in Germany. Since this is a very well-defined use case based on European legislation, we decided to bring it into the GO-PEG project and to expand it from one country to as many as we could.<\/p>\n

    \"screenshot<\/p>\n

    The core idea for the GO-PEG part was to use INSPIRE data, e.g., on transport networks and protected areas, and to transform that to the ED-269 model that is being used in the UAV community to exchange such data. We used hale\u00bbstudio<\/a> to do the relevant transformations. To support the required input and output formats, several smaller improvements to hale\u00bbstudio were made.<\/p>\n

    No stone left unturned with GO-DEPTH<\/h3>\n

    The GO-DEPTH use case focuses on interoperable, easy-to-use, high-quality subsurface information for sustainable planning and use of natural resources.<\/p>\n

    \"screenshot<\/p>\n

    As the use of geological data was particularly relevant to this project, existing data models and code lists were extended. It was also decided to develop a specific GeoPackage<\/a> logical model for the INSPIRE Geology data theme and test the GeoPackage format as an alternative to INSPIRE default encoding for Geology. After data harmonisation has been completed inside hale\u00bbstudio, the GeoPackage data set can be shared via OGC API Features.<\/p>\n

    In short, this project allows you to share easy-to-use, interoperable 3D subsurface information in 4 steps:<\/p>\n

      \n
    1. Harmonise to INSPIRE extended model <\/li>\n
    2. Encode in GeoPackage<\/li>\n
    3. Share via OGC API<\/li>\n
    4. Make discoverable in the European Data Portal<\/a><\/li>\n<\/ol>\n

      Modern solutions for a modern pandemic with geo-COVID Watch<\/h3>\n

      The geo-COVID Watch use case aims to investigate the benefits of geospatial data and edge technologies for a common understanding of the COVID-19 pandemic.<\/p>\n

      \"diagram<\/p>\n

      Direct access mechanisms to measurement data, already successfully applied in the environmental field, can be applied to enhance timely availability and interoperability of COVID-19 pandemic data. In particular, the use case investigates the suitability of using OGC standards (Features<\/a> and SensorThings API<\/a>) and related technologies for sharing and searching COVID-19 data on the web.<\/p>\n

      The impact of GO-PEG on hale\u00bbstudio\u2026<\/h2>\n

      The GO-PEG project was instrumental in the major overhaul and improvement of the Open-Source Software hale\u00bbstudio.<\/a> Originally developed in 2008 to 2011, hale\u00bbstudio has become the most used FOSS ETL tool for geospatial and environmental data in Europe. <\/p>\n

      Over 99% of hale\u00bbstudio's users are happy with the product the way it is, so development slowed down from about 2018 to 2020. In 2020, it was clear that the software needed to be updated: It still used Java 8, and many of the libraries and frameworks used were also outdated. While it would have been nice to directly implement the innovations identified to resolve challenges in the GO-PEG use cases, it was clear to us that we should first fix the foundation.<\/p>\n

      Therefore, we started what should become a work of almost 18 months: Upgrade the software from Java 8 to Java 17 and update pretty much every library and component to the most current versions. After that work was concluded, and hale studio 5.0 was released, we investigated several key innovations such as:<\/p>\n