{"id":9103,"date":"2024-11-18T09:53:19","date_gmt":"2024-11-18T09:53:19","guid":{"rendered":"https:\/\/wetransform.to\/?post_type=news-and-events&p=9103"},"modified":"2024-11-18T09:53:19","modified_gmt":"2024-11-18T09:53:19","slug":"high-value-dataset-metadata","status":"publish","type":"news-and-events","link":"https:\/\/wetransform.to\/de\/neuigkeiten-veranstaltungen\/high-value-dataset-metadata\/","title":{"rendered":"High Value Dataset Metadata"},"content":{"rendered":"

The High Value Dataset (HVD) Implementing Regulation<\/a> entered into force on June 9, 2024, with the aim of making a range of public sector data freely available to the public. The concept of high value datasets<\/a> first appeared in the Open Data Directive,<\/a> as a means of improving access to important datasets necessary for economic, social and environmental development. <\/p>\n

As discussed in our introduction to the EU Implementing Act on High-Value Datasets,<\/a> there are six categories of data into which HVDs can be classified: <\/p>\n

    \n
  1. Geospatial data<\/li>\n
  2. Earth observation<\/li>\n
  3. Environmental data<\/li>\n
  4. Statistics<\/li>\n
  5. Company ownership <\/li>\n
  6. Mobility <\/li>\n<\/ol>\n

    To promote the interoperability and accessibility of datasets subject to the HVD IR, a DCAT-AP HVD metadata profile<\/a> was introduced to govern the provision of metadata for these datasets.<\/p>\n

    For organisations that have been making public sector data available for years under the INSPIRE Directive,<\/a> several important questions arose surrounding overlapping requirements and datasets that are subject to both the INSPIRE Directive and the HVD Implementing Regulation, and metadata requirements, as the INSPIRE Directive requires metadata based on ISO\/TS 19139:2007.<\/a><\/p>\n

    The discussion and development of high-value dataset metadata reporting practices has required alignment between a large number of groups at the EU and national level including national metadata reporting authorities, open data portals, groups responsible for standards such DCAT-AP, GeoDCAT-AP, and INSPIRE,<\/a> as well as data owners and providers.<\/p>\n

    What are the metadata requirements for HVD datasets?<\/h2>\n

    The Data Catalog Vocabulary (DCAT) is now in version 3.0.<\/a> It is a Resource Description Language (RDF) vocabulary which supports communication between general data catalogs on the web.<\/a> RDF is a graph composed of triples: a node for the subject, an arc between a subject to an object, and a node for the object. The DCAT Application profile for data portals in Europe (DCAT-AP) is based on DCAT and is used to describe public sector datasets in Europe and support interoperability. GeoDCAT-AP is a geospatial extension of the DCAT application profile for data portals in Europe (DCAT-AP).<\/p>\n

    The DCAT-AP HVD, or "usage guidelines of DCAT-AP for High-Value Datasets" describes an HVD profile of DCAT-AP which supports the interoperability of catalogued resources. Conformance with the DCAT-AP HVD profile requires compliance with DCAT-AP as a prerequisite. The HVD profile defines additional mandatory and optional metadata elements in addition to existing, required DCAT-AP metadata elements.<\/p>\n

    For INSPIRE implementers currently providing ISO metadata for datasets identified as HVD, the path forward is being developed. <\/p>\n

    Over the past several years, a lot of important groundwork has been done to develop tooling for the INSPIRE community to transform their INSPIRE ISO metadata to GeoDCAT-AP. Version 3.0.0 of GeoDCAT-AP<\/a> was released on July 9, 2024, and under review until September 27th, 2024. The latest version contains guidance on the usage of DCAT-AP HVD for GeoDCAT-AP. An XSLT tool<\/a>, updated to use GeoDCAT-AP v3, and a demo API<\/a> developed by the Semantic Interoperability Community are freely available for use. The tools convert INSPIRE ISO metadata to GeoDCAT-AP, however it has been determined that the ISO 19139 to GeoDCAT-AP XSLT<\/a> will not produce fully compliant DCAT-AP HVD metadata. <\/p>\n

    The RDF generated by the XSLT must be manually corrected on a case-by-case basis to be fully compliant against the DCAT-AP HVD profile. The XSLT limitations are related to HVD requirements for URI based licenses and persistent and dereferenceable IDs. These limitations will drive the need for custom solutions or further extensions, which add the required values as needed.<\/p>\n

    \"Slide
    \n[From Interoperable Europe:     Introductory webinar<\/a> on the revision of GeoDCAT-AP on February 20, 2024]<\/p>\n

    How will HVD metadata be reported, or harvested?<\/h2>\n

    HVD metadata, once harvested, will be available for search on Europe\u2019s open data portal, data.europa.eu.<\/a> The reporting requirements for HVDs<\/a> on data.europa.eu state that an HVD report must be submitted by Member States by February 9th, 2025, and then again every two years. It is stated that the report must contain the following information:<\/p>\n

      \n
    1. list of published high-value datasets at Member State level (and, where relevant, subnational level) with online reference to metadata;<\/li>\n
    2. persistent link to the applicable licensing conditions;<\/li>\n
    3. persistent link to the APIs ensuring access to the high-value datasets<\/li>\n<\/ol>\n

      The DCAT-AP HVD specification also includes a section on reporting<\/a> which includes the requirement for EU Member States (the reporting authority) to provide a list of HVD datasets. Although no explicit technical guidelines on how this should be implemented are provided, it is suggested that a catalogue endpoint which contains all the metadata about Datasets, Distributions and Data Services that are in scope of the HVD could be used to achieve the reporting requirement. The reporting section goes on to describe a scenario<\/a> where Member States could query national catalogs for all resources with http:\/\/data.europa.eu\/eli\/reg_impl\/2023\/138\/oj<\/a> as applicable legislation, and a scenario<\/a> where the EU could query Member States' national catalogs where persistent identifiers are assigned as specified in DCAT-AP HVD section 10.4.1 Persistent identifiers.<\/p>\n

      The Action 2.5 HVD and INSPIRE survey outcomes presented in the 79th INSPIRE MIG-T meeting<\/a> included a recommended scenario for HVD reporting: <\/p>\n

      Automated reporting of a list of HVD from data. europa.eu where data.europa.eu harvests the national open data portal and the national geodata portal (mapping based on GeoDCAT-AP for geoportals). The MIG-T is currently working to develop a best practice guide for HVD metadata and has released the results of a multi-national survey on the topic. The survey results will be included as recommendations in the guidelines. It is evident, due to the number of current discussions in the various standards communities, that the development and alignment of rules and reporting mechanisms for HVD are still on-going. Data.europa.eu recently released a research paper entitled \u201cHigh Value Datasets Best Practices in Europe\u201d<\/a> which highlights this point, among other concerns raised by Member States including Finland, Denmark, Italy, Austria, and others currently implementing the HVD regulation. <\/p>\n

      \"\"<\/p>\n

      How are different Member States providing HVD metadata?<\/h2>\n

      In the 79th MIG-T meeting, the recommendation that high-value datasets should be identified in metadata by providing the link to the HVD IR and HVD categories provided in EuroVoc was decided. The recommendation will be included in the upcoming HVD and INSPIRE best practice document. Additionally, it was recommended that licenses need to be structured and machine readable. Only URI based licenses and access right should be allowed. This recommendation would require an update to existing metadata if URI\/IRIs are not currently used.<\/p>\n

      At wetransform, we have been following the steps that national metadata leads have been taking, despite the lack of clear technical guidelines, to adapt ISO metadata requirements to include information that can be used to query and extract HVD relevant metadata from their CSW catalogs for the purposes of conversion to DCAT-AP HVD.<\/p>\n

      Germany<\/h3>\n

      Geodateninfrastruktur Deutschland (GDI-DE) maintains a helpful wiki<\/a> to discuss HVD related questions, including questions about HVD metadata. In Germany, the \u201cKonventionen zu Metadaten\u201d is currently being updated to include requirements for additional keywords in ISO metadata to identify high-value datasets. A provisional guideline explaining the changes has been provided in the meantime. The keywords will be used to identify high-value datasets and make them available in DCAT-AP.de,<\/a> which is the German profile of DCAT-AP, and in the open data portal GovData.de.<\/a><\/p>\n

      The sample representation<\/a> in the XML metadata set (excerpt) looks as follows:<\/p>\n

      ...\n<gmd:descriptiveKeywords>\n  <gmd:MD_Keywords>\n    <gmd:keyword>\n<gmx:Anchor xlink:href="http:\/\/data.europa.eu\/bna\/c_ac64a52d">Georaum<\/gmx:Anchor>\n    <\/gmd:keyword>\n    ...\n    <gmd:thesaurusName>\n       <gmd:CI_Citation>\n        <gmd:title>\n<gmx:Anchor xlink:href="http:\/\/data.europa.eu\/bna\/asd487ae75">High-value dataset categories<\/gmx:Anchor>\n        <\/gmd:title>\n        <gmd:date>\n          <gmd:CI_Date>\n            <gmd:date>\n              <gco:Date>2023-09-27<\/gco:Date>\n            <\/gmd:date>\n            <gmd:dateType>\n              <gmd:CI_DateTypeCode codeList="https:\/\/standards.iso.org\/iso\/19139\/\nresources\/gmxCodelists.xml#CI_DateTypeCode" codeListValue="publication"\/>\n            <\/gmd:dateType>\n          <\/gmd:CI_Date>\n        <\/gmd:date>\n        ...\n      <\/gmd:CI_Citation>\n    <\/gmd:thesaurusName>\n  <\/gmd:MD_Keywords>\n<\/gmd:descriptiveKeywords>\n...\n<\/code><\/pre>\n
      Denmark<\/h3>\n
      In Denmark, an approach similar to Germany\u2019s will be taken.<\/a> While similar, the keyword and tagging approach is not identical, highlighting the need<\/a> for standardized guidelines on how to add the EU controlled vocabulary on HVD category, or other relevant values, to ISO metadata.<\/p>\n
      As seen in GitHub: Danish proposal for tagging metadata for data sets that are in scope of HVD,<\/a> there is a tag saying that the data set is in scope of HVD (gmx:Anchor xlink:href="https:\/\/eur-lex.europa.eu\/eli\/reg_impl\/2023\/138\/oj<\/a>"<\/em>):<\/p>\n
      gmd:descriptiveKeywords\ngmd:MD_Keywords\ngmd:keyword\n<gmx:Anchor xlink:href="https:\/\/eur-lex.europa.eu\/eli\/dir\/2007\/2\/2019-06-26">INSPIRE<\/gmx:Anchor>\n<\/gmd:keyword>\ngmd:keyword\n<gmx:Anchor xlink:href="https:\/\/eur-lex.europa.eu\/eli\/reg_impl\/2023\/138\/oj">H\u00f8j-v\u00e6rdi datas\u00e6t<\/gmx:Anchor>\n<\/gmd:keyword>\n...\ngmd:thesaurusName\ngmd:CI_Citation\ngmd:title\n<gmx:Anchor xlink:href="https:\/\/registry.geonetwork-opensource.org\/theme\/eu">EU legislation<\/gmx:Anchor>\n<\/gmd:title>\n...\n<\/gmd:thesaurusName>\n<\/gmd:MD_Keywords>\n<\/gmd:descriptiveKeywords><\/code><\/pre>\n
      Another tag that specifies that the data set belongs to Earth observation and environment (da: Jordobservation og milj\u00f8) category (gmx:Anchor xlink:href="http:\/\/data.europa.eu\/bna\/c_dd313021<\/a>"<\/em>):<\/p>\n
      gmd:descriptiveKeywords\ngmd:MD_Keywords\ngmd:keyword\n<gmx:Anchor xlink:href="http:\/\/data.europa.eu\/bna\/c_dd313021">Jordobservation og milj\u00f8<\/gmx:Anchor>\n<\/gmd:keyword>\n...\ngmd:thesaurusName\ngmd:CI_Citation\ngmd:title\n<gmx:Anchor xlink:href="http:\/\/data.europa.eu\/bna\/asd487ae75">High-value dataset categories<\/gmx:Anchor>\n<\/gmd:title>\n...\n<\/gmd:thesaurusName>\n<\/gmd:MD_Keywords>\n<\/gmd:descriptiveKeywords>\n<\/code><\/pre>\n
      The Netherlands<\/h3>\n
      The Netherlands has published guidelines<\/a> on implementing HVDs for data providers in their country. In the Netherlands, geospatial data is delivered using the Dutch metadata profiles for ISO 19115 and 19119. The conversion of INSPIRE metadata to DCAT is handled by the national catalog. Again, similar to Germany and Denmark, the Netherlands recommends the addition of a keyword in the ISO metadata to identify HVDs:<\/p>\n