Metadata Tools in hale»connect: Profile Management and Automated Metadata Generation
A few months back, we described how you can setup hale»connect to fully automate metadata generation, re-use and validation. In the past months, we have worked together with a reference customer, the LGL Baden-Württemberg, to extend the metadata tools in hale»connect substantially. LGL Baden-Württemberg is one of 16 state-level cadastral, mapping and spatial planning authorities in Germany and maintains the state spatial data infrastructure.
The Problem
The problem that the LGL approached us with is one that we have seen many times already: there is a specific metadata profile, but both in terms of process and tools, the maintenance and implementation of this profile requires a lot of extra effort. The LGL used a massive Excel Table with more than 400 rows to describe their profile. This Excel table conflated profile rules from different levels (ISO, INSPIRE, national, and local), which all needed to be updated and kept in sync.
Excel table containing LGL Baden-Württemberg requirements. The excel table contains rules from ISO, INSPIRE, national and local schemas.
In implementation of the profile, there were often ambiguities or different interpretations of certain aspects of the profile. Furthermore, the documentation was a bit unwieldy and hard to read for most people outside the immediate circle of editors.
Thus, LGL reached out to us to understand how they could improve on this situation. Wetransform originally conducted a UML and Metadata workshop to explore whether developing the profile in a UML Tool such as Enterprise Architect would make sense. However, some aspects such as a nice differential documentation and the generation of an executable test suite directly from the profile proved to be quite hard.
Thus, in early 2018, LGL asked wetransform to develop a toolset that would support the creation of a fully formal profile definition of the ISO metadata schemas. This toolset would enable the definition of profiles on any XML-based schema, be it the CSW 2.0.2 Application Profile ISO 1.0.0 or a GML Application Schema.
The Solution
There were several key requirements that the profile toolset should resolve:
- Definition of type and property constraints (e.g. on cardinality, allowed content and allowed property types.)
- Definition of consistency constraints (e.g. “If A exists, B must also exist”) and other advanced validation rules.
- Addition of documentation via tagged values.
- Automated generation of printable documentation.
- Automated generation of differential documentation.
- Automated generation of Executable Testsuites (ETS) for the INSPIRE Reference Validator (ETF.)
- Automated generation of example XML instance documents for the profile.
- Be easy to use for people without in-depth knowledge in technologies such as XML, XML Schema, Schematron and ETF.
To fulfil these, we developed a formal model that could capture all the required information, as well as a set of editing functions that extend the existing schema modelling tools in hale connect. The formal profile contains type and property constraints, consistency constraints and tagged values:
Formal profile model diagram. The diagram displays the formal profile model developed by wetransform.
To create a profile, you first have to create a schema using any of the methods supported in hale connect. For creating a metadata profile, use either the ISO 19139 preset or the CSW 2.0.2 Application Profile ISO 1.0.0 preset.
After this step, you have to create a profile of that schema. This is also done through the “Create new Schema” process, using the “Create Profile” option. When the profile has been created (which takes up to 30 seconds), you can go to the Feature Types section and start applying type and property constraints.
Type level constraints. The toggle switch can be used to make a type mandatory.
To make a single type mandatory, you can just toggle it in the type header. Making it mandatory means that an object of this type has to occur at least once in any document. This is particularly useful in GML Feature Collection, where you want to state which types of objects have to be present.
Most of the constraints are set using the Edit Mode, which you can access through the Pen icon in the type header. Clicking this icon brings you to a powerful editor, where you can set a number of constraints:
- Content Required: If not already true, you can set this to true to make sure that the given property always has to have content. The semantics of this constraint are equivalent to setting the nillable flag to false.
- Minimum Cardinality: The minimum number of times this property needs to occur in any given type. It needs to be higher than the one defined in the schema, but lower or equal than the value given for maximum cardinality.
- Maximum Cardinality: The maximum number of times this property needs to occur in any given type. It needs to be lower than the one defined in the schema, but higher or equal than the value given for minimum cardinality.
- Allowed Values: In this field you can define an enumeration of the actually allowed values. If the property uses an enumeration or code list, you will be able to activate or deactivate options from that enumeration or codelist.
- Allowed Types: If this field uses a Choice, Substitution Group or Union, you can limit the allowed types by deactivating options.
In addition, you can add any number of tagged values to each property. You can use any keys you want, but some control specific behavior:
- label-testcase: Label for this test case in the ETF
Testsuite - description-testcase: Description for this test case in the ETF
Testsuite - module-id: ID of the
TestModulegroup to which the test case generated for this property will be added - example: Content to be used for the generation of XML sample files. If the property is a complex element, setting this value to "true" leads to the element being generated.
Consistency constraint editor: Consistency constraints are added to schema elements.
The second big block of functionality is to be found in the “Consistency Conditions” section of the profile. When you go there, you will be able to define various sets of rules, such as:
- If A exists, B must/must not exist
- If A has value “X”, B must have value “Y”
You can combine any number of such rules through logical operators (AND, OR, XOR). To each Consistency Condition, you can add a label and a description, as well as tagged values.
Generated documentation example. Profile documentation can be printed easily.
When you want to review your profile at any time, the differential documentation is a handy feature, especially in big schemas like the ISO 19139 schema. It hides all elements (types and properties) for which no change or addition was made in the profile, resulting in a compact, well-readable format.
Either the full profile or the differential documentation can be printed to any device, specifically to PDF. This enables you to get a document that you can send around to any user.
By going to the “Files” section of the profile, you can also export sample XML instance documents for your profile.
And finally, you want to make sure that any data you receive complies with your profile. For this purpose, hale connect can also export an Executable Test Suite, again through the Files section. This ETS can be run in the ETF (INSPIRE Reference Validator).
The profile tools described here can also be used in conjunction with the metadata generation tools introduced with hale connect 1.0. Ask us to learn how to do so, especially if you are already a hale connect user.
If you want to know more about how to use these tools, check out the online help.
Getting Access to the Metadata Tools
We presented the new metadata tools at the INSPIRE conference 2018 in Antwerp just a few weeks back and can now announce general availability.
The new metadata tools are part of all hale»connect licenses, independent of deployment mode and license level. While Metadata Generation has been part of hale»connect since version 1.0, the profile management tools have been released with version 1.11.0, which is available since October 17th, 2018.
The tools are available starting from 2.900 € per year (Micro Org Public Cloud license). Special pricing is available for multi-year contracts, partners, and customers with support subscriptions.