{"id":1700,"date":"2017-02-20T16:31:53","date_gmt":"2017-02-20T16:31:53","guid":{"rendered":"https:\/\/wetransform.eu\/?post_type=news-and-events&#038;p=1700"},"modified":"2022-07-28T08:51:14","modified_gmt":"2022-07-28T08:51:14","slug":"tutorial-collectors-halestudio","status":"publish","type":"news-and-events","link":"https:\/\/wetransform.to\/de\/neuigkeiten-veranstaltungen\/tutorial-collectors-halestudio\/","title":{"rendered":"Tutorial: Working with Collectors in hale\u00bbstudio"},"content":{"rendered":"<p>Collectors are a powerful feature that we introduced in hale\u00bbstudio 3.1.0 and have since then expanded on. So, for what use cases should you be looking at collectors?<\/p>\n<ul>\n<li>Your target schema has a collection object, such as a <code>Network<\/code>, that needs to reference many or all objects of a different type, e.g. <code>NetworkLinks<\/code><\/li>\n<li>You are building up a hierarchy of objects such as <code>AdministrativeUnits<\/code>, with their <code>upperLevelUnit<\/code> and <code>lowerLevelUnit<\/code> references<\/li>\n<li>You need to use Merge or Join operations to determine relationships between objects, but these are computationally too expensive<\/li>\n<\/ul>\n<p>With a Collector, you can collect values in one place in your transformation project and then use these values in another place in the transformation process. Let's look at a recent project we've worked on to see how they work in practice.<\/p>\n<p>Please note that this article assumes you have working knowledge of hale\u00bbstudio and know the <a href=\"http:\/\/help.halestudio.org\/latest\/nav\/0_3\">terminology<\/a>.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/wetransform.eu\/wp-content\/uploads\/2022\/05\/keyCollection.jpg\" alt=\"A big collection of keys - which one to use?\" \/><\/p>\n<p>To use a collector, there are two to three steps:<\/p>\n<ol>\n<li>Define the collector<\/li>\n<li>Define where to apply values from the collector<\/li>\n<li>Optionally, define Cell Execution priorities to make sure collected values are available when used<\/li>\n<\/ol>\n<p>We always collect values in the context of another transformation function. As of hale studio 3.2.0, these are the transformation functions that support the definition of collectors:<\/p>\n<ol>\n<li>Groovy Scripts and Groovy Script (Greedy)<\/li>\n<li>Groovy Retype<\/li>\n<li>Groovy Merge<\/li>\n<li>Groovy Join<\/li>\n<li>Custom Functions<\/li>\n<\/ol>\n<p>In all of these functions plus the following ones, you can apply the collected values:<\/p>\n<ol>\n<li>Groovy Create<\/li>\n<li>Assign Collected Values<\/li>\n<\/ol>\n<p>With the <a href=\"https:\/\/github.com\/halestudio\/hale\/milestone\/3\">upcoming 3.3.0 release<\/a>, you'll see more widespread support for the feature. Basically, most existing functions will allow collecting values, and we'll add more functions to assign them.<\/p>\n<h2>The Use Case<\/h2>\n<p>This is the use case we are going to work on for this tutorial:<\/p>\n<blockquote>\n<p>We need to create an <a href=\"https:\/\/www.haleconnect.com\/#\/schema\/user\/1\/275ae507-df43-4dc5-9037-783b5df4b927\/overview\">INSPIRE Hydrographical Network<\/a> dataset from UK Meridian 2 data encoded in a specific schema, using a GML 2.1 encoding. Each river segment from the source will be transformed to a <code>WatercourseLink<\/code>, and in addition, we'll create a <code>Network<\/code> object that references all created <code>WatercourseLink<\/code> features.<\/p>\n<\/blockquote>\n<p><img decoding=\"async\" src=\"https:\/\/wetransform.eu\/wp-content\/uploads\/2022\/05\/sourceAndTargetSchemas.png\" alt=\"The source and target schemas for this project\" \/><\/p>\n<p>You can take a look at the hale transformation project for this tutorial and download it, including source data, <a href=\"https:\/\/www.haleconnect.com\/#\/transformation\/user\/1\/f1410788-baff-4015-be48-e604ac21bbf0\/overview\">here at haleconnect.com<\/a>.<\/p>\n<h2>Implementation<\/h2>\n<h3>Step 1: Define the Collector in a <code>Groovy Script<\/code><\/h3>\n<ol>\n<li>As always, first define the Type-level transformation function. On the source schema, pick <code>River<\/code>, on the target side, select <code>WatercourseLink<\/code>. Click on the double arrow icon and select <code>Retype<\/code>. Use the default values for the function.<\/li>\n<li>Now, select <code>fid<\/code> on the source and <code>id<\/code> on the target feature type. Click on the double arrow icon and select <code>Groovy Script<\/code>. Leave the parameters on the first page as they are and click <code>Next<\/code> to proceed to the actual script editor. After you've entered the script, click <code>Finish<\/code> to let the transformation execute.<\/li>\n<\/ol>\n<p><img decoding=\"async\" src=\"https:\/\/wetransform.eu\/wp-content\/uploads\/2022\/05\/defineCollectorInGroovy.png\" alt=\"Use a Groovy Script function to create and collect IDs for the new features\" \/><\/p>\n<p>This is the actual script to use:<\/p>\n<p><script src=\"https:\/\/gist.github.com\/JBoudewijn\/63f3bce49cbfe9340343b97c827cc49d.js\"><\/script><\/p>\n<p><a href=\"https:\/\/github.com\/wetransform-os\/website-examples\/blob\/af7108ee8f5a838327c3e4b871850212c399e37c\/news\/resources\/collectors.groovy\" title=\"Download\">Download<\/a><\/p>\n<h3>Step 2: Use the collected values in an <code>Assign collected values<\/code> function<\/h3>\n<p>The easiest way to use values from a collector is to use the <code>Assign collected values<\/code> function. Follow these steps to use it:<\/p>\n<ol>\n<li>In the target schema, select the <code>Network<\/code> feature type. Click on the arrow icon and select <code>Create<\/code>. This function will create one or more objects of the target type from thin air. Create exactly one object.<\/li>\n<li>Next, click on the <code>elements<\/code> property in <code>Network<\/code> and then on the arrow icon. Choose <code>Assign collected values<\/code> and click <code>Next<\/code>. Enter the name of the collector we've defined in the script above (<code>linkIDs<\/code>) so that it can be accessed.<\/li>\n<\/ol>\n<p>The <code>Assign collected values<\/code> function has some special behaviour to automatically identify and create local references. If you inspect the created <code>network<\/code>, you'll see it now has 982 references that all look like this:<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/wetransform.eu\/wp-content\/uploads\/2022\/05\/assignCollector.png\" alt=\"Assign collected IDs to any ReferenceType to get local references\" \/><\/p>\n<h3>Step 3: Transform in the right order<\/h3>\n<p>hale studio, in principle, automatically determines execution order of all cells. In some cases, this may not have the desired effect, so you need to provide hints to the transformation engine what should happen first, and what should happen last. For collectors, it's important that the engine first completes collecting values before it tries to apply them in a different place. We do plan to recognize these cases automatically but for now, you'll have to assign cell execution priorities to make sure everything always works as expected.<\/p>\n<p>To ensure that the described steps are executed in the correct sequence, the execution priority has to be defined accordingly. The second mapping cell (<code>Create<\/code> on <code>Network<\/code>) has therefore to be set to a lower priority than the first mapping cell. This can be done via context menu in hale studio.<\/p>\n<p><img decoding=\"async\" src=\"https:\/\/wetransform.eu\/wp-content\/uploads\/2022\/05\/cellPriority.png\" alt=\"Edit the Cell Priority so that the function using the collector is executed last\" \/><\/p>\n<h2>Summary<\/h2>\n<p>With these three steps, you learned how to use the collector feature in hale\u00bbstudio. Let us know what you think of this feature and what we can do to improve its usability!<\/p>\n<p>Happy transforming!<\/p>\n","protected":false},"excerpt":{"rendered":"<p>Collectors are a powerful feature that we introduced in hale\u00bbstudio 3.1.0 and have since then expanded on. So, for what use cases should you be looking at collectors? Your target schema has a collection object, such as a Network, that needs to reference many or all objects of a different type, e.g. NetworkLinks You are [&hellip;]<\/p>\n","protected":false},"author":1,"featured_media":0,"menu_order":0,"comment_status":"closed","ping_status":"closed","template":"","meta":{"_acf_changed":false,"inline_featured_image":false,"footnotes":""},"categories":[48],"tags":[47,21,46],"class_list":["post-1700","news-and-events","type-news-and-events","status-publish","hentry","category-tutorials","tag-groovy","tag-halestudio","tag-tutorial"],"acf":[],"yoast_head":"<!-- This site is optimized with the Yoast SEO Premium plugin v27.3 (Yoast SEO v27.4) - https:\/\/yoast.com\/product\/yoast-seo-premium-wordpress\/ -->\n<title>Tutorial: Working with Collectors in hale\u00bbstudio - wetransform<\/title>\n<meta name=\"description\" content=\"Collectors are a powerful feature, introduced in hale\u00bbstudio 3.1.0 and then expanded on. Which use cases work best with collectors?\" \/>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/wetransform.to\/de\/neuigkeiten-veranstaltungen\/tutorial-collectors-halestudio\/\" \/>\n<meta property=\"og:locale\" content=\"de_DE\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Tutorial: Working with Collectors in hale\u00bbstudio\" \/>\n<meta property=\"og:description\" content=\"Collectors are a powerful feature, introduced in hale\u00bbstudio 3.1.0 and then expanded on. Which use cases work best with collectors?\" \/>\n<meta property=\"og:url\" content=\"https:\/\/wetransform.to\/de\/neuigkeiten-veranstaltungen\/tutorial-collectors-halestudio\/\" \/>\n<meta property=\"og:site_name\" content=\"wetransform\" \/>\n<meta property=\"article:modified_time\" content=\"2022-07-28T08:51:14+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/wetransform.eu\/wp-content\/uploads\/2022\/05\/keyCollection.jpg\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:site\" content=\"@wetransformto\" \/>\n<meta name=\"twitter:label1\" content=\"Gesch\u00e4tzte Lesezeit\" \/>\n\t<meta name=\"twitter:data1\" content=\"4\u00a0Minuten\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\\\/\\\/schema.org\",\"@graph\":[{\"@type\":\"WebPage\",\"@id\":\"https:\\\/\\\/wetransform.to\\\/de\\\/neuigkeiten-veranstaltungen\\\/tutorial-collectors-halestudio\\\/\",\"url\":\"https:\\\/\\\/wetransform.to\\\/de\\\/neuigkeiten-veranstaltungen\\\/tutorial-collectors-halestudio\\\/\",\"name\":\"Tutorial: Working with Collectors in hale\u00bbstudio - wetransform\",\"isPartOf\":{\"@id\":\"http:\\\/\\\/wetransform.to\\\/de\\\/#website\"},\"primaryImageOfPage\":{\"@id\":\"https:\\\/\\\/wetransform.to\\\/de\\\/neuigkeiten-veranstaltungen\\\/tutorial-collectors-halestudio\\\/#primaryimage\"},\"image\":{\"@id\":\"https:\\\/\\\/wetransform.to\\\/de\\\/neuigkeiten-veranstaltungen\\\/tutorial-collectors-halestudio\\\/#primaryimage\"},\"thumbnailUrl\":\"https:\\\/\\\/wetransform.eu\\\/wp-content\\\/uploads\\\/2022\\\/05\\\/keyCollection.jpg\",\"datePublished\":\"2017-02-20T16:31:53+00:00\",\"dateModified\":\"2022-07-28T08:51:14+00:00\",\"description\":\"Collectors are a powerful feature, introduced in hale\u00bbstudio 3.1.0 and then expanded on. Which use cases work best with collectors?\",\"breadcrumb\":{\"@id\":\"https:\\\/\\\/wetransform.to\\\/de\\\/neuigkeiten-veranstaltungen\\\/tutorial-collectors-halestudio\\\/#breadcrumb\"},\"inLanguage\":\"de\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\\\/\\\/wetransform.to\\\/de\\\/neuigkeiten-veranstaltungen\\\/tutorial-collectors-halestudio\\\/\"]}]},{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"https:\\\/\\\/wetransform.to\\\/de\\\/neuigkeiten-veranstaltungen\\\/tutorial-collectors-halestudio\\\/#primaryimage\",\"url\":\"https:\\\/\\\/wetransform.eu\\\/wp-content\\\/uploads\\\/2022\\\/05\\\/keyCollection.jpg\",\"contentUrl\":\"https:\\\/\\\/wetransform.eu\\\/wp-content\\\/uploads\\\/2022\\\/05\\\/keyCollection.jpg\"},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\\\/\\\/wetransform.to\\\/de\\\/neuigkeiten-veranstaltungen\\\/tutorial-collectors-halestudio\\\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\\\/\\\/wetransform.to\\\/de\\\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"News & Events\",\"item\":\"https:\\\/\\\/wetransform.to\\\/de\\\/neuigkeiten-veranstaltungen\\\/\"},{\"@type\":\"ListItem\",\"position\":3,\"name\":\"Tutorial: Working with Collectors in hale\u00bbstudio\"}]},{\"@type\":\"WebSite\",\"@id\":\"http:\\\/\\\/wetransform.to\\\/de\\\/#website\",\"url\":\"http:\\\/\\\/wetransform.to\\\/de\\\/\",\"name\":\"wetransform\",\"description\":\"Making environmental data useful and accessible\",\"publisher\":{\"@id\":\"http:\\\/\\\/wetransform.to\\\/de\\\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"http:\\\/\\\/wetransform.to\\\/de\\\/?s={search_term_string}\"},\"query-input\":{\"@type\":\"PropertyValueSpecification\",\"valueRequired\":true,\"valueName\":\"search_term_string\"}}],\"inLanguage\":\"de\"},{\"@type\":\"Organization\",\"@id\":\"http:\\\/\\\/wetransform.to\\\/de\\\/#organization\",\"name\":\"wetransform\",\"url\":\"http:\\\/\\\/wetransform.to\\\/de\\\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"de\",\"@id\":\"http:\\\/\\\/wetransform.to\\\/de\\\/#\\\/schema\\\/logo\\\/image\\\/\",\"url\":\"https:\\\/\\\/wetransform.to\\\/wp-content\\\/uploads\\\/2022\\\/07\\\/large-logo-whitebg.png\",\"contentUrl\":\"https:\\\/\\\/wetransform.to\\\/wp-content\\\/uploads\\\/2022\\\/07\\\/large-logo-whitebg.png\",\"width\":1024,\"height\":1024,\"caption\":\"wetransform\"},\"image\":{\"@id\":\"http:\\\/\\\/wetransform.to\\\/de\\\/#\\\/schema\\\/logo\\\/image\\\/\"},\"sameAs\":[\"https:\\\/\\\/x.com\\\/wetransformto\",\"https:\\\/\\\/www.linkedin.com\\\/company\\\/wetransform-gmbh\\\/\"]}]}<\/script>\n<!-- \/ Yoast SEO Premium plugin. -->","yoast_head_json":{"title":"Tutorial: Working with Collectors in hale\u00bbstudio - wetransform","description":"Collectors are a powerful feature, introduced in hale\u00bbstudio 3.1.0 and then expanded on. Which use cases work best with collectors?","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/wetransform.to\/de\/neuigkeiten-veranstaltungen\/tutorial-collectors-halestudio\/","og_locale":"de_DE","og_type":"article","og_title":"Tutorial: Working with Collectors in hale\u00bbstudio","og_description":"Collectors are a powerful feature, introduced in hale\u00bbstudio 3.1.0 and then expanded on. Which use cases work best with collectors?","og_url":"https:\/\/wetransform.to\/de\/neuigkeiten-veranstaltungen\/tutorial-collectors-halestudio\/","og_site_name":"wetransform","article_modified_time":"2022-07-28T08:51:14+00:00","og_image":[{"url":"https:\/\/wetransform.eu\/wp-content\/uploads\/2022\/05\/keyCollection.jpg","type":"","width":"","height":""}],"twitter_card":"summary_large_image","twitter_site":"@wetransformto","twitter_misc":{"Gesch\u00e4tzte Lesezeit":"4\u00a0Minuten"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"WebPage","@id":"https:\/\/wetransform.to\/de\/neuigkeiten-veranstaltungen\/tutorial-collectors-halestudio\/","url":"https:\/\/wetransform.to\/de\/neuigkeiten-veranstaltungen\/tutorial-collectors-halestudio\/","name":"Tutorial: Working with Collectors in hale\u00bbstudio - wetransform","isPartOf":{"@id":"http:\/\/wetransform.to\/de\/#website"},"primaryImageOfPage":{"@id":"https:\/\/wetransform.to\/de\/neuigkeiten-veranstaltungen\/tutorial-collectors-halestudio\/#primaryimage"},"image":{"@id":"https:\/\/wetransform.to\/de\/neuigkeiten-veranstaltungen\/tutorial-collectors-halestudio\/#primaryimage"},"thumbnailUrl":"https:\/\/wetransform.eu\/wp-content\/uploads\/2022\/05\/keyCollection.jpg","datePublished":"2017-02-20T16:31:53+00:00","dateModified":"2022-07-28T08:51:14+00:00","description":"Collectors are a powerful feature, introduced in hale\u00bbstudio 3.1.0 and then expanded on. Which use cases work best with collectors?","breadcrumb":{"@id":"https:\/\/wetransform.to\/de\/neuigkeiten-veranstaltungen\/tutorial-collectors-halestudio\/#breadcrumb"},"inLanguage":"de","potentialAction":[{"@type":"ReadAction","target":["https:\/\/wetransform.to\/de\/neuigkeiten-veranstaltungen\/tutorial-collectors-halestudio\/"]}]},{"@type":"ImageObject","inLanguage":"de","@id":"https:\/\/wetransform.to\/de\/neuigkeiten-veranstaltungen\/tutorial-collectors-halestudio\/#primaryimage","url":"https:\/\/wetransform.eu\/wp-content\/uploads\/2022\/05\/keyCollection.jpg","contentUrl":"https:\/\/wetransform.eu\/wp-content\/uploads\/2022\/05\/keyCollection.jpg"},{"@type":"BreadcrumbList","@id":"https:\/\/wetransform.to\/de\/neuigkeiten-veranstaltungen\/tutorial-collectors-halestudio\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/wetransform.to\/de\/"},{"@type":"ListItem","position":2,"name":"News & Events","item":"https:\/\/wetransform.to\/de\/neuigkeiten-veranstaltungen\/"},{"@type":"ListItem","position":3,"name":"Tutorial: Working with Collectors in hale\u00bbstudio"}]},{"@type":"WebSite","@id":"http:\/\/wetransform.to\/de\/#website","url":"http:\/\/wetransform.to\/de\/","name":"wetransform","description":"Making environmental data useful and accessible","publisher":{"@id":"http:\/\/wetransform.to\/de\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"http:\/\/wetransform.to\/de\/?s={search_term_string}"},"query-input":{"@type":"PropertyValueSpecification","valueRequired":true,"valueName":"search_term_string"}}],"inLanguage":"de"},{"@type":"Organization","@id":"http:\/\/wetransform.to\/de\/#organization","name":"wetransform","url":"http:\/\/wetransform.to\/de\/","logo":{"@type":"ImageObject","inLanguage":"de","@id":"http:\/\/wetransform.to\/de\/#\/schema\/logo\/image\/","url":"https:\/\/wetransform.to\/wp-content\/uploads\/2022\/07\/large-logo-whitebg.png","contentUrl":"https:\/\/wetransform.to\/wp-content\/uploads\/2022\/07\/large-logo-whitebg.png","width":1024,"height":1024,"caption":"wetransform"},"image":{"@id":"http:\/\/wetransform.to\/de\/#\/schema\/logo\/image\/"},"sameAs":["https:\/\/x.com\/wetransformto","https:\/\/www.linkedin.com\/company\/wetransform-gmbh\/"]}]}},"_links":{"self":[{"href":"https:\/\/wetransform.to\/de\/wp-json\/wp\/v2\/news-and-events\/1700","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/wetransform.to\/de\/wp-json\/wp\/v2\/news-and-events"}],"about":[{"href":"https:\/\/wetransform.to\/de\/wp-json\/wp\/v2\/types\/news-and-events"}],"author":[{"embeddable":true,"href":"https:\/\/wetransform.to\/de\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/wetransform.to\/de\/wp-json\/wp\/v2\/comments?post=1700"}],"version-history":[{"count":6,"href":"https:\/\/wetransform.to\/de\/wp-json\/wp\/v2\/news-and-events\/1700\/revisions"}],"predecessor-version":[{"id":3785,"href":"https:\/\/wetransform.to\/de\/wp-json\/wp\/v2\/news-and-events\/1700\/revisions\/3785"}],"wp:attachment":[{"href":"https:\/\/wetransform.to\/de\/wp-json\/wp\/v2\/media?parent=1700"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/wetransform.to\/de\/wp-json\/wp\/v2\/categories?post=1700"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/wetransform.to\/de\/wp-json\/wp\/v2\/tags?post=1700"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}