KNIME User Training ( PDFDrive )

Flow variables: Usage example • Each month I am asked to produce a sales report for the most popular product Filter only rows With Products = Gold Investment 3 151 Licensed under a Creative Commons Attribution- ® Copyright © 2017 KNIME AG Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Flow variables: Usage example • Each month I need to launch the Analytics Platform, run the workflow up to the GroupBy node, then get the most popular product and update the Row Filter. • Or do I? Perhaps flow variables can help… 4 152 Licensed under a Creative Commons Attribution- ® Copyright © 2017 KNIME AG Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Automatically filter by most popular product Count products, and Create a variable put most important at containing the most the top of the list popular product Pass the flow variable to the row filter 5 153 Licensed under a Creative Commons Attribution- ® Copyright © 2017 KNIME AG Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Flow Variable Ports 6 154 Licensed under a Creative Commons Attribution- ® Copyright © 2017 KNIME AG Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Apply a Flow Variable (button) The Flow Variable button 7 155 Licensed under a Creative Commons Attribution- ® Copyright © 2017 KNIME AG Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Apply a Flow Variable (advanced) The Flow Variable tab List of available Flow Variables 8 156 Licensed under a Creative Commons Attribution- ® Copyright © 2017 KNIME AG Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Create a Flow Variable (button) Name of the new Flow Variable 9 157 Licensed under a Creative Commons Attribution- ® Copyright © 2017 KNIME AG Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Create a Flow Variable (advanced) Converting a setting value into a Flow Variable Name of the new variable Copyright © 2017 KNIME AG 10 158 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Summary: Flow Variables • Flow Variables are workflow parameters used to overwrite existing node settings. • A Flow Variable is carried along workflow branches (parallel branches don’t share local flow variables). • Flow Variables can be of type String, Integer, or Double. • Flow Variables can be created in the “Flow Variables” tab of any node, using the “Table Row to Variable Node” or using QuickForms. Copyright © 2017 KNIME AG 11 159 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

New Node: Sorter • Sorts a table! • Choice of ascending or descending • Sort by multiple columns Copyright © 2017 KNIME AG 12 160 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Flow Variables Exercise: Activity I Starting with exercise: Flow Variables • Create a workflow that filters the products to contain only rows for the ‘Gold Investment’ product. • Find the most common product type and automatically filter the table according to that value. Copyright © 2017 KNIME AG 13 161 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Quickform Nodes for Variable Creation and Output Copyright © 2017 KNIME AG 14 162 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Quickform Node Configuration Use Quickforms to create Flow Variables Default value is selected (here P+B Investment) Copyright © 2017 KNIME AG 15 163 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Wrapped Metanodes • Similar to Metanodes • Differ in key areas: – Limited variable scope (c.f. global scope for Metanodes) – Use with Quick Form nodes (Analytics Platform 3.1+) • Key to advanced functionality in KNIME products: – Use for new WebPortal pages Copyright © 2017 KNIME AG 16 164 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Metanodes vs. Wrapped Metanodes Quick Forms Metanodes Wrapped Metanodes* Variable scope Legacy Standard WebPortal Execution Global Local JavaScript views in Old New (work with loops/switches) WebPortal Not supported Supported WebPortal Usage Quickforms used globally Views/Quickforms must be Recommended uses embedded in a Wrapped Metanode Compatibility Legacy workflows KNIME Server 3.x/4.x New developments KNIME Server 4.2+ * Valid for KNIME Analytics Platform 3.1 and above Copyright © 2017 KNIME AG 17 165 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Create a Wrapped Metanode • Select nodes to wrap • Right-click a node • Choose ‘Encapsulate into Wrapped Metanode’ Copyright © 2017 KNIME AG 18 166 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Simple configuration of Wrapped Metanode • Right-click a Wrapped Metanode to configure • Use in WebPortal Copyright © 2017 KNIME AG 19 167 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Configure Wrapped Metanode Ports • Add ports to metanodes or remove them to adapt to changes after creation of metanode Copyright © 2017 KNIME AG 20 168 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Passing Variables from Wrapped Metanodes • Flow variables by default only available locally within wrapped metanode • If needed, they can be passed to context outside metanode Copyright © 2017 KNIME AG 21 169 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Workflows with Reports in the KNIME WebPortal Step 1 Step 2 Select workflow to run, Run settings and set of configure email notification, workflow parameters and report formats Last Step Optionally: Embed See results and report Interactive JavaScript visualizations Available in KNIME Server 22 170 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license Copyright © 2017 KNIME AG https://creativecommons.org/licenses/by-nc-sa/4.0/

Flow Variables Exercise: Activity II Starting with exercise: Flow Variables • Create a Wrapped Metanode that allows users to filter records by choosing a value from the Products column. Copyright © 2017 KNIME AG 171 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Date/Time Data 1 172 Licensed under a Creative Commons Attribution- ® Copyright © 2017 KNIME AG Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Date & Time Overview • Dedicated data type for date and time data • Supported in Date&Time nodes • (and others: GroupBy, Pivot, Line Plot) • Complete re-write in KNIME 3.4 2 173 Licensed under a Creative Commons Attribution- ® Copyright © 2017 KNIME AG Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

New Node: String to Date&Time • Convert date/time data from string Select columns to into a native Date&time cell transform • Automaticall guesses correct format Enter date format for many different types of date manually formatting Click to auto-guess format • Enter format manually if auto- guessing didn‘t work • KNIME automatically adds custom formats to auto-guess list • Convert multiple columns of same date format in one node Select type of output column 3 174 Licensed under a Creative Commons Attribution- ® Copyright © 2017 KNIME AG Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Date&Time – Data Types Date Time Date & Time Date & Time + Time zone Copyright © 2017 KNIME AG 175 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

New Node: Date&Time-based Row Filter • Filter rows from a specified time period • Range can be limited on upper bound, lower bound or both • Options for end point: • Date&Time: Fixed data and time • Duration: Duration string (e.g. 2y 3M) • Numerical: Select granularity from dropdown and enter number 5 176 Licensed under a Creative Commons Attribution- ® Copyright © 2017 KNIME AG Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

New Node: String to Duration • Takes a string and converts it to a duration cell • Three different options to format input strings • Example: Convert 1 year, 2 months, 3 weeks, and 4 days to duration cell • ISO-8601: “P1Y2M3W4D“ • Short letter: “1y 2M 3w 4d” • Long word: “1 year 2 months 3 weeks 4 days” 6 177 Licensed under a Creative Commons Attribution- ® Copyright © 2017 KNIME AG Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Duration-based Filtering • Date&Time-based Row Filter allows to extract time periods • From the start date, select all rows within the defined period • Hours, days, weeks, months, etc. 1 Month 7 178 Licensed under a Creative Commons Attribution- ® Copyright © 2017 KNIME AG Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

New Node: Date&Time Difference • Choose desired resolution (days, hours, minutes, etc.) • Check the difference between a time column and… • Another time column • Execution time • User defined time • Time from previous row To calculate difference to second column, both columns need to have the same type! 8 179 Licensed under a Creative Commons Attribution- ® Copyright © 2017 KNIME AG Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

New Node: Date&Time Shift • Shifts date or time by either a Duration or a numerical value • Use Duration: – Use duration column – Or shift by user-defined value • E.g. 1y, 2M, 5h, etc. • Use Numerical in combination with user-defined granularity – Use numerical column – Or shift by user-defined value Select granularity Copyright © 2017 KNIME AG 180 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

New Nodes: Modify Time / Modify Date • Modify Date&Time columns • Three options: • Append time (date) to to date (time) column • Change time (date) to fixed value • Remove time (date) from Date&Time column • Column selection shows only columns suitable for currently selected option Copyright © 2017 KNIME AG 181 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Modify Time / Modify Date: Options Modify Time: Available options Modify Date: Available options • Input: Date • Input: Date – Append time – N/A • Input: Time • Input: Time – N/A – Append date • Input: Date&Time • Input: Date&Time – Change time; Remove time – Change date; Remove date • Input: Date&Time (Time zone) • Input: Date&Time (Time zone) – Change time; Remove time – Change date; Remove date Copyright © 2017 KNIME AG 182 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

New Node: Modify Time Zone • Similar to Modify Time/Modify Date • Input: Date&Time – Set time zone • Input: Date&Time (Time zone) – Set time zone – Shift time zone – Remove time zone Select time zone from dropdown list Copyright © 2017 KNIME AG 183 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Date and Time Analysis Exercise, Activity I Starting with exercise: Date and Time Analysis, Activity I • Read the file: meter_data.csv • Use String Manipulation to construct a date/timestamp column from the individual date and time columns • Convert the date/time stamp column to a KNIME timestamp with the String to Date&Time node • Extract entries from January 2007 using the Date&Time-based Row Filter Copyright © 2017 KNIME AG 13 184 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

New Node: Extract Date&Time Fields • Extract date fields (year, day, month) or time fields (hour, minute, second) from a date&time cell. • Pick and choose which fields to include • Useful when used in combination with data aggregation nodes (groupby, pivot etc.) Copyright © 2017 KNIME AG 14 185 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

New Node: Moving Average • Effectively a “smoothing” node • Smoothing defined by a window type (centered, forward or backward) and weighted or not • Useful when plotting aggregated time-series data to more easily see trends Copyright © 2017 KNIME AG 15 186 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

New Node: Moving Aggregation • Blend of GroupBy + Moving Average Functionality • Group by moving window • Aggregate using standard KNIME methods Copyright © 2017 KNIME AG 16 187 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

New Node: JavaScript Line Plot • Line plot with support for Date columns Copyright © 2017 KNIME AG 17 188 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Date and Time Analysis Exercise, Activity II Starting with exercise: Date and Time Analysis, Activity II • Read the file: sampled_meter_data.table • Use Extract Date&Time Fields to pull out the Year and Day of Year and Hour values from the timestamp • Use GroupBy to aggregate the data by year, day and hour. Group by these values and calculate mean time and intensity • Calculate a Gaussian centered moving average for the Intensity column and plot this data in a line plot • Calculate the Maximum of the intensity column for the preceding day (1440 minutes) Optional: Plot the raw intensity, moving average, and moving maximum in a Line Chart Copyright © 2017 KNIME AG 18 189 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Workflow Control Loops, switches, try-catch 1 190 Licensed under a Creative Commons Attribution- ® Copyright © 2017 KNIME AG Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Workflow Control Structures • Loops – Iterate over a workflow snippet with variable inputs. • Switches – Direct the path of a workflow by selectively executing one or more workflow branches. • Try-Catch – Handle workflow branches that may fail in execution and you don‘t know before execution 2 191 Licensed under a Creative Commons Attribution- ® Copyright © 2017 KNIME AG Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

The Loop Block • A loop block is defined by appropriate loop start and loop end nodes. • Loop body = Nodes in between and side branches. Loop body Loop start Loop node end node Copyright © 2017 KNIME AG 3 192 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

New Node: Group Loop Start • Similar to GroupBy except without aggregation tab. • Each iteration of the loop passes the next group of rows. • You implement the aggregation task. It can be anything from a complex calculation to updating a database. 4 193 Licensed under a Creative Commons Attribution- ® Copyright © 2017 KNIME AG Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

New Node: Create File Name • Inputs: Directory, base file name, file Extension • Output: Flow variable -> use as input for e.g. writer node 5 194 Licensed under a Creative Commons Attribution- ® Copyright © 2017 KNIME AG Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Example: Writing aggregated files • Group Loop Start  Variable Loop End • Group data by specific column values • Iterate over all groups of data • Create an appropriate file name • Write grouped data to tables with new file name 6 195 Licensed under a Creative Commons Attribution- ® Copyright © 2017 KNIME AG Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Workflow Control Exercise, Activity I Starting with exercise: Workflow Control, Activity I • Read the file: CurrentDetailData.table • Group over all of the values in the Products column • For each group of data, write a new KNIME table to your KNIME Workspace. Give it an appropriate filename. (Hint: Group Loop Start creates a flow variable naming the current group) 7 196 Licensed under a Creative Commons Attribution- ® Copyright © 2017 KNIME AG Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

New Node: List Files • List all files in a directory • Restrict to: • top level directory (i.e. not recursive), • specific file extensions • matching name patterns (regex or wildcard) • Provides file references as a table of URLs and absolute paths 8 197 Licensed under a Creative Commons Attribution- ® Copyright © 2017 KNIME AG Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

New Node: TableRow to Variable Loop Start • Similar to TableRow to Variable node. • Each iteration of the loop converts the next row of the input table into flow variables. • Inject variables into other nodes (often file readers) to re- execute subflows with a progression of settings 9 198 Licensed under a Creative Commons Attribution- ® Copyright © 2017 KNIME AG Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Example: Reading Many Files • List all files in a directory • Convert each file to a flow variable (1 per iteration) • In each iteration, read the file, collect the results Copyright © 2017 KNIME AG 10 199 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Workflow Control Exercise, Activity II Starting with exercise: Workflow Control, Activity II • Use List files to find the file names of the files created in exercise 2 • Iterate over that list of files, and read them into a single KNIME Table Copyright © 2017 KNIME AG 11 200 Licensed under a Creative Commons Attribution- ® Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/