KNIME User Training ( PDFDrive )

KNIME User Training KNIME AG Copyright © 2017 KNIME AG

Overview KNIME Analytics Platform Copyright © 2017 KNIME AG Licensed under a Creative Commons Attribution- ® 1 2 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

What is KNIME Analytics Platform? • A tool for data analysis, manipulation, visualization, and reporting • Based on the graphical programming paradigm • Provides a diverse array of extensions: • Text Mining • Network Mining • Cheminformatics • Weka machine learning • Many integrations, such as Java, R, Python, etc. Copyright © 2017 KNIME AG Licensed under a Creative Commons Attribution- ® 2 3 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Additional Resources KNIME pages (www.knime.org) • SOLUTIONS for example workflows • RESOURCES/LEARNING HUB www.knime.org/learning-hub • RESOURCES/NODE GUIDE https://www.knime.org/nodeguide KNIME Tech pages (tech.knime.org) • FORUM for questions and answers • DOCUMENTATION for docs, FAQ, changelogs, ... • COMMUNITY CONTRIBUTIONS for dev instructions and third party nodes KNIME TV on YouTube https://www.youtube.com/user/KNIMETV Copyright © 2017 KNIME AG Licensed under a Creative Commons Attribution- ® 3 4 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

The KNIME® Analytics Platform Copyright © 2017 KNIME AG Licensed under a Creative Commons Attribution- ® 4 5 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Visual KNIME Workflows NODES perform tasks on data Inputs Outputs Not Configured Status Idle Executed Error Nodes are combined to create WORKFLOWS Copyright © 2017 KNIME AG Licensed under a Creative Commons Attribution- ® 5 6 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Data Access • Databases ® Copyright © 2017 KNIME AG • MySQL, PostgreSQL • any JDBC (Oracle, DB2, MS SQL Server) • Files • Csv, txt • Excel, Word, PDF • SAS, SPSS • XML • PMML • Images, texts, networks, chem • Web, Cloud • REST, Web services • Twitter, Google Licensed under a Creative Commons Attribution- 6 7 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Big Data • Spark • HDFS support • Hive • Impala • HP Vertica • In-database processing Copyright © 2017 KNIME AG Licensed under a Creative Commons Attribution- ® 7 8 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Transformation • Preprocessing • Row, column, matrix based • Data blending • Join, concatenate, append • Aggregation • Grouping, pivoting, binning • Feature Creation and Selection Copyright © 2017 KNIME AG Licensed under a Creative Commons Attribution- ® 8 9 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Analyze & Data Mining • Regression • Linear, logistic • Classification • Decision tree, ensembles, SVM, MLP, Naïve Bayes • Clustering • k-means, DBSCAN, hierarchical • Validation • Cross-validation, scoring, ROC • Misc • PCA, MDS, item set mining • External • R, Weka Copyright © 2017 KNIME AG Licensed under a Creative Commons Attribution- ® 9 10 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Visualization • Interactive Copyright © 2017 KNIME AG • Scatter plot, histogram, pie charts, box plot • Highlighting (brushing) • JFreeChart • JavaScript • Misc • Tag cloud, open street map, networks, molecules • External •R 10 Licensed under a Creative Commons Attribution- ® 11 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Deployment • Database • Files Copyright © 2017 KNIME AG • Excel, csv, txt • XML • PMML • to: local, KNIME Server, SSH-, FTP-Server • BIRT Reporting 11 Licensed under a Creative Commons Attribution- ® 12 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Over 1500 native and embedded nodes included: Data Access Transformation Analysis & Mining Visualization Deployment MySQL, Oracle, ... Row, Statistics R via BIRT SAS, SPSS, ... Column Data Mining JFreeChart PMML Excel, Flat, ... Matrix Machine Learning JavaScript XML, JSON Hive, Impala, ... Text, Image Web Analytics Community / 3rd Databases XML, JSON, PMML Time Series Text Mining Excel, Flat, etc. Text, Doc, Image, ... Java Network Analysis Text, Doc, Image Web Crawlers Python Social Media Analysis Industry Specific Industry Specific Community / 3rd R, Weka, Python Community / 3rd Community / 3rd Community / 3rd Copyright © 2017 KNIME AG 12 Licensed under a Creative Commons Attribution- ® 13 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Overview • Installing KNIME Analytics Platform • The KNIME Workspace • The KNIME File Extensions • The KNIME Workbench • Workflow editor • Explorer • Node repository • Node description • Installing new features Copyright © 2017 KNIME AG 13 Licensed under a Creative Commons Attribution- ® 14 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Install KNIME Analytics Platform • Select the KNIME version for your computer: • Mac, Win, or Linux and 32 / 64bit • Note different downloads (minimal or full) • Download archive and extract the file, or download installer package and run it Copyright © 2017 KNIME AG 14 Licensed under a Creative Commons Attribution- ® 15 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Start KNIME Analytics Platform • Go to the installation directory and launch KNIME, or use the shortcut created on your Desktop. Copyright © 2017 KNIME AG 15 Licensed under a Creative Commons Attribution- ® 16 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

The KNIME Workspace • The workspace is the folder/directory in which workflows (and potentially data files) are stored for the current KNIME session. • Workspaces are portable (just like KNIME) Copyright © 2017 KNIME AG 16 Licensed under a Creative Commons Attribution- ® 17 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Welcome Page Copyright © 2017 KNIME AG 17 Licensed under a Creative Commons Attribution- ® 18 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

The KNIME Workbench Servers and Workflows Workflow Editor Node Recommendations Node Description Node Repository Console Outline Copyright © 2017 KNIME AG 18 Licensed under a Creative Commons Attribution- ® 19 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Creating New Workflows, Importing and Exporting • Right-click Workspace in KNIME Explorer to create new workflow or workflow group or to import workflow • Right-click on workflow or workflow group to export Copyright © 2017 KNIME AG Licensed under a Creative Commons Attribution- ® 20 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

KNIME File Extensions • Dedicated file extensions for Workflows and Workflow groups associated with KNIME Analytics Platform • *.knwf for KNIME Workflow Files • *.knar for KNIME Archive Files Copyright © 2017 KNIME AG 20 Licensed under a Creative Commons Attribution- ® 21 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

More on Nodes… A node can have 3 states: Idle: The node is not yet configured and cannot be executed with its current settings. Configured: The node has been set up correctly, and may be executed at any time Executed: The node has been successfully executed. Results may be viewed and used in downstream nodes. Copyright © 2017 KNIME AG 21 Licensed under a Creative Commons Attribution- ® 22 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Inserting and Connecting Nodes • Insert nodes into workspace by dragging them from Node Repository or by double-clicking in Node Repository • Connect nodes by left-clicking output port of Node A and dragging the cursor to (matching) input port of Node B • Common port types: Model Image Flow Variable Data Database Database Conection Query Copyright © 2017 KNIME AG 22 Licensed under a Creative Commons Attribution- ® 23 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Node Configuration • Most nodes require configuration • To access a node configuration window: • Double-click the node • Right-click > Configure Copyright © 2017 KNIME AG 23 Licensed under a Creative Commons Attribution- ® 24 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Node Execution • Right-click node • Select Execute in context menu • If execution is successful, status shows green light • If execution encounters errors, status shows red light Copyright © 2017 KNIME AG 24 Licensed under a Creative Commons Attribution- ® 25 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Node Views • Right-click node • Select Views in context menu • Select output port to inspect execution results Plot View Copyright © 2017 KNIME AG 25 Licensed under a Creative Commons Attribution- Data View 26 Noncommercial-Share Alike license ® https://creativecommons.org/licenses/by-nc-sa/4.0/

Workflow Coach • Recommendation engine – It gives hints about which node use next in the workflow – Based on KNIME communities' usage statistics – Usage statistics available also with Personal Productivity Extension and KNIME Server products (these products require a purchased license) Copyright © 2017 KNIME AG 26 Licensed under a Creative Commons Attribution- ® 27 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Getting Started: KNIME Example Server • Public repository with large selection of example workflows for many, many applications • Connect via KNIME Explorer Copyright © 2017 KNIME AG 27 Licensed under a Creative Commons Attribution- ® 28 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Online Node Guide • Workflows from Example Server also available online – https://www.knime.org/nodeguide Copyright © 2017 KNIME AG 28 Licensed under a Creative Commons Attribution- ® 29 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Hot Keys (for future reference) Task Hot key Description Node Configuration F6 opens the configuration window of the selected node F7 executes selected configured nodes Node Execution Shift + F7 executes all configured nodes Shift + F10 executes all configured nodes and opens all views Move Nodes and F9 cancels selected running nodes Annotations Shift + F9 cancels all running nodes Ctrl + Shift + Arrow moves the selected node in the arrow direction Workflow Operations Ctrl + Shift + moves the selected annotation in the front or in the back PgUp/PgDown of all overlapping annotations Meta-node F8 resets selected nodes Ctrl + S saves the workflow Copyright © 2017 KNIME AG Ctrl + Shift + S saves all open workflows Ctrl + Shift + W closes all open workflows Shift + F12 opens meta-node wizard 29 Licensed under a Creative Commons Attribution- ® 30 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Importing Data Accessing files and databases Copyright © 2017 KNIME AG Licensed under a Creative Commons Attribution- ® 1 31 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Data Source Nodes Output port Typically characterized by: • Orange color • No input ports, 1-2 output ports Status Copyright © 2017 KNIME AG Node description ® Licensed under a Creative Commons Attribution- 2 32 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

New Node: File Reader Workhorse of the KNIME Source nodes • Reads text based files • Many advanced features allow it to read most ‘weird’ files • Short lines, inline comments, headers and special encoding YouTube KNIME TV Channel video: 3 Licensed under a Creative Commons Attribution- ® https://youtu.be/flaHQw-Qhlg 33 Noncommercial-Share Alike license Copyright © 2017 KNIME AG https://creativecommons.org/licenses/by-nc-sa/4.0/

File Reader Configuration File path Basic Advanced Settings Settings Preview Help Button ® Copyright © 2017 KNIME AG Licensed under a Creative Commons Attribution- 4 34 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Alternative Faster Way … Drag & Drop OR Copy & Paste Copyright © 2017 KNIME AG Licensed under a Creative Commons Attribution- ® 5 35 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Filenames and the knime:// protocol Absolute URL Mountpoint-relative URL Local path Copyright © 2017 KNIME AG Licensed under a Creative Commons Attribution- ® 36 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Workflow Relative File Paths • Best choice if workflows are to be shared • Requires matching folder structure within workflow group • Independent of environment outside of workflow group Example: Path to „Sentiment Analysis.table“ • Local path: C:\\Users\\rb\\knime-workspace\\KNIMEUserTraining\\data\\Sentiment Analysis.table • Workflow relative: YouTube KNIME TV Channel: https://youtu.be/U9sP4g4yGwY Copyright © 2017 KNIME AG 7 Licensed under a Creative Commons Attribution- ® 37 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

New Node: Excel Reader (XLS) • Reads .xls and .xlsx file from Microsoft Excel – Supports reading from multiple sheets Copyright © 2017 KNIME AG Licensed under a Creative Commons Attribution- ® 8 38 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Excel Reader Configuration File path Sheet Preview ® specific settings Licensed under a Creative Commons Attribution- Copyright © 2017 KNIME AG 9 39 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

New Node: Table Reader • Reads tables from the native KNIME Format. – Maximum performance, minimum configuration File path YouTube KNIME TV channel video: 10 Licensed under a Creative Commons Attribution- ® https://youtu.be/tid1qi2HAOo 40 Noncommercial-Share Alike license Copyright © 2017 KNIME AG https://creativecommons.org/licenses/by-nc-sa/4.0/

Database Connectivity • Read data from any JDBC enabled database • Write your own SQL or model it using dedicated nodes Copyright © 2017 KNIME AG 11 Licensed under a Creative Commons Attribution- ® 41 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

New Nodes: Database Connectors • Native: Postgres, MySQL, SQLite, MSSQL (SQL Server) • Database Connector (e.g. Oracle, DB2, HANA). • Commercial extensions: HIVE and Impala Copyright © 2017 KNIME AG Licensed under a Creative Commons Attribution- ® 12 42 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

New Node: SQLite • Propagate connection information to other DB nodes • File-based database • Useful for prototyping (switch to real connector on deployment) Copyright © 2017 KNIME AG Licensed under a Creative Commons Attribution- ® 13 43 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

New Node: Database Table Selector • Take connection information and construct a query • Explore DB metadata • Output is an SQL query Copyright © 2017 KNIME AG 14 Licensed under a Creative Commons Attribution- ® 44 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

New Node: Database Connection Table Reader • Executes SQL Query • Reads results into a KNIME table Copyright © 2017 KNIME AG 15 Licensed under a Creative Commons Attribution- ® 45 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Other Useful Data Sources • PMML Reader – reads standard predictive models • XML Reader with XPATH support • Python/R Source nodes • SAS7BDAT (Labs) • REST/SOAP for web services, and many more Copyright © 2017 KNIME AG 16 Licensed under a Creative Commons Attribution- ® 46 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Importing Data Exercise Starting with exercise: Importing Data Read the following files – Sentiment Analysis.table – Sentiment Rating.csv – Product Data2.xls Optional: Read table web_activity from the database WebActivity.sqlite (hint: drag and drop the file to your workflow to get started) Copyright © 2017 KNIME AG 17 Licensed under a Creative Commons Attribution- ® 47 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Data Manipulation Clean, join, aggregate Copyright © 2017 KNIME AG Licensed under a Creative Commons Attribution- ® 1 48 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

Data Manipulation Nodes • Yellow color with a variety of input and output ports • Apply a transformation to input data • Many, many nodes! Copyright © 2017 KNIME AG Licensed under a Creative Commons Attribution- ® 2 49 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/

New Node: Concatenate Combine rows from 2 tables with shared columns • Handles duplicate row keys gracefully • Take the union or intersection of columns Copyright © 2017 KNIME AG Licensed under a Creative Commons Attribution- ® 3 50 Noncommercial-Share Alike license https://creativecommons.org/licenses/by-nc-sa/4.0/