FlowSense: A natural language interface for visual data exploration within a dataflow system

Research output: Contribution to journalArticle

Abstract

Dataflow visualization systems enable flexible visual data exploration by allowing the user to construct a dataflow diagram that composes query and visualization modules to specify system functionality. However learning dataflow diagram usage presents overhead that often discourages the user. In this work we design FlowSense, a natural language interface for dataflow visualization systems that utilizes state-of-The-Art natural language processing techniques to assist dataflow diagram construction. FlowSense employs a semantic parser with special utterance tagging and special utterance placeholders to generalize to different datasets and dataflow diagrams. It explicitly presents recognized dataset and diagram special utterances to the user for dataflow context awareness. With FlowSense the user can expand and adjust dataflow diagrams more conveniently via plain English. We apply FlowSense to the VisFlow subset-flow visualization system to enhance its usability. We evaluate FlowSense by one case study with domain experts on a real-world data analysis problem and a formal user study.

Original languageEnglish (US)
Article number8807265
Pages (from-to)1-11
Number of pages11
JournalIEEE Transactions on Visualization and Computer Graphics
Volume26
Issue number1
DOIs
StatePublished - Jan 2020

Fingerprint

Visualization
Flow visualization
Semantics
Processing

Keywords

  • Dataflow visualization system
  • Natural language interface
  • Visual data exploration

ASJC Scopus subject areas

  • Software
  • Signal Processing
  • Computer Vision and Pattern Recognition
  • Computer Graphics and Computer-Aided Design

Cite this

FlowSense : A natural language interface for visual data exploration within a dataflow system. / Yu, Bowen; Silva, Claudio T.

In: IEEE Transactions on Visualization and Computer Graphics, Vol. 26, No. 1, 8807265, 01.2020, p. 1-11.

Research output: Contribution to journalArticle

@article{feb0c467826247ebab8392a879a0ce79,
title = "FlowSense: A natural language interface for visual data exploration within a dataflow system",
abstract = "Dataflow visualization systems enable flexible visual data exploration by allowing the user to construct a dataflow diagram that composes query and visualization modules to specify system functionality. However learning dataflow diagram usage presents overhead that often discourages the user. In this work we design FlowSense, a natural language interface for dataflow visualization systems that utilizes state-of-The-Art natural language processing techniques to assist dataflow diagram construction. FlowSense employs a semantic parser with special utterance tagging and special utterance placeholders to generalize to different datasets and dataflow diagrams. It explicitly presents recognized dataset and diagram special utterances to the user for dataflow context awareness. With FlowSense the user can expand and adjust dataflow diagrams more conveniently via plain English. We apply FlowSense to the VisFlow subset-flow visualization system to enhance its usability. We evaluate FlowSense by one case study with domain experts on a real-world data analysis problem and a formal user study.",
keywords = "Dataflow visualization system, Natural language interface, Visual data exploration",
author = "Bowen Yu and Silva, {Claudio T.}",
year = "2020",
month = "1",
doi = "10.1109/TVCG.2019.2934668",
language = "English (US)",
volume = "26",
pages = "1--11",
journal = "IEEE Transactions on Visualization and Computer Graphics",
issn = "1077-2626",
publisher = "IEEE Computer Society",
number = "1",

}

TY - JOUR

T1 - FlowSense

T2 - A natural language interface for visual data exploration within a dataflow system

AU - Yu, Bowen

AU - Silva, Claudio T.

PY - 2020/1

Y1 - 2020/1

N2 - Dataflow visualization systems enable flexible visual data exploration by allowing the user to construct a dataflow diagram that composes query and visualization modules to specify system functionality. However learning dataflow diagram usage presents overhead that often discourages the user. In this work we design FlowSense, a natural language interface for dataflow visualization systems that utilizes state-of-The-Art natural language processing techniques to assist dataflow diagram construction. FlowSense employs a semantic parser with special utterance tagging and special utterance placeholders to generalize to different datasets and dataflow diagrams. It explicitly presents recognized dataset and diagram special utterances to the user for dataflow context awareness. With FlowSense the user can expand and adjust dataflow diagrams more conveniently via plain English. We apply FlowSense to the VisFlow subset-flow visualization system to enhance its usability. We evaluate FlowSense by one case study with domain experts on a real-world data analysis problem and a formal user study.

AB - Dataflow visualization systems enable flexible visual data exploration by allowing the user to construct a dataflow diagram that composes query and visualization modules to specify system functionality. However learning dataflow diagram usage presents overhead that often discourages the user. In this work we design FlowSense, a natural language interface for dataflow visualization systems that utilizes state-of-The-Art natural language processing techniques to assist dataflow diagram construction. FlowSense employs a semantic parser with special utterance tagging and special utterance placeholders to generalize to different datasets and dataflow diagrams. It explicitly presents recognized dataset and diagram special utterances to the user for dataflow context awareness. With FlowSense the user can expand and adjust dataflow diagrams more conveniently via plain English. We apply FlowSense to the VisFlow subset-flow visualization system to enhance its usability. We evaluate FlowSense by one case study with domain experts on a real-world data analysis problem and a formal user study.

KW - Dataflow visualization system

KW - Natural language interface

KW - Visual data exploration

UR - http://www.scopus.com/inward/record.url?scp=85075756323&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85075756323&partnerID=8YFLogxK

U2 - 10.1109/TVCG.2019.2934668

DO - 10.1109/TVCG.2019.2934668

M3 - Article

C2 - 31443010

AN - SCOPUS:85075756323

VL - 26

SP - 1

EP - 11

JO - IEEE Transactions on Visualization and Computer Graphics

JF - IEEE Transactions on Visualization and Computer Graphics

SN - 1077-2626

IS - 1

M1 - 8807265

ER -