NL4DV: Natural Language toolkit for Data Visualization | Overview

we present nl4dv a toolkit that helps prototype natural language interfaces for data visualizations there are a growing number of natural language based systems for data visualization

however developing these systems remains a challenging task to create natural language interfaces developers need to implement a user interface a natural language processing module that interprets

queries to infer relevant information such as data attributes and analytic tasks a visualization recommendation engine that determines appropriate visualizations based on the extracted information

and finally update the user interface to present the recommended charts to add visualization system developers and support rapid prototyping nl4gv provides a high-level python api for interpreting natural language

queries the api automates the core tasks of processing natural language queries to infer relevant information and determine appropriate visualizations allowing visualization developers to

focus more on designing and implementing the user interface given a tabular data set and a natural language query nl4db automatically infers data attributes and analytic tasks from the

query and also generates a list of vagalite specifications for visualizations relevant to the input query formalizing the inferred information into an analytic specification

nl4dv returns a json object that developers can parse to implement and support a range of natural language interaction capabilities across different scenarios by automatically interpreting vegalite specifications from natural language

queries nl4db enables creating visualizations in python-based data science programming environments such as jupyter notebook using natural language for example in this first query nl4dv generates a

box plot based on a fully specified query explicitly stating both the attribute and the visualization type to be used on the other hand the second query is under specified and only list attributes that the user wants to

visualize however detecting the attributes mentioned in the query nl4dv implicitly infers an appropriate visualization based on underlying data types and generates a colored scatter

plot nl4dv can also be used to develop web-based visualization systems here an l4 db is used to create a modified version of a vega light editor that supports natural language input and

can help novices start using vega light given a natural language query about a dataset with a single function call developers can get not just one chart but a ranked list of vagalite

specifications relevant to the query this list can then be parsed in javascript and presented as design alternatives that users can choose from and edit further

natural language queries are often ambiguous detecting ambiguities in queries however is challenging and requires writing custom nlp code to match different parts of a query to a data set and derive all

possible inferences nl4 dv internally handles challenges like ambiguity detection and highlights ambiguous references as part of its output json

as shown in this example this json can be parsed by developers to recreate features like attribute and value level ambiguity widgets from datatone which was an early example of a natural language interface for data

visualization in addition to developing new natural language interfaces nl4 dv can also help developers add natural language querying capabilities to complement wimp and direct manipulation-based

interactions in existing tools consider touch plot an example tablet-based visualization system that allows specifying and interacting with scatter plots using control panels and touch-based interactions

to improve the user experience we modified touch plot by incorporating speech input to augment touch interactions with this new multimodal system mm plot users can directly utter voice commands

for actions they wish to perform avoiding tedious and repetitive interactions with drop down menus and control panels show a scatterplot of age and salary for

players under the age of 30.

to augment touch plot with speech all a developer needs to do is provide the spoken query as a text string to nl4dv then similar to the previous examples with a single function call

nl4dv returns the desired visualization and analytic task that are parsed in javascript to make required changes to the view nl4dv is available as open source

software and we hope that it enables prototyping new natural language interfaces and interactions for data visualization