Planned Tasks

List of Tasks and their Dependencies

1. Uniform Data Storage

1. Dockerize IDA.
2. [depends on 1.1] Add RDF database service to container stack. (e.g. Blazegraph or Jena TDB)
3. [depends on 1.2] Uniform data format. Store all datasets in the RDF database. Maybe only limited to CSV->RDF conversion for now, since this is the only format relevant for the historians usecase.

2. User Experience

1. UI revamp:
   1. Clean up IDA chatbot interface: Give more space to data visualizations, reduce chat area.
   2. Redesign user management interface to be consistent with the IDA chatbot interface.
2. [depends on 1.2] User management. (user data stored in the RDF DB as well)
3. [depends on 1.3, 2.2] Allow users to upload custom datasets.
   1. Server-side: Implement server endpoint that accepts user data in various formats, converts it to RDF triples and persists it to the RDF DB.
   2. Client-side: Add an upload form.
4. [depends on 2.1, 2.2] User workspace persistence.
   1. Server-side: Store users chat history, opened datasets, visualizations and data analyses in the RDF DB.
   2. Client-side: Sync the users UI state with the server.

3. Code Generation

1. Implement an uninteractive code generator which gets a user request (task and parameters) and generates a code snippet that fulfills that request.
2. [depends on 3.1] Make the code generator interactive, i.e. if the generator does not have enough information to find a likely code snippet, ask the user for more specific requirements.

4. Modularization

1. Modularize visualization plugins
   1. Server-side: Design a plugin interface to allow the IDA core to communicate with external visualization data providers. The external visualization providers will most likely be moved to separate containers and communicate with IDA over the network.
   2. Client-side: Split all existing visualizations out of ida-chatbot into modular front-end bundles that can be dynamically loaded into the ida-chatbox frontend.
2. [depends on 3.1] Modularize library support: Add separate containers for each supported library which runs data analysis code which is generated by the code generator.

5. Natural Language Engine

1. Specify the technical architecture for a new natural language engine that should replace the current RiveScript implementation. The main aspects to consider are:
   • Specify how exactly the conversation state will be represented by the engine. Maybe take ideas from Google Dialogflow?
   • How will the engine interact with the visualization plugins? Specify interfaces for data exchange.
   • How will the engine interact with the code generator? Specify how the generator can ask for more input from the user and how that question will be translated to a natural language request to the user etc.

Planned tasks for SS 2019

Out of the listed tasks, these are planned for this semester with rough time estimates:

1. Uniform data storage
   • 1.1 #120
   • 1.2 (until 03.05.) #134
   • 1.3 (until 13.05.) #136
2. User experience
   • 2.1.1 (until 06.05.) #137
   • 2.1.2 (until 06.05.) #138
   • 2.2 (until 13.05.) #139
   • 2.3 (until 20.05.)
   • 2.4 (until 31.05.) shelved
3. Code generation
   • 3.1 (until 31.05. ? will maybe take a bit longer than that since research is involved) #135
   • 3.2 (? maybe not realistic for this semester, only if there is time left)
4. Modularization shelved
   • 4.1.1 (until 14.06. ? will maybe take a bit longer than that since research is involved)
   • 4.1.2 (until 28.06.)
5. Natural language engine
   • 5.1 Probably only start with the research and implementation, not sure yet how much will be completed by the end of the semester.