Fourth Workshop on Cognitive Knowledge Acquisition and Applications
Acquire knowledge from text and reason in novel situations in a Question Answering about Banking shared task.
The Fourth Cognitum will focus on the integration of natural language processing and knowledge representation and reasoning, which has proved to be a particularly thorny problem. We solicit papers on this problem as well as participants in a shared task involving knowledge acquisition from text and answering and explaining the answers to natural language banking questions that require reasoning to answer.
To facilitate natural and fruitful interaction with humans, cognitive systems must be able to learn, reason, and communicate in natural language. Ultimately, this interaction aims to extend and enhance human cognition, not by having cognitive systems operate as subsidiary workers that solve problems for humans, but by having cognitive systems act as expert assistants able to collaborate with humans and provide them with help in a form compatible with how humans naturally process and understand information.
Knowledge acquisition is central to the design of such cognitive systems. Knowledge should be in a form that allows cognitive systems to understand natural language questions, perform reasoning to answer questions, and explain their reasoning. Unlike the significant body of work on mining the web for facts or answers to specific questions (such as NELL and IBM’s Watson Jeopardy! system), the workshop’s emphasis is on the acquisition of general knowledge that can be applied by cognitive systems in novel situations to perform reasoning. At the same time, acquired knowledge should be cognitive knowledge, which exhibits characteristics similar to human knowledge and allows systems to explain their reasoning.
We welcome ongoing and exciting preliminary work. Topics of interest include, but are not limited to:
- Integrating natural language processing with knowledge representation and reasoning.
- Acquiring cognitive knowledge (knowledge in a form that supports explanation to humans).
- Formal frameworks for acquiring cognitive knowledge.
- Deep learning for acquiring cognitive knowledge.
- Principled evaluation of acquired cognitive knowledge.
- Psychologically-guided design of the acquisition process.
- Considerations related to scalability and parallelization.
- Active choice among available learning data/resources.
- Representation languages for cognitive knowledge.
- Static versus temporal/causal cognitive knowledge.
- Interaction of acquisition with natural language processing, perception, and reasoning.
- Alternative acquisition methods (such as crowdsourcing).
- Acquisition from media other than text (such as video).
- Architecture and implementation of cognitive systems.
- Real-world applications that utilize cognitive knowledge.
COGNITUM SHARED TASK 1: QUESTION ANSWERING ABOUT BANKING
Shared Task Data Set
The Capital One Social Media Questions and Answers data set consists of 4,810 question-answer pairs about banking. The question-answer pairs can be viewed at https://www.capitalone.com/social-media/answers/
Question: Can I pay my credit card bill from a European bank account?
Answer: Hey there! Payments would need to be made from a US banking institution using US based funds. You can find more info about our cards <a href=“https://www.capitalone.com/credit-cards/faq/”>Here</a>.
Shared Task Description
Submit a system that takes a previously unseen test question as input and returns 3 appropriate question-answer pairs from the data set as output. The system will be run on 10 test questions.
Question: Is it OK if I use my Australian bank account to pay my Quicksilver bill?
The system would return the above question-answer pair.
The score for a system will be computed as follows: For each of the 10 input test questions, a human banking expert will rate the appropriateness of the 3 question-answer pairs output by the system using a 5-point Likert scale: “The question-answer pair is appropriate.” 0 = strongly disagree, 1 = disagree, 2 = neutral, 3 = agree, 4 = strongly agree. The system’s score (0-120) will be the sum of the appropriateness ratings.
Jason Alonso, Luminoso Technologies, Inc.
Ken Barker, IBM
Ernest Davis, New York University
James Fan, HelloVera.ai
Jonathan Gordon, USC Information Sciences Institute
Antonis Kakas, University of Cyprus
Zachary Kulis, Capital One
Margaret Mayer, Capital One
Rob Miller, University College London
J. William Murdock, IBM
Katerina Pastra, Cognitive Systems Research Institute
John Prager, IBM
Alessandra Russo, Imperial College London
Claudia Schulz, TU Darmstadt
Biplav Srivastava, IBM