Fourth Workshop on Cognitive Knowledge Acquisition and Applications
Acquire knowledge from text and reason in novel situations in a Question Answering about Banking shared task.
THANK YOU SPONSORS!
Following the success of the well-attended First, Second, and Third Cognitum workshops at IJCAI, we are excited to continue this workshop series at NAACL HLT 2018 in New Orleans, Louisiana. The Fourth Cognitum will focus on the integration of natural language processing and knowledge representation and reasoning, which has proved to be a particularly thorny problem. We solicit papers on this problem as well as participants in a shared task involving knowledge acquisition from text and answering and explaining the answers to natural language banking questions that require reasoning to answer.
KEYNOTE SPEAKER: Peter Clark, Allen Institute for Artificial Intelligence
To facilitate natural and fruitful interaction with humans, cognitive systems must be able to learn, reason, and communicate in natural language. Ultimately, this interaction aims to extend and enhance human cognition, not by having cognitive systems operate as subsidiary workers that solve problems for humans, but by having cognitive systems act as expert assistants able to collaborate with humans and provide them with help in a form compatible with how humans naturally process and understand information.
Knowledge acquisition is central to the design of such cognitive systems. Knowledge should be in a form that allows cognitive systems to understand natural language questions, perform reasoning to answer questions, and explain their reasoning. Unlike the significant body of work on mining the web for facts or answers to specific questions (such as NELL and IBM’s Watson Jeopardy! system), the workshop’s emphasis is on the acquisition of general knowledge that can be applied by cognitive systems in novel situations to perform reasoning. At the same time, acquired knowledge should be cognitive knowledge, which exhibits characteristics similar to human knowledge and allows systems to explain their reasoning.
We welcome ongoing and exciting preliminary work. Topics of interest include, but are not limited to:
- Integrating natural language processing with knowledge representation and reasoning.
- Acquiring cognitive knowledge (knowledge in a form that supports explanation to humans).
- Formal frameworks for acquiring cognitive knowledge.
- Deep learning for acquiring cognitive knowledge.
- Principled evaluation of acquired cognitive knowledge.
- Psychologically-guided design of the acquisition process.
- Considerations related to scalability and parallelization.
- Active choice among available learning data/resources.
- Representation languages for cognitive knowledge.
- Static versus temporal/causal cognitive knowledge.
- Interaction of acquisition with natural language processing, perception, and reasoning.
- Alternative acquisition methods (such as crowdsourcing).
- Acquisition from media other than text (such as video).
- Architecture and implementation of cognitive systems.
- Real-world applications that utilize cognitive knowledge.
COGNITUM 2018 SHARED TASK 1: QUESTION ANSWERING ABOUT BANKING
Shared Task Data Set
The Capital One Social Media Questions and Answers data set consists of 4,810 question-answer pairs about banking. The question-answer pairs can be viewed at https://www.capitalone.com/social-media/answers/
Question: Can I pay my credit card bill from a European bank account?
Answer: Hey there! Payments would need to be made from a US banking institution using US based funds. You can find more info about our cards <a href=“https://www.capitalone.com/credit-cards/faq/”>Here</a>.
Shared Task Description
Submit a system that takes a previously unseen test question as input and returns 3 appropriate question-answer pairs from the data set as output. The system will be run on 10 test questions.
Question: Is it OK if I use my Australian bank account to pay my Quicksilver bill?
The system would return the above question-answer pair.
Shared Task Submission
To obtain the data set as a CSV file, please send team name, contact name, and contact email address to email@example.com
To submit to the shared task, one or more members of the team must register for and attend the Cognitum 2018 workshop in New Orleans, Louisiana, USA. The shared task will be conducted at the workshop. Teams are encouraged to submit (short or long) papers about their systems to the workshop.
The score for a system will be computed as follows: For each of the 10 input test questions, a human banking expert will rate the appropriateness of the 3 question-answer pairs output by the system using a 5-point Likert scale: “The question-answer pair is appropriate.” 0 = strongly disagree, 1 = disagree, 2 = neutral, 3 = agree, 4 = strongly agree. The system’s score (0-120) will be the sum of the appropriateness ratings.
Shared Task Prize
$1000 for the system with the highest score greater than 60, provided that 5 or more systems are submitted.
February 3, 2018: Shared task registration begins.
March 15, 2018: Workshop papers due. EXTENDED
April 9, 2018: Notification of workshop paper acceptance.
April 16, 2018: Camera-ready workshop papers due.
June 6, 2018: Workshop held at NAACL HLT 2018 in New Orleans, Louisiana, USA.
June 6, 2018: Shared task conducted at workshop.
July 6, 2018: Shared task winner announced.
WORKSHOP PAPER FORMAT GUIDELINES
Workshop paper submissions should follow the NAACL HLT 2018 style guidelines. Long paper submissions must follow the two-column format of ACL proceedings without exceeding eight (8) pages of content. Short paper submissions must also follow the two-column format of ACL proceedings, and must not exceed four (4) pages. References do not count against these limits. We strongly recommend the use of the official NAACL HLT 2018 style templates:
All submissions must be in PDF format.
WORKSHOP PAPER SUBMISSION
Workshop paper submission is electronic, using the Softconf START conference management system at:
Jason Alonso, Luminoso Technologies, Inc.
Ken Barker, IBM
Ernest Davis, New York University
James Fan, HelloVera.ai
Jonathan Gordon, USC Information Sciences Institute
Antonis Kakas, University of Cyprus
Zachary Kulis, Capital One
Margaret Mayer, Capital One
Rob Miller, University College London
J. William Murdock, IBM
Katerina Pastra, Cognitive Systems Research Institute
John Prager, IBM
Alessandra Russo, Imperial College London
Claudia Schulz, TU Darmstadt
Biplav Srivastava, IBM