Events

  • CRAC: NAACL Workshop on Computational Models of Reference, Anaphora, and Coreference - June 6th, 2018, New Orleans

    Workshop description

    Background There has been a lot of research activity in anaphora / coreference resolution in recent years, but once the DAARC series ended, there were no events in Computational Linguistics entirely dedicated to this type of work. The Coreference Beyond Ontonotes (CORBON) workshops held in 2016 (with NAACL) and 2017 (with EACL) partially addressed this need, but their focus was primarily on under-investigated coreference phenomena. The CRAC series of workshops is intended to be an event of interest to the entire anaphora / coreference / reference community.

    Objectives The aim of the proposed workshop(s) is to provide a forum where work on all aspects of computational work on anaphora resolution and annotation, including both coreference and types of anaphora such as bridging references resolution and discourse deixis, can be presented. We also intend to attract work on reference (e.g., deictical reference to objects displayed in a multimodal interface).

    Important dates

    The important dates for CRAC are the same as for every other NAACL 2018 workhop:
    • Deadline for submissions: Extended to 9 March 2018 (11:59pm Hawaii Standard Time)
    • Notification of acceptance: 2 April 2018
    • Camera-ready version due: 16 April 2018
    • Workshop date: 6th June 2018

    Data and Resources

    The ARRAU corpus is distributed by LDC, that will make it available to the participants to this shared task. Participants should register for the shared task by email to Yulia Grishina (grishina@uni-potsdam.de) and Massimo Poesio (m.poesio@qmul.ac.uk). After registering, participants should download the Agreement form from this site, sign it, and send it to LDC, which will then release the data.

    Program

    • 9:00-9:10 Welcome (Massimo Poesio, Vincent Ng, Maciej Ogrodniczuk)
    • 9:10-10:30 Session 1
      • 9:10-10:00 Invited Talk: Ana Marasovic. Resolving Abstract Anaphors in Discourse — Uphill Battles with Neural Ranking Models and Automatic Data Extraction

        Abstract: Abstract anaphora resolution (AAR) is a challenging task that aims to resolve anaphoric reference of pronominal and nominal expressions that refer to abstract objects like facts, events, propositions, actions or situations, in the (typically) preceding discourse. A central property of abstract anaphora is that it establishes a relation between the anaphor embedded in the anaphoric sentence and its (typically non-nominal) antecedent. In this talk, I will present a mention-ranking model that learns how abstract anaphors relate to their antecedents with an LSTM-Siamese Net. I will describe how we harvested training data from a parsed corpus using a common syntactic pattern consisting of a verb with an embedded sentential argument. I will show results of the mention-ranking model trained for shell noun resolution and results on an abstract anaphora subset of the ARRAU corpus. The latter corpus presents a greater challenge due to a mixture of nominal and pronominal anaphors and a greater range of confounders. Finally, I will talk about ideas on how the training data extraction method and the mention-ranking model could be further improved for the challenges ahead.

      • 10:00-10:30, Berfin Aktaş, Tatjana Scheffler and Manfred Stede. Anaphora Resolution for Twitter Conversations: An Exploratory Study
    • 10:30-11:00 Coffee break
    • 11:00-12:30 Session 2: Shared Talk, Plural Reference
      • 11:00-11:30 Massimo Poesio, Yulia Grishina, Varada Kolhatkar, Nafise Moosavi, Ina Roesiger, Adam Roussel, Fabian Simonjetz, Alexandra Uma, Olga Uryupina, Juntao Yu and Heike Zinsmeister. Anaphora Resolution with the ARRAU Corpus
      • 11:30-12:00 Ina Roesiger. Rule and learning-based models for bridging resolution in the ARRAU corpus
      • 12:00-12:30 Amir Zeldes. A Predictive Model for Notional Anaphora in English.
    • 12:30-14:00 Lunch
    • 14:00-15:30 Session 3: Bridging, Discourse deixis, Anaphora in German, Corpus annotation 1.
      • 14:00-14:20 Ina Roesiger, Maximilian Köper, Kim Anh Nguyen and Sabine Schulte im Walde. Integrating Predictions from Neural-Network Relation Classifiers into Coreference and Bridging Resolution.
      • 14:20-14:50 Janis Pagel and Ina Roesiger. Towards Bridging Resolution in German: Data Analysis and Rule-based Experiments.
      • 14:50-15:10 Adam Roussel. Detecting and Resolving Shell Nouns in German.
      • 15:10-15:30 Anna Nedoluzhko, Michal Novák and Maciej Ogrodniczuk. PAWS: A Multi-lingual Parallel Treebank with Anaphoric Relations.
    • 15:30-16:00 Mid-afternoon snack
    • 16:00-17:30 Session 4: Corpus Annotation 2, Cognitive Models
      • 16:00-16:30 Michal Novák. A Fine-grained Large-scale Analysis of Coreference Projection.
      • 16:30-17:00 Jixing Li, Murielle Fabre, Wen-Ming Luh and John Hale. Modeling Brain Activity Associated with Pronoun Resolution in English and Chinese
      • 17:00-17:30 Sharid Loáiciga, Luca Bevacqua, Hannah Rohde and Christian Hardmeier. Event versus entity co-reference: Effects of context and form of referring expression.

    Travel awards

    A limited number of travel awards are available to student authors of accepted CRAC papers who are affiliated with U.S. institutions.

    Enquiries

    For all enquiries, please contact m.poesio@qmul.ac.uk

    Workshop Organizers

    Co-chairs:

    • Massimo Poesio (Queen Mary University of London, UK)
    • Maciej Ogrodniczuk (Institute of Computer Science, Polish Academy of Sciences, Poland)
    • Vincent Ng (University of Texas at Dallas, USA)

    Program committee:

    • Anders Bjorkelund, University of Stuttgart
    • Antonio Branco,  University of Lisbon
    • Dan Cristea, A. I. Cuza University of Iasi
    • Pascal Denis, MAGNET, INRIA Lille Nord-Europe
    • Sobha Lalitha Devi, AU-KBC Research Center, Anna University of Chennai
    • Yulia Grishina, University of Potsdam
    • Veronique Hoste, Ghent University
    • Ryu Iida, National Institute of Information and Communications Technology (NICT), Kyoto
    • Varada Kolhatkar, Simon Fraser University
    • Katja Markert, Heidelberg University
    • Costanza Navaretta, University of Copenhagen
    • Anna Nedoluzhko, Charles University in Prague
    • Michal Novak,  Charles University in Prague
    • Simone Paolo Ponzetto, University of Mannheim
    • Sameer Pradhan, cemantix.org and Boulder Learning Inc.
    • Marta Recasens, Google Inc.
    • Dan Roth, University of Pennsylvania.
    • Veselin Stoyanov, Facebook
    • Olga Uryupina, University of Trento
    • Yannick Versley, IBM
    • Sam Wiseman, Harvard University
    • Heike Zinsmeister, University of Hamburg

    Shared Task

    The proposed (series of( workshop(s) will be accompanied, like CORBON, by a shared task. The shared task(s) in 2018 will be on anaphora resolution using ARRAU (Uryupina et al, to appear), a corpus designed to

    • cover a more diverse range of genres than typical coreference corpora, including news, task-oriented dialogue, and narrative;
    • provide annotations for all noun phrases, including non-referring NPs and singletons (unlike in OntoNotes where only coreferential mentions are an- notated) and carefully annotated their boundaries catering also for non- continuous mentions;
    • include annotation of the full range of anaphoric phenomena, including bridging and discourse deixis;
    • include the annotation of a variety of morpho-syntactic and semantic attributes of mentions, including genericity.

    For more details, see the shared task page .

    Shared Task Organizers

    • Yulia Grishina, University of Potsdam (chair)
    • Varada Kolhatkar, Simon Fraser University
    • Anna Nedoluzhko, Charles University in Prague
    • Massimo Poesio, Queen Mary University of London
    • Adam Roussel, University of the Ruhr at Bochum
    • Fabian Simonjetz, University of the Ruhr at Bochum
    • Olga Uryupina, University of Trento
    • Heike Zinsmeister, University of Hamburg

    Previous Events

    • CORBON '17
      CORBON 2017 was the 2nd edition of the workshop on Coreference Resolution Beyond OntoNotes, co-located with EACL 2017
    • CORBON '16
      CORBON 2017 was the 1st edition of the workshop on Coreference Resolution Beyond OntoNotes, co-located with NAACL 2016

  • Games4NLP: Games and Gamification for Natural Language Processing

    7 May 2018, Phoenix Seagaia Resort, Miyazaki (Japan)

    Workshop description

    It is our pleasure to announce the Games and Gamification for NLP (Games4NLP) workshop hosted at the 11th edition of the Language Resources and Evaluation Conference (LREC), 7-12 May 2018, Miyazaki (Japan).

    The Games4NLP workshop aims to promote and explore the possibilities for research and practical applications of using games and gamification for the creation of language resources for Natural Language Processing. The main objective is to provide a forum for researchers and practitioners to discuss and share ideas regarding how the NLP research community can benefit from using game and gamification strategies.

    The potential topics of interest include, but are not limited to:

    • Games for collecting data useful for NLP (games in planning, under development, or deployed);
    • Games for under-resourced languages such as many Asian languages;
    • Analysis of game data from established games;
    • Gamification of NLP tasks (techniques, best practice and evaluation of gamification strategies);
    • Player motivation and experience (strategies for recruitment/retention and player profiling);
    • Game design (conceptual design elements and reports of success and/or failure of designs);
    • Processing NLP game data (aggregation methods and strategies for minimising noise and cheating);
    • Use of NLP game data (how game-generated data has been used for NLP applications);
    • Evaluation of games for NLP (metrics for evaluating game performance, evaluating player performance, motivation and bias, and evaluating task difficulty);
    • Directive strategies that attempt to use game metrics in real-time.

    The workshop will be an interactive and dynamic full-day event that will be a mix of keynotes, paper and demo presentations as well as discussion sessions. The workshop will be held on Monday 7 May 2018 in the Phoenix Seagaia Resort, Miyazaki (Japan).

    Program (0900-1300 7 May 2018)

    09:00-09:10 Introduction by Workshop Chair

    09.10-10.30 Session 1 (Chair: Karën Fort)
    Invited talk: Did you also spend the last weekend playing your NLP game? (Ivan Habernal) Slides
    Faseeh: A Serious Game for Arabic Syn-onym Acquisition (Hend Al-Khalifa, Hadil Faisal and Rawan N. Al-Matham)
    Testing TileAttack with Three Key Audiences (Chris Madge, Massimo Poesio, Udo Kruschwitz and Jon Chamberlain)

    10:30-10.50 Coffee break

    10.50-11.50 Session 2 (Chair: Jon Chamberlain)
    LieCatcher: Game Framework for Collecting Human Judgments of Deceptive Speech (Sarah Ita Levitan, James Shin, Ivy Chen and Julia Hirschberg) Slides
    Cheap, fast and good! Voting Games with a Purpose (Karën Fort, Mathieu Lafourcade and Nathalie Le Brun) Slides
    The JeuxDeMots Project is 10 Years Old: what Assessments? (Alain Joubert, Mathieu Lafourcade and Nathalie Le Brun) Slides

    12.30-12.50 Project Updates
    enetCollect COST Action (Verena Lyding and Lionel Nicolas) Slides
    Lingo Boingo (Christopher Cieri) Slides

    12.50-13.00 Concluding Remarks by Workshop Chair

    Workshop Proceedings

    Invited talk: Did you also spend the last weekend playing your NLP game? (Dr Ivan Habernal)

    Serious games for NLP are believed to be a win-win situation: people enjoy playing a game and NLP researchers get their annotated data for free. But how come that only few of them succeed and hit their target audience even though they are elaborated and well-designed? In this talk, I will first review the landscape of current games for NLP and will present our attempt for a game from the perspective of the shoemaker's children who go barefoot. However, I will try to show that "pivoting" the product, starting asking the right questions, and leveraging human natural abilities might be a game changer! What if the answers are already out there but not yet in our Games4NLP community?

    Dr Ivan Habernal is a postdoctoral researcher at UKP, Technische Universität Darmstadt. His main research interests are computational argumentation and argument mining, as well as Web corpora, serious games, and sentiment analysis. He has published numerous papers at major NLP venues (ACL, EMNLP, NAACL, LREC, Computational Lingustics) and recently co-chaired the 4th Workshop on Argument Mining co-located with EMNLP'17 in Copenhagen. Currently, he is the main organizer of the SemEval 2018 Task: The Argument Reasoning Comprehension Task, co-located with NAACL'18.

    Enquiries

    For all enquiries, please contact games4nlp@dali-ambiguity.eu

    People

    Co-chairs:

    • Jon Chamberlain (University of Essex, UK)
    • Udo Kruschwitz (University of Essex, UK)
    • Karën Fort (Université Paris-Sorbonne, France)
    • Chris Cieri (Linguistic Data Consortium, University of Pennsylvania, US)

    Program committee:

    • Richard Bartle (University of Essex, UK)
    • Johan Bos (University of Groningen, Netherlands)
    • Eric de la Clergerie (INRIA, France)
    • James Fiumara (Linguistic Data Consortium, University of Pennsylvania, US)
    • Bruno Guillaume (Inria Nancy Grand Est, France)
    • Ivan Habernal (Technische Universität Darmstadt, Germany)
    • Frank Hopfgartner (University of Glasgow, UK)
    • Michael Meder (TU Berlin, Germany)
    • Mathieu Lafourcade (LIRMM, France)
    • Verena Lyding (EURAC, Italy)
    • Lionel Nicolas (EURAC, Italy)
    • Massimo Poesio (Queen Mary University, UK)
    • Pontus Stenetorp (University College London, UK)

    Previous Events

    Games4NLP'17
    Games4NLP held an independent symposium co-located with the 15th European Chapter of the Association for Computational Linguistics (EACL) Conference in April 2017 in Valencia, Spain.

  • Games4NLP: Using Games and Gamification for Natural Language Processing

    Symposium description

    Games4NLP is an independent symposium co-located with the 15th European Chapter of the Association for Computational Linguistics (EACL) Conference that will take place on April 3-7, 2017 in Valencia, Spain.

    The Games4NLP symposium aims to promote and explore the possibilities for research and practical applications of using games and gamification for Natural Language Processing (NLP). The main objective is to provide a forum for researchers and practitioners to discuss and share ideas regarding how the NLP research community can benefit from using game/gamification strategies. For example, games can be used to collect large numbers of annotations of human language provided that there are sufficient numbers of players who are motivated to play, and these annotations, when aggregated, can be used as labels that replace or compliment the effort of expert annotators. The symposium welcomes the participation of both academics and industry practitioners interested in the use of games and gamification for NLP.

    This symposium is aimed at researchers interested in using games and gamification to collect language data and to connect a community who are already developing and maintaining such games.

    Attendance to the workshop if free; however, all attendees must register using the Eventbrite page. Registration to EACL is not required.

    The potential topics of interest include, but are not limited to:

    • Games for collecting data useful for NLP (games in planning, under development, or deployed);
    • Presentation and/or analysis of game data from established games;
    • Gamification of NLP tasks (Techniques, best practice and evaluation of gamification strategies);
    • Player motivation (Strategies for recruitment and retention of players and player profiling);
    • Game design (Conceptual design elements and reports of success and/or failure of designs);
    • Processing NLP game data (Aggregation methods and strategies for minimising noise and cheating);
    • Use of NLP game data (How game-generated data has been used for NLP applications);
    • Evaluation of games for NLP (Metrics for evaluating game performance, evaluating player performance, motivation and bias, and evaluating task difficulty);
    • Directive strategies that attempt to use game metrics in real-time.

    The symposium will feature presentations of works-in-progress and best practice from researchers using games and gamification for Natural Language Processing tasks. Workshop presentations will be grouped into themes such as player motivation, game design, using data generated by games and measuring game performance. Key note speakers to be announced soon.

    Location

    Games4NLP will be held in the Las Arenas/El Perellonet rooms (2nd floor) of the Sercotel Sorolla Palace Hotel in Valencia, Spain. Map.

    Program

    1030-1100 Coffee

    1100-1110 Welcome - Massimo Poesio

    1110-1230 Games under development (Session chair: Jon Chamberlain)
    uComp Language Quiz - A Game with a Purpose for Multilingual Language Resource Acquisition (Arno Scharl and Michael Föls)
    Testing game mechanics in games with a purpose for NLP applications (Chris Madge, Udo Kruschwitz, Jon Chamberlain, Richard Bartle and Massimo Poesio)
    Curating an Open Information Extraction Knowledge Base Using Games with a Purpose (Kevin Forand and Philippe Langlais)

    1230-1250 Panel - Challenges of Games for NLP

    1250-1400 Lunch and Demos

    1400-1520 Player motivation (Session chair: Pontus Stenetorp)
    Who wants to play Zombie? A survey of the players on ZOMBILINGO (Karën Fort, Bruno Guillaume and Nicolas Lefebvre)
    Why do we Need Games? Analysis of the Participation on a Crowdsourcing Annotation Platform (Alice Millour and Karën Fort)
    An Exploratory Study of Data Quality and Participation in a Games-for-Science Game Community (Jason Radford and David Lazer)

    1520-1550 Coffee break and Demos

    1550-1710 Game evaluation (Session chair: Massimo Poesio)
    Making NLP games fun to play using Free to Play mechanics: RoboCorp case study (Dagmara Dziedzic and Wojciech Włodarczyk)
    Metrics of games-with-a-purpose for NLP applications (Jon Chamberlain, Richard Bartle, Udo Kruschwitz, Chris Madge and Massimo Poesio)
    Towards a Word Sheriff 2.0: Lessons learnt and the road ahead (Galen Han, Mark Menezes, Leah Halaseh, Jędrzej Stuczyński, Daniele Menara, Sebastian Riedel and Pontus Stenetorp)

    1710-1730 Panel - Future directions for Games for NLP

    1730-1740 Closing comments - Jon Chamberlain

    People

    Co-chairs:

    • Jon Chamberlain (University of Essex, UK)
    • Chris Cieri (Linguistic Data Consortium, University of Pennsylvania, US)
    • Karën Fort (Université Paris-Sorbonne, France)

    Program committee:

    • Richard Bartle (University of Essex, UK)
    • Johan Bos (University of Groningen, Netherlands)
    • Yun-Gyung Cheong (Sungkyunkwan University, South Korea)
    • Eric de la Clergerie (INRIA, France)
    • James Fiumara (Linguistic Data Consortium, University of Pennsylvania, US)
    • Bruno Guillaume (Inria Nancy Grand Est, France)
    • Iryna Gurevych (Technische Universität Darmstadt, Germany)
    • Ivan Habernal (Technische Universität Darmstadt, Germany)
    • Frank Hopfgartner (University of Glasgow, UK)
    • Udo Kruschwitz (University of Essex, UK)
    • Michael Meder (TU Berlin, Germany)
    • Mathieu Lafourcade (LIRMM, France)
    • Verena Lyding (EURAC, Italy)
    • Lionel Nicolas (EURAC, Italy)
    • Massimo Poesio (University of Essex, UK)
    • Federico Sangati (Independent researcher, Italy)
    • Pontus Stenetorp (University College London, UK)

    Submission

    The symposium is inviting extended abstract submissions (2 pages maximum, including references) of proposals for talks, demos and posters.

    Submissions must be in PDF, and must conform to the official style guidelines for EACL 2017. Submissions do not need to be anonymised. We ask you to use the EACL-2017 LaTeX template or adapt the Overleaf template for those less familiar with LaTex.

    Accepted submissions will be made available electronically on this website.

    Please email your submission to games4nlp@dali-ambiguity.eu by Monday 27 February 2017.

    • **EXTENDED** Deadline for submissions: Monday 27 February 2017, 11:59pm Hawaii Standard Time
    • **DELAYED** Notification of acceptance: Weds 8 March 2017
    • Final program: 27 March 2017
    • Symposium date: 4 April 2017