BERT for Coreference Resolution: Baselines and Analysis
SpanBERT: Improving Pre-training by Representing and Predicting Spans

by Juntao Yu
BERT - Coreference

The first paper BERT for coreference resolution demonstrates how original BERT models can be used for coreference resolution task. The proposed system is an extension of the previous SOTA Lee et al system that replaces the LSTMs with BERT and fine-tune the BERT model for the coreference task. The paper introduces the first BERT fine-tune approach for coreference resolution.
The second paper introduces the SpanBERT, which transforms the original BERT to more robust for the span prediction tasks. They replace the random subword masking scheme used in the original BERT model with a random span masking scheme to make the BERT better for the span prediction tasks. They also replace the next sentence prediction (NSP) with their span boundary objective (SBO) to encourage the system to encode information of the spans in their boundary tokens. The SBO has been found especially helpful for coreference task that directly uses the boundary token to represent the spans. Overall they show that the resulted model (SpanBERT) works better than the original BERT on various tasks evaluated.

Joshi, Mandar, et al. "BERT for Coreference Resolution: Baselines and Analysis." Proceedings of EMNLP-IJCNLP. 2019.
Joshi, Mandar, et al. "Spanbert: Improving pre-training by representing and predicting spans." arXiv preprint arXiv:1907.10529 (2019).


Quizz: Targeted Crowdsourcing with a Billion (Potential) Users (2014)

by Doruk Kıcıkoğlu
Gamification - Crowdsourcing - Expert Assessment - Ad Targeting

Quizz is described as a gamified crowdsourcing system that simultaneously assesses the knowledge of users and acquires new knowledge from them. Quizz operates by asking users to complete short quizzes on specific topics; as a user answers the quiz questions, Quizz estimates the user's competence. To acquire new knowledge, Quizz also incorporates questions for which we do not have a known answer; the answers given by competent users provide useful signals for selecting the correct answers for these questions. Quizz actively tries to identify knowledgeable users on the Internet by running advertising campaigns, effectively leveraging the targeting capabilities of existing, publicly available, ad placement services. Quizz quantifies the contributions of the users using information theory and sends feedback to the advertisingsystem about each user. The feedback allows the ad targeting mechanism to further optimize ad placement.

Ipeirotis, P., & Gabrilovich, E. (2014). Quizz. Proceedings Of The 23Rd International Conference On World Wide Web - WWW '14. doi: 10.1145/2566486.2567988


Jerry R. Hobbs: Granularity (1985)

by Janosch Haber
Granularity - Abstraction - Conceptualisation

In 1985 Jerry Hobbs introduced the granularity framework as an attempt to formalise the abstraction of complex global theories into simplified and computationally tractable local theories, something that for example often is seen during planning. In the article discussed during this session, Hobbs presents four central concepts to his granularity framework; abstraction, simplification, idealization and articulation, and deals with the notion of indistinguishability under relevant predicates, fuzzy indistinguishability boundaries, the importance of an explicit representation of granularity levels and the link between seamless granularity shifting and human intelligence.

Jerry R. Hobbs. 1985. Granularity. In Proceedings of the 9th international joint conference on Artificial intelligence - Volume 1 (IJCAI'85), Aravind Joshi (Ed.), Vol. 1. Morgan Kaufmann Publishers Inc., San Francisco, CA, USA, 432-435.


Boaz Keysar et al.: Taking Perspective in Conversation: The Role of Mutual Knowledge in Comprehension (2000)

by Janosch Haber
Visually-grounded Deixis - Collaborative Dialogue - Egocentric Perspective

Many dialogue theories assume a collaborative interaction between participants, often expecting speakers to choose expressions that maximise the probability of being understood correctly under the current conversation model. In the presented paper, Keysar and colleagues investigate the notion of shared perspective in dialogue and ask whether listeners interpret expressions based on a model containing only shared information or whether they rather act based on a personal (egocentric) model. In their experiments, participants move items following the instructions of a confederate instructor. Some of the items are clearly hidden from the instructor’s view and thus only exist in the participants personal model. Under the shared perspective hypothesis, this setup should not lead the participant to consider these hidden items as potential referents as they know that they cannot be the intended referents of the instructor’s expressions. Keysar and colleagues however observe that participants often still look at occluded items and start reaching for them in more than 20% of the trials.

Keysar, B., Barr, D. J., Balin, J. A., & Brauner, J. S. (2000). Taking Perspective in Conversation: The Role of Mutual Knowledge in Comprehension. Psychological Science, 11(1), 32–38.


Barbara Hall Partee: Opacity, Coreference and Pronouns (1972)

by Janosch Haber
Pronoun-Antecedent Relations - Coreference - Pronouns of Laziness

In her seminal 1972 paper, Barbara Hall Partee discusses different approaches of explaining and formalising ambiguous pronoun-antecedent relations, considering first a number of previously proposed attempts such as stipulating antecedent presupposition, quantifier scope modification or explicit specificity marking, finding all of them to be insufficient for explaining the full set of observed phenomena. She then moves to elaborate on pronouns of laziness, definite noun phrases and coreference, and indefinite noun phrases and actualisation to point at a range of future research avenues.

Partee B.H. (1972) Opacity, Coreference, and Pronouns. In: Davidson D., Harman G. (eds) Semantics of Natural Language. Synthese Library (Monographs on Epistemology, Logic, Methodology, Philosophy of Science, Sociology of Science and of Knowledge, and on the Mathematical Methods of Social and Behavioral Sciences), vol 40. Springer, Dordrecht