Games
We aim for accurate and quality data with the power of crowdsourcing via Games With a Purpose.
Lingotowns
Lingotowns, a new GWAP platform targeting language learners.
Lingotowns provides a unified experience integrating games for multiple aspects of lexical and grammatical experience in a single virtual world, whilst simultaneously collecting judgements. Both Lingotowns and its constituent games are designed to provide more engagement to the players/ learners than normal GWAPs. The platform also incorporates knowledge tracing methods ensuring that the players' progress in terms of understanding of grammatical concepts is tracked both at the individual game level and overall.
Word Clicker
WordClicker is a web based one player game in which players annotate part-of-speech tags, although the mechanics are transferrable to similar labelling tasks.
The gameplay involves the player playing the part of a baker making cakes. To produce cakes the player must find a steady stream of ingredients. The ingredient types map to word classes and the ingredients themselves, words. To assign a label to a word and earn ingredients, the player selects the appropriate ingredient jar for that word class and clicks the token. Having labelled the tokens and earned profits, the player can reinvest in their cake production business by purchasing upgrades from the upgrade shop.
WordClicker makes use of a selection of incremental game mechanics that we believe are uniquely suited to the challenge of embedding annotation as part of an entertaining game-like experience, including both GWAPs and learning games.
Wormingo
Wormingo is our newest game. Currently in beta version, the game attempts to gather quality data by employing a technique called 'motivation'.
The gameplay switches between motivation and annotation modes. Motivation mode removes some tokens from the text and asks these tokens to the players through different puzzles. The objective of the motivation mode is to entertain the player for better engagement, meanwhile solving the puzzles reinforces text comprehension.
After motivation mode, the game switches to annotation mode and asks the player to complete an annotation task. This mode resembles Phrase Detectives, in that the player is presented with the text and their task in hand.
One of the gamification techniques Wormingo employs is the Categories setting. This setting allows the player to choose which categories of text should appear more often, letting them customise their gaming experience to their taste of reading.
Tile Attack
Tile Attack is a two player blind game in which players are awarded points based on player agreement on tokens that they identify as being nounphrases.
The design of the game is inspired by scrabble with a tile like visualisation shown in figure 1. The game includes a point system and leaderboard that is shown to the player between rounds.
Phrase Detectives
Phrase Detectives is an online crowdsourcing game with a validation stage, primarily designed to collect data about English (and subsequently Italian) anaphoric co-reference. Anaphoric coreference is a type of linguistic reference where one expression depends on another referential element. An example would be the relation between the entity 'Jon' and the pronoun 'his' in the text 'Jon rode his bike to school.'
The game uses two styles of text annotation for players to complete a linguistic task. Initially text is presented in Annotation Mode (called Name the Culprit in the game). This is a traditional annotation method in which the player makes an interpretation (annotation decision) about a highlighted markable (section of text). If different players enter different interpretations for a markable then each interpretation is presented to more players in Validation Mode (called Detectives Conference in the game). The players in Validation Mode have to agree or disagree with the interpretation. Players may also make comments about the task and/or skip the task if they do not want to provide an interpretation.
Players could label markables as DN (discourse-new, where the markable refers to a newly introduced entity), DO (discourse-old, where the markable refers to an already mentioned entity in the text, NR (non-referring, where the markable does not refer to anything or PR (where the markable represents a property of a previously mentioned entity).
The first publicly available dataset (Phrase Detectives Corpus 1.0) was used to determine what the collective quality of the players were, as well as the quality of individual decisions. Full details of the game and corpora, including processing pipeline, descriptive statistics and gold standard creation, have been published (see publications section).
