Cross-lingual Transfer of Semantics in Low-resource Settings Open Access
Downloadable ContentDownload PDF
Despite the significant improvements yielded by aggregating supervised semantic analysis in various natural language processing applications, annotated data are available for only a few languages, mainly due to the significant costs of producing semantically annotated resources. Stark deficiency in semantically annotated resources for the majority of languages worldwide has led to a growing interest in transfer methods as a cost-effective complement or even alternative to for data annotation. The ultimate goal of transfer methods is to benefit from the semantic knowledge present for one or more language(s) with semantically annotated resources (high-resource) to model semantics in languages suffering from data deficiency. Annotation projection is one of the widely used approaches for transferring semantics from a high-resource language to a low-resource target language using the alignment links acquired from sentence-aligned corpora or bilingual dictionaries.In this dissertation, we demonstrate the effectiveness of annotation projection for producing semantic analysis in a spectrum of low-resource scenarios ranging from the case that the only missing part of the puzzle is semantic annotations to the extreme low-resource scenario where no adequate explicit lexico-syntactic features are available. In this study, we target cross-lingual semantic transfer on both the lexical and sentential levels. On the lexical level, we propose an unsupervised system based on annotation projection to address the phenomenon of word sense divergence, mainly observed when the underlying semantic distribution of the test set is different from that of the train data. We extrinsically evaluate our lexical transfer model in an SMT framework, as one of the NLP applications heavily impacted by the words with sense divergence. We demonstrate that our proposed model for identifying and disambiguating words with sense divergence improves SMT lexical choice. Our method solely relies on the transferred word sense information and does not utilize any labeled or in-domain training data.On the sentential level, we begin with a cross-lingual semantic role labeling (SRL) model that mainly focuses on improving the quality of projection instances used to train the model by taking advantage of cues automatically acquired from word alignments as well as syntactic analysis of the sentence. We devise a customized cost function to effectively weight some projections over other instances. We show that utilizing this simple cost function yields significant improvements over a standard annotation projection method on English to German. We then move to a more realistic low-resource scenario where accurate linguistic features or sizable parallel corpora might not be readily available. We first demonstrate the power of character representations to confront the need for other morpho-syntactic features. We additionally look into a cross-lingual SRL model that uses the Bible, a smaller but widely available parallel corpus and analyze the effectiveness of conventional transfer techniques when applied on smaller projection corpora.Finally, we explore the role of supervised syntactic information on the performance of a cross-lingual semantic dependency parsing (SDP) model built over projections in a multitask framework. We report the performance of various multitasking models on a subset of projections with different densities to find the optimal level of supervision required by each framework. We show empirically that multitask learning yields significantly better performance in annotation projection models compared to supervised baselines.