Xiaobin Li, Stan Szpakowicz and Stan Matwin. "A WordNet-based Algorithm for Word Sense Disambiguation" IJCAI-95, pages 1368-1374. 1 Introduction - project: integration of machine learning and natural language processing - learn production rules from results of text analysis - text analysis based on syntactic parsing and semantic interpretation - use only easily obtainable (not hand-coded) knowledge to aid semantic interpretation (e.g., WordNet) - word sense disambiguation - WordNet has many meanings for a word -> ambiguity - previous approaches - domain-specific, hand-crafted knowledge - statistical data on word/word relationships (bigram model) from large text corpora (unreliable) - authors' WSD algorithm uses neither 2 WordNet and Semantic Similarity - WordNet - lexical database with good coverage - word sense network - word sense node in network is a group of synonyms (synset) - each word mapped to a synset - synsets linked by semantic relationships - ISA (hypernymy/hyponymy) above/below (dominant relationship) - PARTOF (meronymy/holonymy) part/whole - synonymy: similar - antonymy: opposite - semantic similarity - inversely proportional to semantic distance between words in the WordNet ISA hierarchy - semantic similarity divided into four levels - level 1: words in same synset as target word (synonyms) - level 2: words in immediate parent of target word's node in the ISA hierarchy (extended synonyms) - level 3: words in immediate child of target word's node (hyponyms) - level 4: words in a sibling node of the target word's node, i.e., both nodes have same immeidate parent node - WSD algorithm applied only to objective nouns (objects of verbs) - need to define a context in which to disambiguate noun sense - possible contexts include a whole text, 100-word window, sentence, words nearby, etc. - authors' approach uses the noun's dominating verb as the context - co-occuring words in a sentence have same sense - authors' focus on semantic similarity of noun objects and their verb contexts 3 Heuristic Rules and Confidence of Results - given: - noun object NOBJ and its verb context VC - n candidate word senses s(k) of NOBJ in WordNet, 1 <= k <= n - metric for semantic similarity SS - find: best sense of NOBJ within verb context VC - heuristic rule 1 (see fig 1) - search text for verb-object pair (VC,NOBJ'), where NOBJ' has a semantic similarity with NOBJ - select this connecting SS sense for NOBJ - heuristic rule 2 (see fig 2) - search text for verb-object pair (VC',NOBJ), where VC' has a semantic similarity to VC - if word sense of NOBJ in VC' has been determined, then select same sense for NOBJ in VC - heuristic rule 3 (see fig 3) - search text for (VC',NOBJ'), where SS(VC',VC) and SS(NOBJ',NOBJ) - select connecting SS sense between NOBJ' and NOBJ and sense of NOBJ - heuristic rule 4 (see fig 4) - search text for a "such as" following (VC,NOBJ) - object NOBJ' of "such as" has a synonymy or hyponymy relationship with NOBJ - select this word sense for NOBJ - heuristic rule 5 (see fig 5) - search text for a "VC and VC'" or "VC or VC'" with noun object NOBJ - if sense of NOBJ in VC' already determined, then use same sense for NOBJ in VC - confidence of results - different heuristic rules have different results (senses) - assign confidence CF to each result based on semantic similarity metric - CF = 1.0: - applying HR1: NOBJ has only one word sense in WordNet - CF = 0.9: - applying HR1: semantic relationship of level 1, 2 or 3 (synonym, extended synonym or hyponym) between NOBJ and NOBJ' - applying HR4: "such as" - CF = 0.8: - applying HR1: level 4 semantic relationship (sibling) between NOBJ and NOBJ' - CF = 0.7: - applying HR2: previous results having CF of 0.9 or 0.8 - CF = 0.6: - applying HR3: semantic relationship of level 1, 2 or 3 (synonym, extended synonym or hyponym) between NOBJ and NOBJ' - CF = 0.5: - applying HR3: level 4 semantic relationship (sibling) between NOBJ and NOBJ' - WSD algorithm - HR1 always applied first, then HR2 and finally HR3 - HR4 and HR5 applied later due to limited application - higher semantic similarity always considered earlier - results with higher CF values usually acquired early 4 WSD Algorithm - disambiguate senses sense(Wn,k) of noun object Wn in verb context Wv - eight steps, terminates whenever any step succeeds - step 1 - search for Wn in WordNet - if Wn has only one sense, then meaning of Wn is sense(Wn,1) with confidence CF = 1.0 - example - sense("income",1) = "financial gain" (only sense) - step 2 - find verb-object pair (Wv,Wn') in parsed text - Wn' is synonym or hyponym (level 1, 2 or 3) of Wn with sense (Wn,k) - meaning of Wn is sense(Wn,k) with confidence CF = 0.9 - example - Wn = "contribution" having 5 senses in WordNet - Wv = "calculate" - verb-object pair "calculate amount" found in text - "amount" synonymous with sense("contribution",3) - meaning of "contribution" is sense("contribution",3) = "an amount of money contributed" - step 3 - same as step 2 with coordinate (sibling) relationship between Wn' and sense(Wn,k) - confidence CF = 0.8 - example - Wn = "expense" with two senses in WordNet - Wv = "substract" - verb-object pair "subtract grant" found in text - "grant" coordinate with sense("expense",1) - meaning of "expense" is sense("expense",1) = "a possession whose ownership has changed" - step 4 - find in parsed test a verb-object pair (Wv',Wn) - Wv' has level 1, 2, 3 or 4 relationship with Wv - word sense of Wn in Wv' already acquired as sense(Wn,k) - assign same word sense to Wn in Wv with confidence 0.7 - example - Wn = "contribution", Wv = "prorate" - "contribution" has 6 senses in WordNet - verb-object pair "calculate contribution" found in text - "calculate" and "prorate" have synonymous relationship - sense("contribution",3) already selected in Wv' (see step 2) - meaning of "contribution" in "prorate" also sense("contribution",3) - step 5 - find in parsed text verb-object paair (Wv',Wn') - Wv' has level 1, 2, 3 or 4 relationship with Wv - Wn' has level 1, 2 or 3 relationship with sense(Wn,k) - meaning of Wn in Wv assigned sense(Wn,k) with confidence 0.6 - example - Wn = "deduction", Wv = "calculate" - "deduction" has 7 senses in WordNet - verb-object pair "substract allowance" found in text - "substract" synonymous with "calculate" - "allowance" synonymous with sense("deduction",3) - meaning of "deduction" in "calculate" is sense("deduction",3) - step 6 - same as step 5, but level 4 (sibling) relationship between Wn' and sense(Wn,k) - confidence CF = 0.5 - example - Wn = "investment", Wv = "list" - "investment" has 3 senses in WordNet - verb-object pair "enter credit" found in text - "enter" has coordinate relationship with "list" - "credit" coordinate with sense("investment",1) - meaning of "investment" in "list" is sense("investment",1) = "any valuable or useful possession" - step 7 - find in parsed text a "such as" following verb-object pair (Wv,Wn) - object Wn' of "such as" synonymous or hyponymous (level 1, 2 or 3) with sense(Wn,k) - meaning of Wn in Wv is sense(Wn,k) with confidence CF = 0.9 - example - Wn = "property" (5 senses in Wordnet), Wv = "sell" - "such as real estate" found in text after "sell property" - "real estate" hyponymous with sense("property",1) = "any tangible possession that is owned by someone" - meaning of "property" in "sell" is sense("property",1) - step 8 - find in parsed text a "Wv and Wv'" or "Wv or Wv'" whose noun object is Wn - Wn has sense(Wn,k) in Wv' - select sense(Wn,k) for Wn in Wv - confidence same as that for (Wv',Wn) - example - Wn = "property" (5 senses in WordNet), Wv = "dispose of" - "dispose of or sell property" in text - meaning of "property" in "sell" already acquired as sense("property",1) with confidence 0.9 (see step 7) - meaing of "property" in "dispose of" is also sense("property",1) with confidence 0.9 5 Evaluation - WSD implement in Prolog - tested on text from Canadian Income Tax Guide - 1797 sentences - 593 different verb-object pairs - 173 different noun objects in pairs (3 occurrences each on average) - 70 noun objects actually have at least 3 occurrences - 60 noun objects have more than one sense in WordNet - used WSD to determine meaning of 60 noun objects in 397 different verb contexts - WSD has five possible results - one correct solution - example - "report loss" yielded sense("loss",2) of 6 possible senses - sense("loss",2) = "failure to retain possession" - correct multiple solutions - WSD gives multiple meanings (all reasonable for the context) - example - "claim charge" yielded 2 of 13 "charge" senses - sense("charge",1) = "financial liability" - sense("charge",2) = "price charged for some article or service" - partially correct solutions - multiple senses, one of which is reasonable - example - "attach receipt" yielded 2 of 3 "receipt" senses - sense("receipt",2) = "acknowledgement (commercial document)" - sense("receipt",3) = "act of receiving" - only sense("receipt",2) is reasonable with "attach" - wrong solution - all meanings selected by WSD are unreasonable - no solution - insufficient information in text - results on 397 verb-object pairs (see table in paper, copied below) - 225 (57%) one correct solution - 59 (15%) multiple solutions - 20 (5%) partially correct solutions - 17 (4%) wrong solution - 76 (19%) no solution - discussion - 72% correct, 4% wrong :) - 15% multiple solutions indicate more context needed to find one true meaning, esp. for verbs like "include" or "have" - 19% no solution indicates need for background knowledge verb-object pairs - sensitivity in CF values (see table in paper) - change values as long as relative differences for HR1-HR5 retained - results show that a solution with low confidence is more likely to be a wrong solution - thus reported CF values are appropriate 6 Conclusions - WSD for verb-object pairs with minimal knowledge - method easily transformed for other parts of speech - approach focusses on semantic similarity and heuristic rules - experiments demonstrate - viability of WSD algorithm - rationality of confidence value assignment - potential of WordNet-based method - limitations of considering on verb contexts - future work - add more complex contextual information to the WSD process