How to handle negative words in word2vec?

7

6

I am training a big corpus using word2vec and averaging the word vectors to get sentence vectors. What is the best way to address negative words so that negative and positive sentences are away from each other? For e.g.: "After the fix code worked" and "After the fix code did not work" should ideally give sentence vectors which are far from each other. I heard one approach is to look for negative words like "not" and negate the next word vector. Can someone please clarify if that's a good approach or can suggest a better approach?

Shamy

Posted 2016-12-17T11:36:12.127

Reputation: 207

Don't average them; use a document vector model like paragraph2vec. See the sentiment analysis in the Experiments section for a performance evaluation.

– Emre – 2016-12-17T17:32:01.707

You can check this link. A way of handling negation is suggested (https://www.slideshare.net/shailendrakumars1/negation-handling).

– Behzad Mirzababaei – 2019-04-23T06:02:47.727

Answers

3

When you look at the vectors that word2vec generates - negative words may have unique features but can be treated just like positive words. That is to say, as far as the NN is concerned - these are just similar words. You may have to construct "concept vectors" on top of the word vectors to do what you would like to do.

Your parts of speech tagging should automatically mark negating words as ADV. You can then train on these adverbs in conjunction to your verbs as a positive or negative output. Here's an example using spacy:-

import spacy

nlp = spacy.load('en')        # this can take a while
sample_text = u'Do not go.'
parsed_text = nlp(sample_text)
token_text = [token.orth_ for token in parsed_text]
token_pos = [token.pos_ for token in parsed_text]

At this point token_text will be a list of your words and token_pos will be the POS tagging:-

Do - VERB
not - ADV
go - VERB
. - PUNCT

As you can see, "not" is tagged as ADV here. You can now feed this tagged output (or a better parse tree) into a second network to train for a negative or positive output.

Hope this helps.

Daniel Wee

Posted 2016-12-17T11:36:12.127

Reputation: 61

0

There is a possibility of refining word2vec vectors, which as research shows capture both, semantic relatedness and semantic similarity, in such a way, that they would capture the relations between words such as antonymy or negation. You can take a look at Counter-Fitting method (or methods in it's related work). Their implementation should be available online. This may improve results of your sentiment analysis method.

Smarty77

Posted 2016-12-17T11:36:12.127

Reputation: 101

-1

Fatma.S.Gadelrab

Posted 2016-12-17T11:36:12.127

Reputation: 1