23

9

Given a sentence:
"When I open the **??** door it starts heating automatically"

I would like to get the list of possible words in ?? with a probability.

The basic concept used in word2vec model is to "predict" a word given surrounding context.

Once the model is build, what is the right context vectors operation to perform my prediction task on new sentences?

Is it simply a linear sum?

```
model.most_similar(positive=['When','I','open','the','door','it','starts' ,
'heating','automatically'])
```

I implemented the above algorithm and came across a question:

– None – 2016-08-09T08:13:01.173Why is softmax used?Let me show you an example of two normalization functions: def softmax(w, t = 1.0): # Source: https://gist.github.com/stober/1946926 e = np.exp(w / t) return e / np.sum(e) def normalization(w): return w / np.sum(w) a = np.array([.0002, .0001, .01, .03]) print normalization(a) print softmax(a, t=1) Let's compare the outputs: [ 0.00496278 0.00248139 0.24813896 0.74441687] [ 0.24752496 0.24750021 0.24996263 0.25501221] As we can see, softmax gives .03 roughly the same probability as compared to .0001 (which is