What is generative and discriminative model? How are they used in Natural Language Processing?

10

This question asks about generative vs. discriminative algorithm, but can someone give an example of the difference between these forms when applied to Natural Language Processing? How are generative and discriminative models used in NLP?

alvas

Posted 2014-05-18T06:17:37.587

Reputation: 2 242

@knb The link to the video is broken – viebel – 2018-04-29T15:58:27.357

Answers

8

Let's say you are predicting the topic of a document given its words.

A generative model describes how likely each topic is, and how likely words are given the topic. This is how it says documents are actually "generated" by the world -- a topic arises according to some distribution, words arise because of the topic, you have a document. Classifying documents of words W into topic T is a matter of maximizing the joint likelihood: P(T,W) = P(W|T)P(T)

A discriminative model operates by only describing how likely a topic is given the words. It says nothing about how likely the words or topic are by themselves. The task is to model P(T|W) directly and find the T that maximizes this. These approaches do not care about P(T) or P(W) directly.

Sean Owen

Posted 2014-05-18T06:17:37.587

Reputation: 5 987