4

1

In Neural networks [3.8] : Conditional random fields - Markov network by Hugo Larochelle it seems to me that a Markov Random Field is a special case of a CRF.

However, in the Wikipedia article Markov random field it says:

One notable variant of a Markov random field is a conditional random field, in which each random variable may also be conditioned upon a set of global observations o.

This would mean that CRFs are a special case of MRFs.

## Definitions

### Markov Random Field

Again, according to Wikipedia

Given an undirected graph $G=(V,E)$, a set of random variables $X = (X_v)_{v\in V}$ indexed by $V$ form a Markov random field with respect to $G$ if they satisfy the local Markov properties:

Pairwise Markov property: Any two non-adjacent variables are conditionally independent given all other variables: $X_u \perp\!\!\!\perp X_v \mid X_{V \setminus \{u,v\}} \quad \text{if } \{u,v\} \notin E$

Local Markov property: A variable is conditionally independent of all other variables given its neighbors: $X_v \perp\!\!\!\perp X_{V\setminus \operatorname{cl}(v)} \mid X_{\operatorname{ne}(v)}$ where ${\textstyle \operatorname{ne}(v)}$ is the set of neighbors of $v$, and $\operatorname{cl}(v) = v \cup \operatorname{ne}(v)$ is the closed neighbourhood of $v$.

Global Markov property: Any two subsets of variables are conditionally independent given a separating subset: $X_A \perp\!\!\!\perp X_B \mid X_S$ where every path from a node in $A$ to a node in $B$ passes through $S$.

Please note if you know a citable source which gives a good definition.

### Conditional Random Fields

According to Wikipedia:

Lafferty, McCallum and Pereira[1] define a CRF on observations $\boldsymbol{X}$ and random variables $\boldsymbol{Y}$ as follows:

Let $G = (V , E)$ be a graph such that

$\boldsymbol{Y} = (\boldsymbol{Y}_v)_{v\in V}$, so that $\boldsymbol{Y}$ is indexed by the vertices of $G$. Then $(\boldsymbol{X}, \boldsymbol{Y})$ is a conditional random field when the random variables $\boldsymbol{Y}_v$, conditioned on $\boldsymbol{X}$, obey the Markov property with respect to the graph: $$p(\boldsymbol{Y}_v |\boldsymbol{X}, \boldsymbol{Y}_w, w \neq v) = p(\boldsymbol{Y}_v |\boldsymbol{X}, \boldsymbol{Y}_w, w \sim v)$$ where $\mathit{w} \sim v$ means that $w$ and $v$ are neighbors in G.

What this means is that a CRF is an undirected graphical model whose nodes can be divided into exactly two disjoint sets $\boldsymbol{X}$ and $\boldsymbol{Y}$, the observed and output variables, respectively; the conditional distribution $p(\boldsymbol{Y}|\boldsymbol{X})$ is then modeled.

## Question

What is the relationship between Markov Random Fields and Conditional Random Fields?

See also: What's the difference between a Markov Random Field and a Conditional Random Field?

– Martin Thoma – 2016-01-08T21:52:14.980