0

For RNN's to work efficiently we vectorize the problem which results in an input matrix of shape

    (m, max_seq_len)


where m is the number of examples, e.g. sentences, and max_seq_len is the maximum length that a sentence can have. Some examples have a smaller lengths than this max_seq_len. A solution is to pad these sentences.

One method to pad the sentences is called "zero-padding". This means that each sequence is padded by zeros. For example, given a vocabulary where each word is related to some index number, we can represent a sentence with length 4,

    "I am very confused"


by

    [23, 455, 234, 90]


Padding it to achieve a max_seq_len=7, we obtain a sentence represented by:

    [23, 455, 234, 90, 0, 0, 0]


The index 0 is not part of the vocabulary.

    "I am very confused <<pad>>> <<pad>> <<pad>>"

    [23, 455, 234, 90, 1000, 1000, 1000]