深度學習Course5第二周Natural Language Processing & Word Embeddings習題整理

Natural Language Processing & Word Embeddings

1. True/False: Suppose you learn a word embedding for a vocabulary of 20000 words. Then the embedding vectors could be 1000 dimensional, so as to capture the full range of variation and meaning in those words.

解析：The dimension of word vectors is usually smaller than the size of the vocabulary. Most common sizes for word vectors range between 50 and 1000.

1. True/False: t-SNE is a linear transformation that allows us to solve analogies on word vectors.

解析：tr-SNE is a non-linear dimensionality reduction technique.

1. Suppose you download a pre-trained word embedding which has been trained on a huge corpus of text. You then use this word embedding to train an RNN for a language task of recognizing if someone is happy from a short snippet of text, using a small training set.



2. Then even if the word “ecstatic” does not appear in your small training set, your RNN might reasonably be expected to recognize “I’m ecstatic” as deserving a label.

解析： word vectors empower your model with an incredible ability to generalize. The vector for “ecstatic” would contain a positive/happy connotation which will probably make your model classify the sentence as a “1”.(泛化能力增強)

1. Which of these equations do you think should hold for a good word embedding? (Check all that apply)

1. Let be an embedding matrix, and let be a one-hot vector corresponding to word 4567. Then to get the embedding of word 4567, why don’t we call

• The correct formula is

解析：the element-wise multiplication will be extremely inefficient.

1. When learning word embeddings, words are automatically generated along with the surrounding words.

解析： we pick a given word and try to predict its surrounding words or vice versa.

1. In the word2vec algorithm, you estimate , where is the target word and is a context word. How are and

• is the sequence of all the words in the sentence before
• and
• is a sequence of several words immediately before
• is the one word that comes immediately before

1. Suppose you have a 10000 word vocabulary, and are learning 500-dimensional word embeddings. The word2vec model uses the following softmax function:



2. Which of these statements are correct? Check all that apply.

• and
• and
• After training, we should expectto be very close towhenand
• and

1. Suppose you have a 10000 word vocabulary, and are learning 500-dimensional word embeddings. The GloVe model minimizes this objective:
2. Which of these statements are correct? Check all that apply.

• and
• and
• Theoretically, the weighting functionmust satisfy

1. You have trained word embeddings using a text dataset of words. You are considering using these word embeddings for a language task, for which you have a separate labeled dataset of

• Whenis equal to
• Whenis smaller than
• Whenis larger than