Suppose you learn a word embedding for a vocabulary of 10000 words. Then the embedding vectors should be 10000 dimensional, so as to capture the full range of variation and meaning in those words.

True

False

Explanation :
The dimension of word vectors is usually smaller than the size of the vocabulary. Most common sizes for word vectors ranges between 50 and 400.

What is t-SNE?

A non-linear dimensionality reduction technique

A linear transformation that allows us to solve analogies on word vectors

You have trained word embeddings using a text dataset of m₁ words. You are considering using these word embeddings for a language task, for which you have a separate labeled dataset of m₂ words. Keeping in mind that using word embeddings is a form of transfer learning, under which of these circumstance would you expect the word embeddings to be helpful?

m₁ >> m₂

m₁ << m₂

Suppose you have a 10000 word vocabulary, and are learning 500-dimensional word embeddings.The GloVe model minimizes this objective: \( \min \sum_{i=1}^{10,000} \sum_{j=1}^{10,000} f(X_{ij}) (\theta_i^T e_j + b_i + b_j’ - log X_{ij})^2 \) Which of these statements are correct?

θ_i and e_j should be initialized to 0 at the beginning of training.

θ_i and e_j should be initialized randomly at the beginning of training.

Suppose you have a 10000 word vocabulary, and are learning 500-dimensional word embeddings. The word2vec model uses the following softmax function: \( P(t \mid c) = \frac{e^{\theta_t^T e_c}}{\sum_{t’=1}^{10000} e^{\theta_{t’}^Te_c}} \) Which of these statements are correct?

θ_t and e_c are both 500 dimensional vectors.

θ_t and e_c are both 1000 dimensional vectors.

In the word2vec algorithm, you estimate P(t∣c), where tt is the target word and c is a context word. How are t and c chosen from the training set? Pick the best answer.

c and t are chosen to be nearby words.

c is the one word that comes immediately before t.

When learning word embeddings, we create an artificial task of estimating P(target∣context). It is okay if we do poorly on this artificial prediction task; the more important by-product of this task is that we learn a useful set of word embeddings.

True

False

Let E be an embedding matrix, and let o₁₂₃₄ be a one-hot vector corresponding to word 1234. Then to get the embedding of word 1234, why don’t we call E * o₁₂₃₄ in Python?

It is computationally wasteful.

his doesn’t handle unknown words ().

Explanation :
Yes, the element-wise multiplication will be extremely inefficient.

Which of these equations do you think should hold for a good word embedding?

e_boy - e_girl = e_brother - e_sister

e_girl - e_boy= e_brother - e_sister

Suppose you download a pre-trained word embedding which has been trained on a huge corpus of text. You then use this word embedding to train an RNN for a language task of recognizing if someone is happy from a short snippet of text, using a small training set.

x (input text)	y (happy?)
I'm feeling wonderful today!	1
I'm bummed my cat is ill.	0
Really enjoying this!	1

Then even if the word “ecstatic” does not appear in your small training set, your RNN might reasonably be expected to recognize “I’m ecstatic” as deserving a label y = 1.

True

False

Explanation :
Yes, word vectors empower your model with an incredible ability to generalize. The vector for "ecstatic would contain a positive/happy connotation which will probably make your model classified the sentence as a "1".

UPSCFEVER - POPULAR PAGES