I have seen both terms used while reading papers about BERT and ELMo so I wonder if there is a difference between them.
The duck is swimming
and You shall duck when someone shoots at you
. With traditional word embeddings, the word vector for duck
would be the same in both sentences, whereas it should be a different one in the contextualized case. So in short, a conextualized word embedding represents a word in a context, whereas a sentence encoding represents a whole sentence.