History:
Prehistory (4th centry BCE): grammarian of Sanskrit (sacred language of India), first formal model of language
Andrey Markov: model Russian poetry statistically, invented Markov model
Claude Shannon: N-Gram model, noisy channel model
Chomsky (rules, grammar) reject Harris (statistic modeling)
Roger Schank: reject Chomsky to embrace context
Spark Jones: term frequency, inverse document frequency
Peter Brown: first NLP model
Martha Palmer: large annotated datasets
Now: Nvidia, Embedding, LSTM, transformer, BERT, Huggingface, pretrain
Table of Content