A commonly applied formula for making predictions in context is 'Bayes' Rule'

• What is Bayes rule used for?

• What do each of the terms in Bayes' rule signify?

What puts the 'N' in 'N-gram'.

• What does the 'N' in 'N-grams' indicate?

We can use N-grams build a probabilistic finite state machine.

• In the illustration, say we're in the state marked 'went to'. Assuming a deterministic model, what state will we be in if the next input is 'the'?

• In the illustration, say we're in the state 'to the'. How much does the model remember about the state we were in before that?

What happens when we increase N

• Since the higher the 'N' in our N-gram language model, the higher the accuracy, why wouldn't we want to use, say 100-grams to model a language?

• What is the sparse data problem?