Inference
Inference is the model running: reading your input and generating a response.
Training is where a model learns, once, expensively. Inference is every time it is used after that.
The model does not change during inference. It cannot learn from your conversation, which is exactly why memory has to be built on top.
Was this page helpful?