Context Cascade Architecture
It started with a user’s frustration: repeating to a chatbot what it had already forgotten. The answer was to stop keeping everything in one place.
- A cascade: context window, temporary memory, persistent memory, external database. Queried in order, and only when the level above is not enough.
- Historical blocks with metadata, searched from the most recent, skipped on their metadata before their content is read, with a threshold to stop early.
- Redis for the temporary buffer, and any vector, SQL or NoSQL store for the archive.
Among the use cases: personalised tutors, and assistants for the elderly. Both come back at the end of this page.