15% of exam score
Context Management & Reliability
Smallest weight. Failures cascade everywhere.
Context window management, RAG and retrieval, multi-pass and independent review, prompt caching, cost/latency tradeoffs, escalation logic.
Key insight
Progressive summarization silently CORRUPTS transactional data — amounts, dates, and order numbers become vague paraphrases. A persistent 'case facts' block that is never summarized and always included verbatim in every prompt is the correct fix.
The exam rewards
- Extract transactional specifics (amounts, dates, IDs) into a persistent case-facts block and include it verbatim in every prompt — never let it be summarized
- Three valid escalation triggers: customer explicitly requests a human, a policy gap exists, unable to make meaningful progress. Sentiment and self-reported confidence are NOT reliable triggers
- The 'lost in the middle' effect is real — place key summaries and findings at the beginning or end of long inputs, not buried in the middle
- When propagating errors in multi-agent systems, include: failure type, what was attempted, partial results obtained, and alternative approaches
- Never silently suppress failures or terminate the full pipeline on one error — annotate, continue, report
Common anti-patterns
- Using progressive summarization on conversations containing transactional data
- Escalating based on customer frustration or detected sentiment
- Escalating based on the model's self-reported confidence score
- Terminating an entire pipeline workflow because a single tool failed