fix(insights): show cache tokens in /insights token breakdown#18632
Open
liuhao1024 wants to merge 1 commit into
Open
fix(insights): show cache tokens in /insights token breakdown#18632liuhao1024 wants to merge 1 commit into
liuhao1024 wants to merge 1 commit into
Conversation
The /insights report total_tokens includes cache_read_tokens + cache_write_tokens, but the (in: / out: ) breakdown only showed input and output tokens. For Anthropic users with prompt caching, the cache tokens dominate the total, making the in/out numbers appear wrong or swapped. - Terminal format: add a 'Cache tokens' line when cache > 0 - Gateway format: append '/ cache: N' to the token breakdown - Both formats hide cache when all values are 0 (no noise) - Add 5 regression tests for cache display behavior Closes NousResearch#18615
alt-glitch
added
type/bug
Cyrene963
pushed a commit
to Cyrene963/hermes-agent
that referenced
this pull request
May 3, 2026
Community PRs applied: - NousResearch#18596: Enable secret redaction by default (SECURITY) - NousResearch#18650: Sanitize malformed tool messages + auto-recover on API 400 - NousResearch#18607: Emergency compression before max_iterations exhaustion - NousResearch#18603: Compression fallback to main model on 413 rate limit - NousResearch#18638: Pass threshold_percent on model switch - NousResearch#18663: Strip extra_content from tool_calls for strict APIs - NousResearch#18618: Forward explicit_api_key to OpenRouter - NousResearch#18632: Show cache tokens in /insights breakdown - NousResearch#18614: Add idempotency guard for patch duplicate loops - NousResearch#18600: Raise ValueError when HERMES_HOME unset in profile mode - NousResearch#18616: Allow ZWJ emoji in context files - NousResearch#18582: Reload .env on /restart - NousResearch#18547: Stabilize system prompt prefix for KV cache reuse - NousResearch#18692: Strip FTS5 operators from session search truncation terms Fix: Add order_by_last_active=True to list_sessions_rich call (pre-existing commit 142b4bf code sync)
teknium1
reviewed
Jul 12, 2026
teknium1
left a comment
teknium1
left a comment
Contributor
There was a problem hiding this comment.
Thanks for identifying that the visible input/output figures do not sum to total_tokens.
Problems
agent/insights.py:765reintroduces cache-token display that merged PR #11477 deliberately removed as unreliable (commite33cb65a9). Current main preserves that policy intests/agent/test_insights.py:465-468, andformat_gateway()still omits cache atagent/insights.py:871.
Suggested changes
- Please reconcile the cache-total proposal with #11477's reliability rationale before restoring the metric. If maintainers want a combined cache total despite that prior decision, make it an explicit presentation-policy change and update the current display-contract tests accordingly.
Automated hermes-sweeper review.
| lines.append(f" Sessions: {o['total_sessions']:<12} Messages: {o['total_messages']:,}") | ||
| lines.append(f" Tool calls: {o['total_tool_calls']:<12,} User messages: {o['user_messages']:,}") | ||
| lines.append(f" Input tokens: {o['total_input_tokens']:<12,} Output tokens: {o['total_output_tokens']:,}") | ||
| cache_total = (o.get('total_cache_read_tokens') or 0) + (o.get('total_cache_write_tokens') or 0) |
Contributor
There was a problem hiding this comment.
This restores cache-token output that current main deliberately removed in #11477 (e33cb65a9) because cache metrics were unreliable. Current tests retain the no-cache display contract; please reconcile this change with that policy before reintroducing the value.
teknium1
added
sweeper:risk-compatibility
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What does this PR do?
The
/insightsreport'stotal_tokensincludescache_read_tokens + cache_write_tokens, but the(in: / out: )breakdown only showed input and output tokens. For Anthropic users with prompt caching enabled, cache tokens can dominate the total (e.g., 5.2M total but only 2.6K input / 88K output), making the in/out numbers appear swapped or wrong.Root Cause
In
agent/insights.py, the_compute_overview()method correctly sums all four token categories intototal_tokens:But both display methods (
format_terminalandformat_gateway) only showed input and output:Related Issue
N/A
Type of Change
Changes Made
How to Test
pytest tests/ -q— all tests should passChecklist
Code
fix(scope):,feat(scope):, etc.)pytest tests/ -qand all tests passDocumentation & Housekeeping
docs/, docstrings) — or N/Acli-config.yaml.exampleif I added/changed config keys — or N/ACONTRIBUTING.mdorAGENTS.mdif I changed architecture and workflows — or N/A