fix(libsy): validate classifier scores, fix packaging and strict rustdoc#104
Open
elyasmnvidian wants to merge 2 commits into
Open
fix(libsy): validate classifier scores, fix packaging and strict rustdoc#104elyasmnvidian wants to merge 2 commits into
elyasmnvidian wants to merge 2 commits into
Conversation
WalkthroughChangesLLM classifier routing
Estimated code review effort: 3 (Moderate) | ~20 minutes Poem
🚥 Pre-merge checks | ✅ 5✅ Passed checks (5 passed)
Comment |
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@crates/libsy/src/algorithms/llm_class.rs`:
- Around line 417-419: Remove the tracker reference “NvBug 6485976” from the NaN
fail-open comment at crates/libsy/src/algorithms/llm_class.rs lines 417-419
while preserving its rationale; likewise remove “NvBug 6485975” from the
streamed-score rationale at lines 439-441 without changing the surrounding
behavior or explanation.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Path: .coderabbit.yaml
Review profile: CHILL
Plan: Enterprise
Run ID: 81f87868-d92e-4236-97ce-2e768d7abaac
📒 Files selected for processing (4)
crates/libsy/Cargo.tomlcrates/libsy/src/algorithms/llm_class.rscrates/libsy/src/core/algorithm.rscrates/libsy/src/lib.rs
elyasmnvidian
force-pushed
the
emehtabuddin/qa-libsy
branch
from
July 21, 2026 21:30
2e417fe to
3d3b31c
Compare
grahamking
approved these changes
Jul 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Three fixes to the libsy LLM-classifier router (
crates/libsy/src/algorithms/llm_class.rs), plus a packaging and a rustdoc fix.Streaming classifier verdicts were ignored. The router read the score with
as_agg(), which returnsNonefor a streaming response, so a classifier target that streamed its score always fell open to the strong model no matter what it returned. It now drains the stream withinto_agg()and routes on the real score. A genuine stream error still propagates instead of being swallowed.Invalid scores were trusted as verdicts. A classifier reply that parsed as a float but was not a probability —
NaN,±inf, or anything outside[0, 1]— was compared against the threshold directly.NaNand negative values compared below the threshold and were routed to the weak model. The router now keeps only scores in[0.0, 1.0]; everything else fails open to the strong model, same as an unparseable reply.Packaging:
cargo package -p switchyard-libsynow succeeds. Theswitchyard-protocolpath dependency carriesversion = "0.1.0", which cargo requires before it will package a crate that has a path dependency.Docs:
RUSTDOCFLAGS="-D warnings" cargo docis clean again. Fixed two intra-doc links that pointed at private items (DriverRequest,Driver::stream) and dropped redundant explicit link targets inlib.rs.Testing
LlmClassifieris a Rust library type incrates/libsy. You can't reach it throughswitchyard serve— the Python bindings expose onlynoopandrandom— and the triggers (an invalid score, or a score that arrives as a stream) can't be forced from a real gateway model. So the proof is the router's own regression tests: they fail onmain's logic and pass on the fix. New tests cover the streaming drain and the invalid-score fail-open.Run them:
cargo clippy -p switchyard-libsy --all-targetsis clean.Proof
I ran the
llm_classtests twice: once with the score handling reverted tomain(as_agg(), no range filter), once with the fix (into_agg()to drain the stream, plus.filter(|s| (0.0..=1.0).contains(s))).On
main's logic, three tests fail:With the fix, all pass:
This push also adds
below_range_score_defaults_to_strong— a parseable-0.1thatmainroutes to the weak model.