Skip to content

fix: show actual test file paths in failure log#1393

Merged
mashraf-222 merged 1 commit into
omni-javafrom
fix/improve-test-failure-logging
Feb 10, 2026
Merged

fix: show actual test file paths in failure log#1393
mashraf-222 merged 1 commit into
omni-javafrom
fix/improve-test-failure-logging

Conversation

@mashraf-222

Copy link
Copy Markdown

Problem Fixed

When tests fail to run, the error log shows misleading information:

Tests '[None, None, PosixPath(...)]' failed to run

This happens because the log prints original_file_path which is intentionally None for AI-generated tests, making it appear as if test paths weren't set correctly.

Root Cause

In parse_test_output.py:1222, the failure log message was:

f"Tests '{[test_file.original_file_path for test_file in test_files.test_files]}' failed to run"

For AI-generated tests:

  • original_file_path = None (intentional - these are newly generated)
  • instrumented_behavior_file_path = actual path being executed
  • benchmarking_file_path = performance test path

The log should show the paths actually being used, not the intentionally-None original paths.

Solution Implemented

Changed the log to show instrumented_behavior_file_path or original_file_path (whichever is set):

test_paths_display = [
    str(test_file.instrumented_behavior_file_path or test_file.original_file_path)
    for test_file in test_files.test_files
]
logger.info(f"Tests {test_paths_display} failed to run, skipping")

Code Changes

File: codeflash/verification/parse_test_output.py

  • Line 1220-1223: Updated failure log to show actual test paths
  • Added comment explaining why we prefer instrumented_behavior_file_path

Testing

Verified the change improves log clarity when:

  • AI-generated tests fail to compile/run
  • Existing unit tests fail to execute
  • Mixed test scenarios (generated + existing)

Impact

This is a logging-only change with no behavioral impact. It improves debuggability by:

  • Showing actual file paths being executed
  • Eliminating confusing None values in logs
  • Making test execution failures easier to diagnose

🤖 Generated with Claude Code

…ile_path

For AI-generated tests, original_file_path is intentionally None.
When tests fail to run, the log now shows instrumented_behavior_file_path
(the actual path being executed) instead of original_file_path. This makes
debugging test execution failures much clearer.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@mashraf-222 mashraf-222 merged commit 4c93285 into omni-java Feb 10, 2026
15 of 27 checks passed
@mashraf-222 mashraf-222 deleted the fix/improve-test-failure-logging branch February 10, 2026 17:33
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants