Skip to content

fix: increase Java test timeout from 15s to 120s#1337

Merged
misrasaurabh1 merged 1 commit into
omni-javafrom
fix/java-test-timeout-issue
Feb 4, 2026
Merged

fix: increase Java test timeout from 15s to 120s#1337
misrasaurabh1 merged 1 commit into
omni-javafrom
fix/java-test-timeout-issue

Conversation

@mashraf-222

Copy link
Copy Markdown

Problem

Java optimization was completely broken due to test timeouts. When running benchmarks on Java code, all optimization candidates were failing with:

ERROR: Maven test execution timed out after 15 seconds

This meant no Java code could be optimized through codeflash.

Root Cause

The 15-second test timeout was designed for Python's pytest, which starts almost instantly. Java's Maven has significant overhead:

  • Maven startup: 2-5 seconds
  • Test compilation: additional time
  • Test execution: actual test time

With only 15 seconds total, Maven couldn't even finish starting up before timing out.

Solution

Added a Java-specific timeout of 120 seconds that gives Maven enough time to:

  1. Start up (2-5s)
  2. Compile tests
  3. Run benchmarks with multiple loops
  4. Collect timing data

Code Changes

config_consts.py: Added Java timeout constant

INDIVIDUAL_TESTCASE_TIMEOUT = 15  # For Python pytest
JAVA_TESTCASE_TIMEOUT = 120  # Java Maven tests need more time

test_runner.py: Applied Java timeout when framework is JUnit5

  • Checks if test framework is "junit5"
  • Uses max(pytest_timeout, JAVA_TESTCASE_TIMEOUT) to respect explicit higher values
  • Adds debug logging when timeout is increased

Safety

This change ONLY affects Java tests:

  • Python/pytest: Still uses 15s ✓
  • JavaScript/Jest: Still uses 15s ✓
  • Java/JUnit5: Now uses 120s minimum ✓

If you explicitly pass a higher timeout, it's respected.

Testing

  • ✅ All 339 existing tests pass (2 test_runner + 337 Java tests)
  • ✅ E2E Java optimization now completes successfully
  • ✅ Original code benchmarked at 64.7ms per loop over 5 loops
  • ✅ 143 tests passed in E2E run (12 existing + 131 generated)

Before/After

Before:

❌ Maven test execution timed out after 15 seconds
❌ No optimizations found

After:

✅ Original code: 64.7ms per loop (5 loops completed)  
✅ All tests passing
✅ Optimization candidates being evaluated successfully

This unblocks Java optimization entirely.

Maven startup takes 2-5 seconds before tests even run, causing Java
optimization benchmarks to timeout at the default 15 second limit.

This fix adds a Java-specific timeout of 120 seconds that only applies
to JUnit5 tests. Python and JavaScript tests remain unchanged at 15s.

The timeout logic uses max(pytest_timeout, JAVA_TESTCASE_TIMEOUT) so
explicit higher timeouts are still respected.

Verified: All 339 tests pass, E2E Java optimization now completes
successfully without timeout errors.

Co-Authored-By: Claude Sonnet 4.5 <noreply@anthropic.com>
@mashraf-222 mashraf-222 requested a review from a team February 3, 2026 23:29
@misrasaurabh1 misrasaurabh1 merged commit dbb26df into omni-java Feb 4, 2026
13 of 24 checks passed
@misrasaurabh1 misrasaurabh1 deleted the fix/java-test-timeout-issue branch February 4, 2026 03:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants