Skip to content

⚡️ Speed up method TestGenRequest.to_payload by 20% in PR #1199 (omni-java)#1826

Merged
claude[bot] merged 2 commits into
omni-javafrom
codeflash/optimize-pr1199-2026-03-13T01.03.06
Mar 13, 2026
Merged

⚡️ Speed up method TestGenRequest.to_payload by 20% in PR #1199 (omni-java)#1826
claude[bot] merged 2 commits into
omni-javafrom
codeflash/optimize-pr1199-2026-03-13T01.03.06

Conversation

@codeflash-ai

@codeflash-ai codeflash-ai Bot commented Mar 13, 2026

Copy link
Copy Markdown
Contributor

⚡️ This pull request contains optimizations for PR #1199

If you approve this dependent PR, these changes will be merged into the original PR branch omni-java.

This PR will be automatically closed if the original PR is merged.


📄 20% (0.20x) speedup for TestGenRequest.to_payload in codeflash/api/schemas.py

⏱️ Runtime : 1.97 milliseconds 1.64 milliseconds (best of 141 runs)

📝 Explanation and details

The optimization hoists platform.python_version() out of the per-call loop by caching it at module import time as _PLATFORM_PYTHON_VERSION, eliminating a 500+ns system call on every invocation when the language is not Python. The original code imported platform and called platform.python_version() inside to_payload, incurring repeated overhead even though the value never changes within a process. Line profiler shows the import platform statement itself consumed 6.6% of runtime, and the conditional evaluation another 7%. The payload dict construction was also reordered to compute python_version upfront, reducing incremental dict updates. This yields a 20% speedup (1.97ms → 1.64ms) with no correctness trade-offs.

Correctness verification report:

Test Status
⚙️ Existing Unit Tests 🔘 None Found
🌀 Generated Regression Tests 1640 Passed
⏪ Replay Tests 🔘 None Found
🔎 Concolic Coverage Tests 2 Passed
📊 Tests Coverage 100.0%
🌀 Click to see Generated Regression Tests
from dataclasses import dataclass
from enum import Enum

# imports
import pytest
from codeflash.api.schemas import (LanguageInfo, ModuleSystem, TestFramework,
                                   TestGenRequest, TestInfo)

# test fixtures
@pytest.fixture
def basic_language_info():
    """Create a basic LanguageInfo instance for testing."""
    return LanguageInfo(name="python", version="3.9")

@pytest.fixture
def basic_test_info():
    """Create a basic TestInfo instance for testing."""
    return TestInfo(framework=TestFramework.PYTEST, timeout=30)

@pytest.fixture
def basic_request(basic_language_info, basic_test_info):
    """Create a basic TestGenRequest instance for testing."""
    return TestGenRequest(
        source_code="def add(a, b):\n    return a + b",
        function_name="add",
        trace_id="trace-123",
        language_info=basic_language_info,
        test_info=basic_test_info,
    )

# ============================================================================
# BASIC TESTS - Verify fundamental functionality under normal conditions
# ============================================================================

def test_to_payload_basic_structure(basic_request):
    """Test that to_payload returns a dict with all required keys."""
    payload = basic_request.to_payload() # 3.86μs -> 2.85μs (35.6% faster)
    
    # Verify return type is dict
    assert isinstance(payload, dict)
    
    # Verify all essential keys are present
    assert "source_code_being_tested" in payload
    assert "function_to_optimize" in payload
    assert "helper_function_names" in payload
    assert "module_path" in payload
    assert "test_module_path" in payload
    assert "test_framework" in payload
    assert "test_timeout" in payload
    assert "trace_id" in payload
    assert "test_index" in payload
    assert "language" in payload
    assert "codeflash_version" in payload
    assert "is_async" in payload
    assert "is_numerical_code" in payload
    assert "language_version" in payload
    assert "python_version" in payload

def test_to_payload_basic_values(basic_request):
    """Test that to_payload correctly maps basic field values."""
    payload = basic_request.to_payload() # 3.55μs -> 2.58μs (37.7% faster)
    
    # Verify core fields are correctly mapped
    assert payload["source_code_being_tested"] == "def add(a, b):\n    return a + b"
    assert payload["trace_id"] == "trace-123"
    assert payload["test_index"] == 0
    assert payload["language"] == "python"

def test_to_payload_function_to_optimize_structure(basic_request):
    """Test that function_to_optimize nested object has correct structure."""
    payload = basic_request.to_payload() # 3.46μs -> 2.56μs (34.8% faster)
    
    # Verify nested object exists and has correct structure
    assert isinstance(payload["function_to_optimize"], dict)
    assert payload["function_to_optimize"]["function_name"] == "add"
    assert payload["function_to_optimize"]["is_async"] is False

def test_to_payload_with_async_function(basic_language_info, basic_test_info):
    """Test to_payload with async function flag set to True."""
    request = TestGenRequest(
        source_code="async def fetch():\n    return data",
        function_name="fetch",
        trace_id="trace-async",
        language_info=basic_language_info,
        test_info=basic_test_info,
        is_async=True,
    )
    payload = request.to_payload() # 3.39μs -> 2.46μs (37.4% faster)
    
    # Verify async flag is propagated correctly
    assert payload["is_async"] is True
    assert payload["function_to_optimize"]["is_async"] is True

def test_to_payload_with_helper_functions(basic_language_info, basic_test_info):
    """Test to_payload with non-empty helper_function_names list."""
    request = TestGenRequest(
        source_code="def main():\n    pass",
        function_name="main",
        trace_id="trace-helper",
        language_info=basic_language_info,
        test_info=basic_test_info,
        helper_function_names=["helper1", "helper2", "helper3"],
    )
    payload = request.to_payload() # 3.39μs -> 2.46μs (37.4% faster)
    
    # Verify helper function names are correctly included
    assert payload["helper_function_names"] == ["helper1", "helper2", "helper3"]

def test_to_payload_test_framework_value(basic_language_info, basic_test_info):
    """Test that test_framework is correctly converted to its enum value."""
    payload = basic_language_info, basic_test_info
    request = TestGenRequest(
        source_code="def test():\n    pass",
        function_name="test",
        trace_id="trace-fw",
        language_info=basic_language_info,
        test_info=basic_test_info,
    )
    payload = request.to_payload() # 3.41μs -> 2.46μs (38.7% faster)
    
    # Verify framework value is the string representation of the enum
    assert payload["test_framework"] == TestFramework.PYTEST.value
    assert isinstance(payload["test_framework"], str)

def test_to_payload_test_timeout(basic_language_info, basic_test_info):
    """Test that test_timeout is correctly included in payload."""
    payload = basic_language_info, basic_test_info
    request = TestGenRequest(
        source_code="def test():\n    pass",
        function_name="test",
        trace_id="trace-timeout",
        language_info=basic_language_info,
        test_info=basic_test_info,
    )
    payload = request.to_payload() # 3.52μs -> 2.52μs (39.3% faster)
    
    # Verify timeout is correctly mapped
    assert payload["test_timeout"] == 30

def test_to_payload_python_version_backward_compat(basic_language_info, basic_test_info):
    """Test that python_version is set from language_info for Python language."""
    request = TestGenRequest(
        source_code="def test():\n    pass",
        function_name="test",
        trace_id="trace-pyver",
        language_info=basic_language_info,
        test_info=basic_test_info,
    )
    payload = request.to_payload() # 3.44μs -> 2.52μs (36.1% faster)
    
    # Verify python_version is set from language_info.version for Python
    assert payload["python_version"] == "3.9"
    assert payload["language_version"] == "3.9"

def test_to_payload_codeflash_version(basic_language_info, basic_test_info):
    """Test that codeflash_version is correctly included."""
    request = TestGenRequest(
        source_code="def test():\n    pass",
        function_name="test",
        trace_id="trace-version",
        language_info=basic_language_info,
        test_info=basic_test_info,
        codeflash_version="1.2.3",
    )
    payload = request.to_payload() # 3.48μs -> 2.50μs (39.3% faster)
    
    # Verify codeflash_version is correctly included
    assert payload["codeflash_version"] == "1.2.3"

def test_to_payload_with_module_paths(basic_language_info, basic_test_info):
    """Test to_payload with non-empty module paths."""
    request = TestGenRequest(
        source_code="def test():\n    pass",
        function_name="test",
        trace_id="trace-paths",
        language_info=basic_language_info,
        test_info=basic_test_info,
        module_path="src/module.py",
        test_module_path="tests/test_module.py",
    )
    payload = request.to_payload() # 3.39μs -> 2.48μs (36.3% faster)
    
    # Verify module paths are correctly included
    assert payload["module_path"] == "src/module.py"
    assert payload["test_module_path"] == "tests/test_module.py"

def test_to_payload_with_test_index(basic_language_info, basic_test_info):
    """Test to_payload with non-zero test_index."""
    request = TestGenRequest(
        source_code="def test():\n    pass",
        function_name="test",
        trace_id="trace-idx",
        language_info=basic_language_info,
        test_info=basic_test_info,
        test_index=5,
    )
    payload = request.to_payload() # 3.42μs -> 2.51μs (35.9% faster)
    
    # Verify test_index is correctly included
    assert payload["test_index"] == 5

def test_to_payload_is_numerical_code_true(basic_language_info, basic_test_info):
    """Test to_payload with is_numerical_code set to True."""
    request = TestGenRequest(
        source_code="def calculate():\n    return 42",
        function_name="calculate",
        trace_id="trace-num",
        language_info=basic_language_info,
        test_info=basic_test_info,
        is_numerical_code=True,
    )
    payload = request.to_payload() # 3.42μs -> 2.45μs (39.2% faster)
    
    # Verify is_numerical_code is correctly included
    assert payload["is_numerical_code"] is True

def test_to_payload_is_numerical_code_false(basic_language_info, basic_test_info):
    """Test to_payload with is_numerical_code set to False."""
    request = TestGenRequest(
        source_code="def greet():\n    return 'hello'",
        function_name="greet",
        trace_id="trace-text",
        language_info=basic_language_info,
        test_info=basic_test_info,
        is_numerical_code=False,
    )
    payload = request.to_payload() # 3.34μs -> 2.44μs (37.0% faster)
    
    # Verify is_numerical_code is correctly set to False
    assert payload["is_numerical_code"] is False

# ============================================================================
# EDGE TESTS - Evaluate behavior under extreme or unusual conditions
# ============================================================================

def test_to_payload_with_empty_source_code(basic_language_info, basic_test_info):
    """Test to_payload with empty source_code string."""
    request = TestGenRequest(
        source_code="",
        function_name="empty",
        trace_id="trace-empty-src",
        language_info=basic_language_info,
        test_info=basic_test_info,
    )
    payload = request.to_payload() # 3.31μs -> 2.50μs (32.0% faster)
    
    # Verify empty source code is preserved
    assert payload["source_code_being_tested"] == ""

def test_to_payload_with_empty_function_name(basic_language_info, basic_test_info):
    """Test to_payload with empty function_name string."""
    request = TestGenRequest(
        source_code="def f():\n    pass",
        function_name="",
        trace_id="trace-empty-fn",
        language_info=basic_language_info,
        test_info=basic_test_info,
    )
    payload = request.to_payload() # 3.34μs -> 2.46μs (35.4% faster)
    
    # Verify empty function name is preserved
    assert payload["function_to_optimize"]["function_name"] == ""

def test_to_payload_with_empty_trace_id(basic_language_info, basic_test_info):
    """Test to_payload with empty trace_id string."""
    request = TestGenRequest(
        source_code="def test():\n    pass",
        function_name="test",
        trace_id="",
        language_info=basic_language_info,
        test_info=basic_test_info,
    )
    payload = request.to_payload() # 3.37μs -> 2.49μs (35.0% faster)
    
    # Verify empty trace_id is preserved
    assert payload["trace_id"] == ""

def test_to_payload_with_empty_helper_functions_list(basic_language_info, basic_test_info):
    """Test to_payload with default empty helper_function_names list."""
    request = TestGenRequest(
        source_code="def test():\n    pass",
        function_name="test",
        trace_id="trace-empty-helpers",
        language_info=basic_language_info,
        test_info=basic_test_info,
        helper_function_names=[],
    )
    payload = request.to_payload() # 3.41μs -> 2.52μs (35.4% faster)
    
    # Verify empty list is preserved
    assert payload["helper_function_names"] == []
    assert isinstance(payload["helper_function_names"], list)

def test_to_payload_with_empty_module_paths(basic_language_info, basic_test_info):
    """Test to_payload with default empty module_path and test_module_path."""
    request = TestGenRequest(
        source_code="def test():\n    pass",
        function_name="test",
        trace_id="trace-empty-paths",
        language_info=basic_language_info,
        test_info=basic_test_info,
        module_path="",
        test_module_path="",
    )
    payload = request.to_payload() # 3.37μs -> 2.46μs (37.1% faster)
    
    # Verify empty paths are preserved
    assert payload["module_path"] == ""
    assert payload["test_module_path"] == ""

def test_to_payload_with_empty_codeflash_version(basic_language_info, basic_test_info):
    """Test to_payload with default empty codeflash_version."""
    request = TestGenRequest(
        source_code="def test():\n    pass",
        function_name="test",
        trace_id="trace-empty-ver",
        language_info=basic_language_info,
        test_info=basic_test_info,
        codeflash_version="",
    )
    payload = request.to_payload() # 3.46μs -> 2.44μs (41.3% faster)
    
    # Verify empty version is preserved
    assert payload["codeflash_version"] == ""

def test_to_payload_with_is_numerical_code_none(basic_language_info, basic_test_info):
    """Test to_payload with is_numerical_code set to None."""
    request = TestGenRequest(
        source_code="def test():\n    pass",
        function_name="test",
        trace_id="trace-none-num",
        language_info=basic_language_info,
        test_info=basic_test_info,
        is_numerical_code=None,
    )
    payload = request.to_payload() # 3.38μs -> 2.48μs (35.9% faster)
    
    # Verify None value is preserved
    assert payload["is_numerical_code"] is None

def test_to_payload_with_test_index_zero(basic_language_info, basic_test_info):
    """Test to_payload with test_index explicitly set to 0."""
    request = TestGenRequest(
        source_code="def test():\n    pass",
        function_name="test",
        trace_id="trace-idx-zero",
        language_info=basic_language_info,
        test_info=basic_test_info,
        test_index=0,
    )
    payload = request.to_payload() # 3.36μs -> 2.46μs (36.2% faster)
    
    # Verify test_index 0 is correctly included
    assert payload["test_index"] == 0

def test_to_payload_with_negative_test_index(basic_language_info, basic_test_info):
    """Test to_payload with negative test_index."""
    request = TestGenRequest(
        source_code="def test():\n    pass",
        function_name="test",
        trace_id="trace-idx-neg",
        language_info=basic_language_info,
        test_info=basic_test_info,
        test_index=-1,
    )
    payload = request.to_payload() # 3.34μs -> 2.44μs (37.0% faster)
    
    # Verify negative test_index is preserved
    assert payload["test_index"] == -1

def test_to_payload_with_large_test_index(basic_language_info, basic_test_info):
    """Test to_payload with very large test_index."""
    request = TestGenRequest(
        source_code="def test():\n    pass",
        function_name="test",
        trace_id="trace-idx-large",
        language_info=basic_language_info,
        test_info=basic_test_info,
        test_index=999999,
    )
    payload = request.to_payload() # 3.29μs -> 2.49μs (31.8% faster)
    
    # Verify large test_index is preserved
    assert payload["test_index"] == 999999

def test_to_payload_with_special_characters_in_source_code(basic_language_info, basic_test_info):
    """Test to_payload with special characters in source_code."""
    special_code = "def test():\n    return '\\n\\t\\\\\"'"
    request = TestGenRequest(
        source_code=special_code,
        function_name="test",
        trace_id="trace-special",
        language_info=basic_language_info,
        test_info=basic_test_info,
    )
    payload = request.to_payload() # 3.32μs -> 2.37μs (39.7% faster)
    
    # Verify special characters are preserved
    assert payload["source_code_being_tested"] == special_code

def test_to_payload_with_special_characters_in_function_name(basic_language_info, basic_test_info):
    """Test to_payload with special characters in function_name."""
    request = TestGenRequest(
        source_code="def test():\n    pass",
        function_name="test_func_v2_beta",
        trace_id="trace-special-fn",
        language_info=basic_language_info,
        test_info=basic_test_info,
    )
    payload = request.to_payload() # 3.35μs -> 2.44μs (36.9% faster)
    
    # Verify function name with special valid characters is preserved
    assert payload["function_to_optimize"]["function_name"] == "test_func_v2_beta"

def test_to_payload_with_unicode_in_source_code(basic_language_info, basic_test_info):
    """Test to_payload with unicode characters in source_code."""
    unicode_code = "def greet():\n    return 'Héllo Wørld'"
    request = TestGenRequest(
        source_code=unicode_code,
        function_name="greet",
        trace_id="trace-unicode",
        language_info=basic_language_info,
        test_info=basic_test_info,
    )
    payload = request.to_payload() # 3.47μs -> 2.46μs (41.2% faster)
    
    # Verify unicode characters are preserved
    assert payload["source_code_being_tested"] == unicode_code

def test_to_payload_with_very_long_source_code(basic_language_info, basic_test_info):
    """Test to_payload with very long source_code."""
    long_code = "def test():\n    " + "x = 1\n    " * 1000
    request = TestGenRequest(
        source_code=long_code,
        function_name="test",
        trace_id="trace-long-src",
        language_info=basic_language_info,
        test_info=basic_test_info,
    )
    payload = request.to_payload() # 3.40μs -> 2.45μs (38.4% faster)
    
    # Verify long source code is preserved
    assert payload["source_code_being_tested"] == long_code
    assert len(payload["source_code_being_tested"]) > 5000

def test_to_payload_with_very_long_function_name(basic_language_info, basic_test_info):
    """Test to_payload with very long function_name."""
    long_name = "test_" + "a" * 500
    request = TestGenRequest(
        source_code="def test():\n    pass",
        function_name=long_name,
        trace_id="trace-long-fn",
        language_info=basic_language_info,
        test_info=basic_test_info,
    )
    payload = request.to_payload() # 3.41μs -> 2.42μs (40.5% faster)
    
    # Verify long function name is preserved
    assert payload["function_to_optimize"]["function_name"] == long_name

def test_to_payload_with_single_helper_function(basic_language_info, basic_test_info):
    """Test to_payload with single helper function name."""
    request = TestGenRequest(
        source_code="def test():\n    pass",
        function_name="test",
        trace_id="trace-single-helper",
        language_info=basic_language_info,
        test_info=basic_test_info,
        helper_function_names=["helper"],
    )
    payload = request.to_payload() # 3.28μs -> 2.50μs (30.8% faster)
    
    # Verify single helper function is correctly included
    assert payload["helper_function_names"] == ["helper"]
    assert len(payload["helper_function_names"]) == 1

def test_to_payload_with_many_helper_functions(basic_language_info, basic_test_info):
    """Test to_payload with many helper function names."""
    helpers = ["helper_" + str(i) for i in range(50)]
    request = TestGenRequest(
        source_code="def test():\n    pass",
        function_name="test",
        trace_id="trace-many-helpers",
        language_info=basic_language_info,
        test_info=basic_test_info,
        helper_function_names=helpers,
    )
    payload = request.to_payload() # 3.38μs -> 2.42μs (39.3% faster)
    
    # Verify all helper functions are correctly included
    assert payload["helper_function_names"] == helpers
    assert len(payload["helper_function_names"]) == 50

def test_to_payload_non_python_language(basic_test_info):
    """Test to_payload with non-Python language."""
    language_info = LanguageInfo(name="javascript", version="16.0.0")
    request = TestGenRequest(
        source_code="function add(a, b) { return a + b; }",
        function_name="add",
        trace_id="trace-js",
        language_info=language_info,
        test_info=basic_test_info,
    )
    payload = request.to_payload() # 4.11μs -> 2.42μs (69.4% faster)
    
    # Verify language and version are correctly set
    assert payload["language"] == "javascript"
    assert payload["language_version"] == "16.0.0"
    # python_version should still be populated with platform version for non-Python
    assert isinstance(payload["python_version"], str)

def test_to_payload_with_module_system_esm(basic_test_info):
    """Test to_payload with JavaScript ESM module system."""
    language_info = LanguageInfo(
        name="javascript",
        version="16.0.0",
        module_system=ModuleSystem.ESM
    )
    request = TestGenRequest(
        source_code="export function add(a, b) { return a + b; }",
        function_name="add",
        trace_id="trace-esm",
        language_info=language_info,
        test_info=basic_test_info,
    )
    payload = request.to_payload() # 4.38μs -> 2.92μs (50.2% faster)
    
    # Verify module_system is correctly included
    assert "module_system" in payload
    assert payload["module_system"] == ModuleSystem.ESM.value

def test_to_payload_with_module_system_commonjs(basic_test_info):
    """Test to_payload with JavaScript CommonJS module system."""
    language_info = LanguageInfo(
        name="javascript",
        version="16.0.0",
        module_system=ModuleSystem.COMMONJS
    )
    request = TestGenRequest(
        source_code="module.exports = { add: (a, b) => a + b };",
        function_name="add",
        trace_id="trace-cjs",
        language_info=language_info,
        test_info=basic_test_info,
    )
    payload = request.to_payload() # 4.21μs -> 2.79μs (50.6% faster)
    
    # Verify module_system is correctly included
    assert "module_system" in payload
    assert payload["module_system"] == ModuleSystem.COMMONJS.value

def test_to_payload_with_module_system_unknown(basic_test_info):
    """Test to_payload with unknown module system (should not include it)."""
    language_info = LanguageInfo(
        name="python",
        version="3.9",
        module_system=ModuleSystem.UNKNOWN
    )
    request = TestGenRequest(
        source_code="def add(a, b):\n    return a + b",
        function_name="add",
        trace_id="trace-unknown-mod",
        language_info=language_info,
        test_info=basic_test_info,
    )
    payload = request.to_payload() # 3.07μs -> 2.30μs (33.1% faster)
    
    # Verify module_system is not included for UNKNOWN
    assert "module_system" not in payload or payload.get("module_system") is None

def test_to_payload_with_different_test_frameworks(basic_language_info):
    """Test to_payload with different test frameworks."""
    for framework in [TestFramework.PYTEST, TestFramework.UNITTEST]:
        test_info = TestInfo(framework=framework, timeout=30)
        request = TestGenRequest(
            source_code="def test():\n    pass",
            function_name="test",
            trace_id="trace-fw-" + framework.name,
            language_info=basic_language_info,
            test_info=test_info,
        )
        payload = request.to_payload() # 4.59μs -> 3.41μs (34.6% faster)
        
        # Verify framework value matches
        assert payload["test_framework"] == framework.value

def test_to_payload_does_not_mutate_request(basic_language_info, basic_test_info):
    """Test that to_payload does not mutate the original request object."""
    original_helpers = ["helper1", "helper2"]
    request = TestGenRequest(
        source_code="def test():\n    pass",
        function_name="test",
        trace_id="trace-immutable",
        language_info=basic_language_info,
        test_info=basic_test_info,
        helper_function_names=original_helpers.copy(),
    )
    
    # Call to_payload
    payload = request.to_payload() # 3.14μs -> 2.39μs (31.0% faster)
    
    # Verify request object is unchanged
    assert request.helper_function_names == original_helpers
    assert request.source_code == "def test():\n    pass"
    assert request.function_name == "test"

# ============================================================================
# LARGE-SCALE TESTS - Assess performance and scalability
# ============================================================================

def test_to_payload_with_large_number_of_helpers(basic_language_info, basic_test_info):
    """Test to_payload with 1000 helper function names."""
    helpers = ["helper_func_" + str(i) for i in range(1000)]
    request = TestGenRequest(
        source_code="def test():\n    pass",
        function_name="test",
        trace_id="trace-1k-helpers",
        language_info=basic_language_info,
        test_info=basic_test_info,
        helper_function_names=helpers,
    )
    payload = request.to_payload() # 3.45μs -> 2.65μs (30.3% faster)
    
    # Verify all 1000 helpers are included
    assert len(payload["helper_function_names"]) == 1000
    assert payload["helper_function_names"] == helpers
    # Verify first and last elements
    assert payload["helper_function_names"][0] == "helper_func_0"
    assert payload["helper_function_names"][999] == "helper_func_999"

def test_to_payload_with_extremely_large_source_code(basic_language_info, basic_test_info):
    """Test to_payload with extremely large source code (10000+ lines)."""
    # Create source code with 1000 function definitions
    large_code = "\n".join([f"def func_{i}():\n    return {i}" for i in range(1000)])
    request = TestGenRequest(
        source_code=large_code,
        function_name="func_500",
        trace_id="trace-10k-lines",
        language_info=basic_language_info,
        test_info=basic_test_info,
    )
    payload = request.to_payload() # 3.68μs -> 2.73μs (34.9% faster)
    
    # Verify large source code is preserved
    assert payload["source_code_being_tested"] == large_code
    assert len(payload["source_code_being_tested"]) > 10000
    assert payload["source_code_being_tested"].count("def func_") == 1000

def test_to_payload_multiple_calls_consistency(basic_language_info, basic_test_info):
    """Test that multiple calls to to_payload return identical results."""
    request = TestGenRequest(
        source_code="def test():\n    pass",
        function_name="test",
        trace_id="trace-consistent",
        language_info=basic_language_info,
        test_info=basic_test_info,
        helper_function_names=["helper1", "helper2"],
        test_index=10,
        codeflash_version="1.0.0",
    )
    
    # Call to_payload multiple times
    payloads = [request.to_payload() for _ in range(100)]
    
    # Verify all payloads are identical
    first_payload = payloads[0]
    for payload in payloads[1:]:
        assert payload == first_payload

def test_to_payload_with_varied_field_combinations(basic_language_info, basic_test_info):
    """Test to_payload with 1000 varied field combinations."""
    for i in range(1000):
        request = TestGenRequest(
            source_code=f"def func_{i}():\n    return {i}",
            function_name=f"func_{i}",
            trace_id=f"trace-{i}",
            language_info=basic_language_info,
            test_info=basic_test_info,
            module_path=f"module_{i}.py" if i % 2 == 0 else "",
            test_module_path=f"test_module_{i}.py" if i % 3 == 0 else "",
            helper_function_names=[f"helper_{j}" for j in range(i % 10)],
            is_async=i % 2 == 0,
            is_numerical_code=i % 3 == 0,
            test_index=i,
            codeflash_version=f"1.{i}.0",
        )
        payload = request.to_payload() # 1.13ms -> 948μs (19.5% faster)
        
        # Verify payload structure for each iteration
        assert isinstance(payload, dict)
        assert payload["function_to_optimize"]["function_name"] == f"func_{i}"
        assert payload["trace_id"] == f"trace-{i}"
        assert payload["test_index"] == i
        assert payload["is_async"] == (i % 2 == 0)
        assert payload["is_numerical_code"] == (i % 3 == 0)

def test_to_payload_key_count_consistency(basic_language_info, basic_test_info):
    """Test that to_payload always returns same number of keys."""
    payloads_keys_count = []
    
    # Generate 100 different requests
    for i in range(100):
        request = TestGenRequest(
            source_code=f"def func_{i}():\n    pass",
            function_name=f"func_{i}",
            trace_id=f"trace-keys-{i}",
            language_info=basic_language_info,
            test_info=basic_test_info,
            helper_function_names=[f"h_{j}" for j in range(i % 3)],
            is_async=bool(i % 2),
            is_numerical_code=bool(i % 3),
        )
        payload = request.to_payload() # 115μs -> 97.6μs (18.5% faster)
        payloads_keys_count.append(len(payload.keys()))
    
    # Verify all payloads have same number of keys (excluding optional module_system)
    # The base set should be consistent
    base_count = payloads_keys_count[0]
    for count in payloads_keys_count:
        # Allow for module_system key which may or may not be present
        assert count == base_count or count == base_count + 1

def test_to_payload_all_values_are_serializable(basic_language_info, basic_test_info):
    """Test that all payload values are JSON-serializable for 500 requests."""
    import json
    
    for i in range(500):
        request = TestGenRequest(
            source_code=f"def func_{i}():\n    return {i}",
            function_name=f"func_{i}",
            trace_id=f"trace-json-{i}",
            language_info=basic_language_info,
            test_info=basic_test_info,
            helper_function_names=[f"helper_{j}" for j in range(i % 5)],
            is_async=i % 2 == 0,
            is_numerical_code=i % 3 == 0,
            test_index=i,
            codeflash_version=f"1.{i}.0",
        )
        payload = request.to_payload() # 582μs -> 491μs (18.5% faster)
        
        # Verify all values are JSON serializable
        try:
            json.dumps(payload)
        except (TypeError, ValueError) as e:
            pytest.fail(f"Payload at iteration {i} is not JSON serializable: {e}")
from codeflash.api.schemas import LanguageInfo
from codeflash.api.schemas import ModuleSystem
from codeflash.api.schemas import TestFramework
from codeflash.api.schemas import TestGenRequest
from codeflash.api.schemas import TestInfo

def test_TestGenRequest_to_payload():
    TestGenRequest.to_payload(TestGenRequest('', '', '', LanguageInfo('', version=None, module_system=ModuleSystem.ESM, file_extension='', has_type_annotations=False, type_checker=None), TestInfo(TestFramework.JEST, timeout=0, test_patterns=[''], tests_root=''), module_path='', test_module_path='', helper_function_names=[], is_async=False, is_numerical_code=True, test_index=0, codeflash_version=''))

def test_TestGenRequest_to_payload_2():
    TestGenRequest.to_payload(TestGenRequest('', '', '', LanguageInfo('python', version=None, module_system=ModuleSystem.ESM, file_extension='', has_type_annotations=False, type_checker=None), TestInfo(TestFramework.JEST, timeout=0, test_patterns=[], tests_root=''), module_path='', test_module_path='', helper_function_names=[], is_async=False, is_numerical_code=False, test_index=0, codeflash_version=''))
🔎 Click to see Concolic Coverage Tests

To edit these changes git checkout codeflash/optimize-pr1199-2026-03-13T01.03.06 and push.

Codeflash Static Badge

The optimization hoists `platform.python_version()` out of the per-call loop by caching it at module import time as `_PLATFORM_PYTHON_VERSION`, eliminating a 500+ns system call on every invocation when the language is not Python. The original code imported `platform` and called `platform.python_version()` inside `to_payload`, incurring repeated overhead even though the value never changes within a process. Line profiler shows the `import platform` statement itself consumed 6.6% of runtime, and the conditional evaluation another 7%. The payload dict construction was also reordered to compute `python_version` upfront, reducing incremental dict updates. This yields a 20% speedup (1.97ms → 1.64ms) with no correctness trade-offs.
@codeflash-ai codeflash-ai Bot added ⚡️ codeflash Optimization PR opened by Codeflash AI 🎯 Quality: High Optimization Quality according to Codeflash labels Mar 13, 2026
@codeflash-ai codeflash-ai Bot mentioned this pull request Mar 13, 2026
@claude

claude Bot commented Mar 13, 2026

Copy link
Copy Markdown
Contributor

Claude finished @codeflash-ai[bot]'s task in 18s —— View job


PR Review Summary

Prek Checks

  • ruff-format: Had a minor formatting issue — the 3-line ternary expression in to_payload could fit on one line. Auto-fixed and pushed in commit ee6749fb.
  • ruff check: Passed.
  • mypy: No issues on codeflash/api/schemas.py.

Code Review

The optimization is correct and clean:

  • Moves import platform from inside the method to module level (schemas.py:14)
  • Caches platform.python_version() as _PLATFORM_PYTHON_VERSION at import time (schemas.py:20), eliminating repeated system calls
  • Inlines the python_version computation directly into the dict literal

No bugs, no security issues, no breaking API changes found.

Minor note: OptimizeRequest.to_payload (schemas.py:126) still contains import platform inside the method body — the same pattern this PR fixes in TestGenRequest.to_payload. However, PR #1825 addresses exactly that: ⚡️ Speed up method OptimizeRequest.to_payload by 33%.

Duplicate Detection

No duplicates introduced. The sibling method OptimizeRequest.to_payload has the same pre-optimization pattern but is being addressed in PR #1825.

Test Coverage

Test suite ran (3437 passed, 57 skipped). One pre-existing failure in tests/test_tracer.py::TestTracer::test_tracer_initialization_normal (unrelated to this PR) caused coverage data collection to be incomplete. The correctness verification in the PR body reports 1640 regression tests passed with 100% coverage of the changed code.

Optimization PR Handling

PR #1825 (OptimizeRequest.to_payload 33% speedup) — CI still in progress. Will not merge until all checks complete.


Last updated: 2026-03-13T01:13Z

@claude claude Bot merged commit 13fcb04 into omni-java Mar 13, 2026
26 of 27 checks passed
@claude claude Bot deleted the codeflash/optimize-pr1199-2026-03-13T01.03.06 branch March 13, 2026 01:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

⚡️ codeflash Optimization PR opened by Codeflash AI 🎯 Quality: High Optimization Quality according to Codeflash

Projects

None yet

Development

Successfully merging this pull request may close these issues.

0 participants