Refactor to Remove Config Manager and Optimize Settings

[Umpire2018]

Summary

This PR focuses on removing the deprecated config_manager and associated configuration file saving functionality while introducing key improvements across the codebase.

Key Changes

1. Remove config_manager

• Eliminated the outdated config_manager module and configuration file handling.

2. Enhance Settings Management

• Implemented lazy loading in settings to optimize performance by deferring initialization until necessary.
• Added initialize_with_params method for dynamic configuration.
• Removed model name restrictions and streamlined default model naming.

3. Integrate LlamaIndex

• Migrated chat_with_repo to use llama_index for enhanced functionality:
  • Integrated llama_index.vector_stores.chroma and llama_index.embeddings.openai.
  • Added semantic chunking for better document segmentation.
  • Introduced ChatPromptTemplate for consistent prompt management.

4. Refactor CLI

• Converted print_hierarchy from a subcommand to an option (--print-hierarchy) in the run command for streamlined usage.

5. Logging Improvements

• Fixed logger to capture full error stack traces.
• Removed tiktoken dependency; token usage is now retrieved directly from API responses.
• Simplified and standardized import sorting.

6. Additional Updates

• Updated README to include GitHub Actions support.
• Declared chat-with-repo as an optional dependency in pyproject.toml.
• Removed deprecated files (config.toml.template and sync_func).
• Updated markdown documentation to reflect code changes.

Checklist

• Removed config_manager and its dependencies.
• Updated documentation to reflect changes.
• Verified compatibility with GitHub Actions.

Details

pdm run repoagent run --base-url 
2024-11-20 17:04:22.632 | SUCCESS  | repo_agent.log:set_logger_level_from_config:88 - Log level set to INFO!
parsing parent relationship: 0it [00:00, ?it/s]
Loading MetaInfo: .project_doc_record
MetaInfo is Refreshed and Saved
2024-11-20 17:04:22.636 | INFO     | repo_agent.runner:run:281 - Starting to detect changes.
Initializing MetaInfo: from .
generating repo structure: llm.py: 100%|█████████████████████████████████████████████████████| 1/1 [00:00<00:00, 195.72it/s]
parsing parent relationship: 100%|██████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 6141.00it/s]
parsing bidirectional reference:   0%|                                                                | 0/1 [00:00<?, ?it/s]2024-11-20 17:04:22.720 | INFO     | repo_agent.doc_meta_info:load_doc_from_older_meta:718 - merge doc from an older version of metainfo
parsing bidirectional reference: 100%|████████████████████████████████████████████████████████| 1/1 [00:00<00:00,  1.54it/s]
parsing topology task-list: 100%|████████████████████████████████████████████████████████| 10/10 [00:00<00:00, 16002.69it/s]
+---------+----------------------------+--------------------------------------------------+-------------+
| task_id |   Doc Generation Reason    |                       Path                       |  dependency |
+---------+----------------------------+--------------------------------------------------+-------------+
|    0    | doc_has_not_been_generated |             llm.py/is_url_reachable              |     None    |
|    1    | doc_has_not_been_generated |               llm.py/LLM/__init__                |     None    |
|    2    | doc_has_not_been_generated |     llm.py/LLM/_print_final_response_details     |     None    |
|    3    | doc_has_not_been_generated |   llm.py/LLM/_chat_with_ollama/fetch_response    |      2      |
|    4    | doc_has_not_been_generated |           llm.py/LLM/parse_json_string           |     None    |
|    5    | doc_has_not_been_generated | llm.py/LLM/_chat_with_openai_like/fetch_response |      4      |
|    6    | doc_has_not_been_generated |           llm.py/LLM/_chat_with_ollama           |      3      |
|    7    | doc_has_not_been_generated |        llm.py/LLM/_chat_with_openai_like         |      5      |
|    8    | doc_has_not_been_generated |                 llm.py/LLM/chat                  |    0,6,7    |
|    9    | doc_has_not_been_generated |                    llm.py/LLM                    | 1,2,4,6,7,8 |
+---------+----------------------------+--------------------------------------------------+-------------+
[process 0]: get task(0), remain(10)
 -- Generating document  _class_function: llm.py/is_url_reachable
[process 1]: get task(1), remain(10)
 -- Generating document  _class_function: llm.py/LLM/__init__
[process 2]: get task(2), remain(10)
 -- Generating document  _class_function: llm.py/LLM/_print_final_response_details
[process 3]: get task(4), remain(10)
 -- Generating document  _class_function: llm.py/LLM/parse_json_string
MetaInfo is Refreshed and Saved
2024-11-20 17:04:27.878 | INFO     | httpx._client:_send_single_request:1038 - HTTP Request: POST https://toollearning.cn/v1/chat/completions "HTTP/1.1 200 OK"
MetaInfo is Refreshed and Saved
2024-11-20 17:04:30.485 | INFO     | httpx._client:_send_single_request:1038 - HTTP Request: POST https://toollearning.cn/v1/chat/completions "HTTP/1.1 200 OK"
MetaInfo is Refreshed and Saved
2024-11-20 17:04:31.016 | INFO     | httpx._client:_send_single_request:1038 - HTTP Request: POST https://toollearning.cn/v1/chat/completions "HTTP/1.1 200 OK"
[process 2]: get task(3), remain(7)
 -- Generating document  _class_function: llm.py/LLM/_chat_with_ollama/fetch_response
MetaInfo is Refreshed and Saved
2024-11-20 17:04:39.907 | INFO     | httpx._client:_send_single_request:1038 - HTTP Request: POST https://toollearning.cn/v1/chat/completions "HTTP/1.1 200 OK"
[process 3]: get task(5), remain(6)
 -- Generating document  _class_function: llm.py/LLM/_chat_with_openai_like/fetch_response
MetaInfo is Refreshed and Saved
2024-11-20 17:04:41.247 | INFO     | httpx._client:_send_single_request:1038 - HTTP Request: POST https://toollearning.cn/v1/chat/completions "HTTP/1.1 200 OK"
[process 2]: get task(6), remain(5)
 -- Generating document  _class_function: llm.py/LLM/_chat_with_ollama
MetaInfo is Refreshed and Saved
2024-11-20 17:04:47.725 | INFO     | httpx._client:_send_single_request:1038 - HTTP Request: POST https://toollearning.cn/v1/chat/completions "HTTP/1.1 200 OK"
[process 3]: get task(7), remain(4)
 -- Generating document  _class_function: llm.py/LLM/_chat_with_openai_like
MetaInfo is Refreshed and Saved
2024-11-20 17:04:51.087 | INFO     | httpx._client:_send_single_request:1038 - HTTP Request: POST https://toollearning.cn/v1/chat/completions "HTTP/1.1 200 OK"
2024-11-20 17:04:57.380 | INFO     | httpx._client:_send_single_request:1038 - HTTP Request: POST https://toollearning.cn/v1/chat/completions "HTTP/1.1 200 OK"
MetaInfo is Refreshed and Saved
[process 3]: get task(8), remain(2)
 -- Generating document  _class_function: llm.py/LLM/chat
MetaInfo is Refreshed and Saved
2024-11-20 17:05:06.943 | INFO     | httpx._client:_send_single_request:1038 - HTTP Request: POST https://toollearning.cn/v1/chat/completions "HTTP/1.1 200 OK"
[process 3]: get task(9), remain(1)
 -- Generating document  _class: llm.py/LLM
MetaInfo is Refreshed and Saved
2024-11-20 17:05:17.322 | INFO     | httpx._client:_send_single_request:1038 - HTTP Request: POST https://toollearning.cn/v1/chat/completions "HTTP/1.1 200 OK"
MetaInfo is Refreshed and Saved
2024-11-20 17:05:17.600 | INFO     | repo_agent.runner:run:340 - Doc has been forwarded to the latest version
100%|███████████████████████████████████████████████████████████████████████████████████████| 1/1 [00:00<00:00, 3179.91it/s]
2024-11-20 17:05:17.602 | INFO     | repo_agent.runner:markdown_refresh:229 - Markdown documents have been refreshed at markdown_docs
2024-11-20 17:05:17.661 | INFO     | repo_agent.runner:run:345 - Starting to git-add DocMetaInfo and newly generated Docs
target_repo_path: .
already_staged_files:['.gitignore', '.project_doc_record/meta-info.json', '.project_doc_record/project_hierarchy.json', 'llm.py', 'pdm.lock', 'pyproject.toml']
untracked_files: ['markdown_docs/llm.md']
unstaged_files: ['.project_doc_record/meta-info.json', '.project_doc_record/project_hierarchy.json']
newly_staged_files: ['markdown_docs/llm.md', '.project_doc_record/meta-info.json', '.project_doc_record/project_hierarchy.json']
2024-11-20 17:05:18.683 | INFO     | repo_agent.runner:run:352 - Added ['markdown_docs/llm.md', '.project_doc_record/meta-info.json', '.project_doc_record/project_hierarchy.json'] to the staging area.
2024-11-20 17:05:18.684 | SUCCESS  | repo_agent.main:run:178 - Documentation task completed.