docs/cursor_rules.md

Okay, this is a great strategic checkpoint. Before diving into more MVP sprints, establishing a comprehensive and optimized set of Cursor rules tailored for Claude 4.0 and your specific project (KGraph-MCP with its 5 MVPs) will significantly enhance development consistency, code quality, and AI-assistance effectiveness.

I'll go through the essence of your 5 MVPs to identify recurring patterns, key technologies, and architectural principles that should be encoded into these rules.

**Analysis of MVP Requirements for Rule Generation:**

*   **MVP 1 (KG-Powered Tool Suggester):**
    *   Python dataclasses (`MCPTool`).
    *   JSON data handling (`initial_tools.json`).
    *   In-memory KG logic (dictionaries, lists).
    *   Vector embeddings (LLM API calls via `EmbeddingService`).
    *   NumPy for cosine similarity.
    *   Basic Gradio UI (`app.py`).
    *   Testing with `pytest` and `unittest.mock`.
*   **MVP 2 (KG Suggests Actionable Tool with Prompt Template):**
    *   New dataclass (`MCPPrompt`, `PlannedStep`).
    *   More JSON data (`initial_prompts.json`).
    *   Enhanced `InMemoryKG` (prompt loading, indexing, search).
    *   Enhanced `SimplePlannerAgent` (tool + prompt selection).
    *   Gradio UI updates to display richer plan info.
*   **MVP 3 (Interactive Prompt Filling & Simulated Execution):**
    *   Dynamic Gradio UI component generation/updates.
    *   User input collection from Gradio.
    *   `StubExecutorAgent` with mocked, tool-specific responses.
    *   Formatting complex data for Gradio Markdown/JSON display.
*   **MVP 4 (Real MCP Tool Execution via Gradio Server):**
    *   **New Sub-Projects (Track 1 Tools):** Each a self-contained Gradio app with `gradio_mcp`. Likely to use `requests` for external API calls (e.g., HF Inference).
    *   Main Project: `McpExecutorAgent` making HTTP `POST` requests to live MCP servers. JSON payload construction and response parsing. Error handling for network calls.
    *   Updating `MCPTool` ontology to include `execution_type` and `mcp_endpoint_url`.
*   **MVP 5 (Dynamic Reasoning via Simulated Sampling - Conceptual):**
    *   (If reached) Agent logic making direct LLM calls (simulating MCP Sampling) for dynamic reasoning/refinement.
    *   Formatting complex MCP `sampling/createMessage` like JSON structures.

**Common Themes & Technologies Across MVPs:**

*   **Core Language:** Python (heavily using dataclasses, type hints).
*   **Primary UI:** Gradio.
*   **KG/Data:** JSON for data definition, Python dicts/lists for in-memory representation. NumPy for vector math.
*   **LLM Interaction:** `openai` or `anthropic` client libraries for embeddings and (later) chat completions. `requests` for general HTTP calls (to MCP servers or other APIs).
*   **Testing:** `pytest`, `unittest.mock`.
*   **Tooling:** `uv`, `just`, Conventional Commits.
*   **Project Structure:** Modular (`kg_services/`, `agents/`, `schemas/`, `data/`, `tests/`).
*   **Error Handling:** Essential for API calls and data processing.
*   **Logging/Printing:** For observability.

---

**Comprehensive & Optimized Cursor Rules (`.cursor/rules/`) for KGraph-MCP Project**

This set of rules is designed to be modular and hierarchical, leveraging Cursor's `.mdc` format with YAML frontmatter.

**Directory Structure for Rules:**

```
.cursor/
└── rules/
    ├── general_python.mdc           # Base Python rules
    ├── data_handling.mdc            # JSON, dataclasses, KG structures
    ├── llm_interaction.mdc          # Embedding, API calls, (future) sampling
    ├── gradio_ui.mdc                # Gradio specific best practices
    ├── agent_logic.mdc              # Rules for Planner, Executor etc.
    ├── testing_pytest.mdc           # Pytest and mocking conventions
    ├── mcp_specific/                # Sub-directory for MCP server rules
    │   └── gradio_mcp_server.mdc
    └── project_specific.mdc         # Overarching KGraph-MCP conventions
```

---

**Content of `.mdc` Rule Files:**

**1. `general_python.mdc`**

```markdown
---
description: Core Python coding standards, formatting, and type hinting for the KGraph-MCP project.
globs: ["**/*.py"]
alwaysApply: true
---

### Formatting & Style
- Adhere strictly to PEP 8 guidelines for code layout.
- Use **Black** for automated code formatting. Max line length: 88 characters.
- Use double quotes for strings, unless single quotes improve readability (e.g., in f-strings containing double quotes).
- Employ f-strings for string formatting where possible.

### Type Hinting (PEP 484)
- **Mandatory Type Hints:** All function/method parameters and return types MUST be type hinted.
- Use types from the `typing` module (e.g., `List`, `Dict`, `Optional`, `Any`, `Tuple`, `Callable`).
- For complex types or dataclasses, ensure they are correctly imported and referenced.
- Variables should also be type hinted where it improves clarity, especially for complex data structures.
- Configure Mypy for strict checking (see `pyproject.toml`). Aim for no Mypy errors.

### Docstrings & Comments
- **PEP 257 for Docstrings:** All public modules, classes, functions, and methods MUST have docstrings.
  - Use reStructuredText or Google-style docstrings for clarity.
  - First line: concise summary.
  - Subsequent paragraphs: more detail, Args, Returns, Raises sections if applicable.
- **Comments:** Use comments to explain *why* something is done, not *what* is being done (if the code is self-explanatory). Avoid obvious comments.

### Imports
- Order imports: standard library, then third-party, then local application imports, each group alphabetized.
- Use absolute imports for local modules where possible (e.g., `from kg_services.ontology import MCPTool`).
- Avoid `from module import *`.

### General Best Practices
- **Readability:** Prioritize clear, readable code. Write code for humans first.
- **Simplicity (KISS):** Prefer simple solutions over complex ones unless absolutely necessary.
- **Modularity:** Keep functions and classes small and focused on a single responsibility (SRP).
- **Error Handling:** Use `try-except` blocks for operations that might fail (I/O, API calls). Be specific about exceptions caught. Log errors appropriately.
- **Constants:** Define constants at the module level using `UPPER_SNAKE_CASE`.
- **Environment Variables:** Access secrets and configurations (like API keys) ONLY through environment variables (e.g., `os.getenv()`). Use `.env.example` and `python-dotenv` for local development. Never commit `.env` files.
- **Avoid Magic Numbers/Strings:** Define them as constants.
- **Logging:** Prefer the `logging` module over `print()` for application diagnostics, especially in library/service code. `print()` is acceptable for CLI tools or immediate debug output during development.
```

**2. `data_handling.mdc`**

```markdown
---
description: Rules for data structures, JSON handling, and in-memory Knowledge Graph components.
globs: ["kg_services/**/*.py", "data/**/*.json", "schemas/**/*.json", "app.py"] # Apply to relevant files
alwaysApply: false # Auto-attach when relevant files are in context
---

### Dataclasses (for Ontology Primitives like `MCPTool`, `MCPPrompt`, `PlannedStep`)
- Utilize Python `dataclasses` for defining structured data objects (MCP primitives).
- All fields MUST have type hints.
- Use `field(default_factory=list/dict)` for mutable default values.
- Ensure dataclasses are easily serializable/deserializable to/from JSON if needed.
- Add a `to_dict()` method if frequently converting to dictionaries for APIs or storage.
- Add a `from_dict(cls, data: Dict[str, Any])` class method for robust instantiation from dictionaries.

### JSON Data (`data/initial_*.json`)
- Ensure all JSON files are well-formed and valid.
- Data in JSON files MUST strictly adhere to the schemas defined by corresponding Python dataclasses or JSON Schemas.
- Use consistent naming conventions for keys.
- For descriptive fields (like `description`, `tags`), provide rich, detailed content to improve embedding quality for semantic search.

### In-Memory KG (`kg_services/knowledge_graph.py`)
- **Structure:** Prefer Python dictionaries for key-based lookups (e.g., `self.tools: Dict[str, MCPTool]`) and lists for storing sequences like embeddings.
- **Loading:** When loading from JSON, include robust error handling for file issues (`FileNotFoundError`) or parsing errors (`json.JSONDecodeError`). Log errors clearly.
- **Vector Storage (InMemory):**
    - Store embeddings as `List[List[float]]` or `List[np.ndarray]`.
    - Maintain a parallel list of IDs (`tool_ids_for_vectors`, `prompt_ids_for_vectors`) to map embeddings back to their original objects. Ensure these lists are always synchronized.
- **Similarity Calculation:**
    - Use `numpy` for efficient vector operations (cosine similarity).
    - Implement cosine similarity carefully, handling potential division by zero (e.g., if a vector norm is zero).
- **Modularity:** Separate structured data storage (dictionaries) from vector index logic within the `InMemoryKG` class.

### NumPy Usage
- When performing vector math (e.g., cosine similarity), convert Python lists of floats to `np.ndarray` for performance and correctness.
- Ensure `numpy` is listed as a core dependency in `requirements.txt`.
```

**3. `llm_interaction.mdc`**

```markdown
---
description: Best practices for interacting with LLM APIs (Embeddings, future Chat/Sampling).
globs: ["kg_services/embedder.py", "agents/**/*.py"] # Where LLM calls happen
alwaysApply: false
---

### Embedding Service (`kg_services/embedder.py`)
- **Client Initialization:** Initialize LLM API clients (OpenAI, Anthropic, AzureOpenAI) once, typically in `__init__`.
- **API Key Management:** Strictly load API keys from environment variables (`os.getenv()`). NEVER hardcode keys.
- **Error Handling:**
    - Wrap all API calls in `try-except` blocks.
    - Catch specific API errors (e.g., `openai.APIError`, `requests.exceptions.RequestException`).
    - Log errors with context (e.g., the text being embedded, the API called).
    - Return a clear failure indicator (e.g., `None` or raise a custom exception) so calling code can handle it.
- **Input Preprocessing:** Preprocess text for embeddings if recommended by the API provider (e.g., replacing newlines with spaces: `text.replace("\n", " ")`).
- **Model Selection:** Clearly define which embedding model is being used (e.g., `text-embedding-3-small`). Make this configurable if possible (e.g., via an environment variable or parameter).
- **Batching (Future Consideration):** If embedding many texts, investigate if the API supports batching for efficiency.

### (Future) MCP Sampling / Direct LLM Calls by Agents
- **Request Construction:** When an agent constructs a request for an LLM (simulating MCP Sampling or direct call):
    - Adhere strictly to the target LLM's API schema (e.g., message roles, content structure).
    - Sanitize any user-provided input included in prompts to prevent prompt injection if applicable.
- **Response Parsing:** Robustly parse LLM responses. Anticipate variations or errors. Validate JSON structure if expecting JSON.
- **Retry Logic:** Implement retry mechanisms (e.g., with exponential backoff) for transient API errors.
- **Cost & Token Tracking (Advanced):** If making many calls, consider logging token usage to monitor costs.
```

**4. `gradio_ui.mdc`**

```markdown
---
description: Rules and best practices for developing Gradio UIs for KGraph-MCP.
globs: ["app.py", "mcp_summarizer_tool_gradio/app.py", "mcp_sentiment_tool_gradio/app.py"] # All Gradio apps
alwaysApply: false
---

### Interface Design (`gr.Blocks` preferred)
- Use `gr.Blocks()` for more control over layout compared to `gr.Interface` for complex apps.
- Organize UI elements logically using `gr.Row()`, `gr.Column()`, `gr.Group()`, `gr.Accordion()`, `gr.Tabs()`.
- Provide clear labels and placeholders for all input components.
- Use `gr.Markdown()` for titles, descriptions, and instructional text.
- Ensure a responsive layout if possible.

### Backend Handler Functions
- Keep Gradio handler functions (those called by `button.click()`, etc.) focused. They should primarily:
    1.  Receive inputs from Gradio components.
    2.  Call underlying business logic (e.g., Planner agent, Executor agent).
    3.  Format the results from the business logic for display in Gradio output components.
    4.  Return updates for Gradio components using `gr.update()` or direct values.
- Handle errors gracefully within these functions and return user-friendly error messages to the UI.
- **State Management:** For simple state needed across interactions (like the current `PlannedStep`), consider passing necessary identifiers back from the UI or using `gr.State` if absolutely necessary (but try to keep handlers stateless if possible by re-deriving context).

### Dynamic UI Updates
- When dynamically showing/hiding components or updating their properties (label, value, visibility):
    - Use `gr.update()` (e.g., `gr.Textbox.update(visible=True, label="New Label")`).
    - Ensure the `outputs` list of a `.click()` handler correctly maps to all components being updated.
    - If updating many components, return a tuple of `gr.update()` objects in the correct order.
- Test dynamic updates thoroughly across different scenarios.

### MCP Server Exposure (for Track 1 Tools)
- Use `gradio_mcp.patch(iface, mcp_path="/mcp")` (or the latest equivalent API from `gradio_mcp`) to expose Gradio interfaces as MCP servers.
- Clearly document the MCP endpoint path (`/mcp`), expected request payload structure (`{"data": [...]}`), and response structure in the tool's `README.md`.
- Ensure the Gradio function's input parameters match the order and types expected in the MCP `data` array.

### Error Handling & User Feedback
- Display clear error messages in the UI if backend operations fail.
- Provide loading indicators or button state changes (`gr.Button.update(interactive=False, value="Processing...")`) for long-running operations (though less critical for fast, local MVP operations).
```

**5. `agent_logic.mdc`**

```markdown
---
description: Design principles and conventions for KGraph-MCP AI Agents (Planner, Executor, etc.).
globs: ["agents/**/*.py"]
alwaysApply: false
---

### Agent Responsibilities
- Each agent class (Planner, Executor, Supervisor) should have a clearly defined, single responsibility.
- Interactions between agents should be through well-defined method calls or (future) message passing.

### Planner Agent (`agents/planner.py`)
- **Input:** User query (string).
- **Core Logic:**
    1.  Utilize `EmbeddingService` to embed the user query.
    2.  Query `InMemoryKG` (both vector and structured parts) to find relevant `MCPTool`s.
    3.  For selected tools, query `InMemoryKG` for relevant `MCPPrompt`s (considering `target_tool_id` and semantic relevance to the query).
    4.  Construct `PlannedStep` objects.
- **Output:** `List[PlannedStep]`.
- **Error Handling:** Gracefully handle cases where no tools/prompts are found, or embedding fails.

### Executor Agent (`agents/executor.py`)
- **Input:** `PlannedStep` object, `Dict[str, str]` of user-provided inputs for the prompt.
- **Core Logic (MVP3 - Simulated):**
    - Return mocked, tool-specific, and input-aware responses.
    - Simulate success and error states.
- **Core Logic (MVP4 - Real MCP Call):**
    - Check `tool.execution_type`.
    - If "remote_mcp_gradio":
        - Construct valid MCP JSON payload (order `inputs` based on `prompt.input_variables`).
        - Make HTTP POST request using `requests` to `tool.invocation_command_stub` (MCP endpoint URL).
        - Parse MCP JSON response. Extract data from `response["data"][0]`.
        - Handle network errors, non-200 HTTP status codes, and MCP response parsing errors robustly.
    - Fallback to simulation for other `execution_type`s.
- **Output:** `Dict[str, Any]` containing status, message, and tool-specific output.

### State & Dependencies
- Agents should receive their dependencies (like `InMemoryKG`, `EmbeddingService`) via their constructor (Dependency Injection).
- Avoid global state where possible.

### Logging
- Implement basic logging within agent methods to trace decision-making and execution flow, especially for debugging.
```

**6. `testing_pytest.mdc`**

```markdown
---
description: Conventions for writing tests using Pytest and unittest.mock.
globs: ["tests/**/*.py"]
alwaysApply: true # Tests are crucial
---

### General Principles
- **Test Coverage:** Aim for high test coverage for all non-UI, non-direct-LLM-output backend logic.
- **AAA Pattern (Arrange, Act, Assert):** Structure tests clearly.
- **Independence:** Tests should be independent and runnable in any order. Avoid tests that depend on the state left by previous tests.
- **Speed:** Unit tests should be fast. Mock external dependencies (APIs, databases, file system) to achieve this.

### Pytest Usage
- Use descriptive test function names (e.g., `test_planner_returns_empty_list_for_no_matching_tools`).
- Utilize Pytest fixtures (`@pytest.fixture`) for setting up common test data or mock objects (e.g., an initialized `InMemoryKG` instance, a mocked `EmbeddingService`).
- Use `tmp_path` fixture for tests involving file I/O.
- Use `capsys` fixture for testing `print()` statements or logged output.
- Use `monkeypatch` fixture for setting/deleting environment variables or patching module attributes.

### Mocking (`unittest.mock`)
- **Target for Patching:** Patch objects where they are *looked up*, not where they are *defined*. (e.g., `patch('app.planner_agent_instance')` if testing a Gradio handler in `app.py` that uses it).
- Use `mocker` fixture (from `pytest-mock`) for convenience.
- **Specificity:** Mock return values or side effects precisely for what the test needs.
- Test that mocked methods were called with expected arguments (`mock_object.assert_called_once_with(...)`).
- Mock external API calls (e.g., `requests.post`, `openai.Embeddings.create`) thoroughly in tests for services that use them (like `EmbeddingService` or `McpExecutorAgent`).

### Test Structure
- Organize tests in the `tests/` directory, mirroring the project structure (e.g., `tests/kg_services/test_knowledge_graph.py`, `tests/agents/test_planner.py`).
- Ensure `__init__.py` files are present in test subdirectories if needed for discovery.
```

**7. `mcp_specific/gradio_mcp_server.mdc`**

```markdown
---
description: Rules specific to creating MCP Servers using Gradio and gradio_mcp.
globs: ["mcp_summarizer_tool_gradio/app.py", "mcp_sentiment_tool_gradio/app.py"] # Apply to Track 1 tools
alwaysApply: false
---

### MCP Endpoint
- Consistently use `/mcp` as the `mcp_path` when patching the Gradio interface with `gradio_mcp.patch()`.
- Ensure the Gradio function (`fn`) that `gr.Interface` wraps is designed to accept inputs in the order they will appear in the MCP `{"data": [...]}` list.
- The function should return a single value or a tuple, which `gradio_mcp` will wrap into `{"data": [...]}`.

### Payload Handling
- The Gradio function should anticipate the types and number of arguments from the MCP `data` list.
- Perform basic validation on inputs if necessary within the function.
- Return clear error messages (as strings or structured dicts) if input is invalid, which will be part of the MCP response's `data` field.

### README Documentation (for each MCP Server Space)
- **Crucial:** The `README.md` for each Track 1 MCP server Space MUST clearly document:
    - The MCP endpoint URL (Space URL + `/mcp`).
    - The expected HTTP method (`POST`).
    - The exact structure of the request JSON payload, especially the `data` array (order and types of arguments).
    - The exact structure of a successful response JSON payload.
    - Example error response structures.
- This documentation is vital for your main KGraph-MCP agent (and others) to correctly call this MCP server.

### Dependency Management
- Keep `requirements.txt` for each MCP server minimal, including only `gradio`, `gradio_mcp`, and libraries essential for that specific tool's function.
```

**8. `project_specific.mdc`**

```markdown
---
description: Overarching conventions, terminology, and architectural principles for the KGraph-MCP project.
globs: ["**/*.py", "**/*.md"] # Apply broadly
alwaysApply: true
---

### Terminology
- Consistently use the defined MCP primitive names: `MCPTool`, `MCPPrompt`, `PlannedStep`, `Resource` (when added), `Root` (when added), `Sampling` (when added).
- Refer to the in-memory knowledge graph components as `InMemoryKG` and `EmbeddingService`.
- Agent names: `SimplePlannerAgent`, `McpExecutorAgent` (or `StubExecutorAgent`).

### Modularity & Separation of Concerns
- **`kg_services/`:** All direct data storage, retrieval, embedding, and KG structure logic. Should not contain agent decision-making or UI code.
- **`agents/`:** Contains the logic for different AI agents. Agents interact with `kg_services` but not directly with Gradio.
- **`app.py`:** Handles Gradio UI and acts as the top-level orchestrator, calling agent methods and formatting results. Should contain minimal business logic itself.
- **`data/`:** Contains static JSON definitions for initial KG population.
- **`schemas/`:** (Future) JSON Schema definitions for validating data, if used beyond dataclasses.

### Conventional Commits
- All Git commit messages MUST follow the Conventional Commits specification (v1.0.0).
- Use types like `feat`, `fix`, `docs`, `style`, `refactor`, `test`, `chore`, `ci`, `build`.
- Use scopes for clarity (e.g., `feat(planner): ...`, `fix(ui): ...`).
- This will be enforced by `commitlint` if set up.

### Error Handling Philosophy
- Services (like `EmbeddingService`, `McpExecutorAgent` making API calls) should catch specific exceptions and return `None` or a structured error response.
- Agents should check for these failure indicators from services and propagate errors or make decisions based on them.
- The Gradio `app.py` handler functions should ultimately present user-friendly error messages.

### Hackathon MVP Focus
- Prioritize implementing the core functionality defined for the current MVP sprint.
- Favor simplicity and clarity over overly complex solutions, especially for UI and initial agent logic.
- Ensure the demo for each MVP is compelling and clearly shows the value of the KG and agent interaction.
- Document sponsor technology usage clearly in `README.md`.
```

**How to Use These Rules in Cursor IDE:**

1.  **Create the Directory Structure:** Make the `.cursor/rules/` directory and the sub-directory `mcp_specific/` in your project root.
2.  **Save Each Rule Set:** Save the content above into the corresponding `.mdc` files within this structure.
3.  **Cursor Auto-Attachment:**
    *   Rules with `alwaysApply: true` (like `general_python.mdc` and `project_specific.mdc`) will generally be active.
    *   Rules with `alwaysApply: false` and specific `globs` (like `data_handling.mdc` or `gradio_ui.mdc`) will be automatically included by Cursor when you are editing files matching those globs or when you `@mention` files that match those globs.
4.  **Explicitly Referencing Rules:** When prompting Claude, you can reinforce specific rules:
    *   "Implement this function for `InMemoryKG`. Apply rules from `@.cursor/rules/data_handling.mdc` and `@.cursor/rules/general_python.mdc`."
5.  **Iterate on Rules:** As the project progresses, you might find the need to refine these rules, add new ones, or adjust scopes. This is a living document.

By setting up this comprehensive rule set, you provide Claude 4.0 with excellent context and guidance, leading to more consistent, higher-quality code generation and a more efficient AI-assisted development experience throughout your hackathon project.