# MVP 2 Comprehensive Plan: "KG Suggests Actionable Tool with Prompt Template" **Plan Created:** 2025-06-08 **Builds Upon:** MVP 1 (Successfully Completed) **Target Duration:** 5 Sprints (Post-Hackathon Development) **Overall Goal:** Extend KGraph-MCP to suggest both relevant MCP tools AND corresponding prompt templates needed to invoke them effectively --- ## 🎯 MVP 2 Vision & Objectives ### **Strategic Goal** Transform the KGraph-MCP from a tool discovery system into an actionable task planning system that provides both: 1. **Relevant MCP Tools** (carried over from MVP 1) 2. **Prompt Templates** with input variables for effective tool usage ### **Core Value Proposition** - **From Discovery to Action:** Users get not just "what tool to use" but "how to use it effectively" - **Prompt Engineering Automation:** System suggests optimized prompts for different use cases - **Template-Driven Execution:** Structured approach to tool invocation with clear input requirements ### **Target User Experience** ``` User: "I need to analyze customer feedback sentiment" System Response: ┌─ Tool: Sentiment Analyzer │ Description: Analyzes text for emotional tone and sentiment └─ Prompt: "Customer Feedback Analysis Template" Description: Optimized for business customer feedback analysis Template: "Analyze the sentiment of this customer feedback and provide a business summary: {{customer_feedback}}" Required Inputs: [customer_feedback] ``` --- ## 🏗️ Technical Architecture Evolution ### **MVP 1 Foundation (Completed)** ``` User Query → Embedding → Tool Discovery → Tool Suggestion ``` ### **MVP 2 Target Architecture** ``` User Query → Embedding → Tool Discovery → Prompt Selection → (Tool + Prompt) Suggestion ↗ ↗ Tool Embeddings Prompt Embeddings ``` ### **New Components in MVP 2** 1. **MCPPrompt Ontology** - Structured prompt representation 2. **Prompt Knowledge Graph** - Semantic prompt storage and retrieval 3. **Enhanced Planner Agent** - Tool + Prompt selection logic 4. **Rich UI Display** - Template visualization and input variable guidance --- ## 📋 Sprint Overview | Sprint | Focus Area | Duration | Key Deliverables | |--------|------------|----------|------------------| | **Sprint 1** | Prompt Ontology & KG Enhancement | 3-4 hours | MCPPrompt dataclass, prompt loading, vector indexing | | **Sprint 2** | Enhanced Planning Logic | 2-3 hours | Tool+Prompt selection, PlannedStep structure | | **Sprint 3** | UI Enhancement | 2-3 hours | Rich prompt display, template visualization | | **Sprint 4** | Dynamic Input Display | 1-2 hours | Input variable visualization, UI polish | | **Sprint 5** | Testing & Documentation | 2-3 hours | End-to-end testing, README updates, CI validation | **Total Estimated Time:** 10-15 hours --- ## 🚀 Sprint 1: Define Prompt Ontology & Enhance KG for Prompts **Duration:** 3-4 hours **Priority:** HIGH (Foundation for all subsequent work) **Dependencies:** MVP 1 completion ### **Sprint 1 Objectives** - Define comprehensive `MCPPrompt` data structure - Create rich initial prompt metadata - Extend `InMemoryKG` for prompt management - Implement prompt vector indexing - Update application initialization ### **Sprint 1 Tasks** #### **Task 1.1: Define `MCPPrompt` Ontology (Dataclass)** - **Status:** Todo - **Priority:** HIGH - **Estimated Time:** 30-45 minutes - **Dependencies:** None - **Description:** Create comprehensive `MCPPrompt` dataclass in `kg_services/ontology.py` - **Acceptance Criteria:** 1. `MCPPrompt` dataclass with all required fields and type hints 2. Proper default values and validation 3. Compatible with JSON serialization 4. Unit tests passing **Implementation Specification:** ```python @dataclass class MCPPrompt: """Represents a prompt template for MCP tool usage.""" prompt_id: str name: str description: str tags: List[str] = field(default_factory=list) target_tool_id: str # Links to MCPTool template_string: str # Template with {{variable}} placeholders input_variables: List[str] = field(default_factory=list) use_case: str = "" # Optional: specific use case description difficulty_level: str = "beginner" # beginner, intermediate, advanced example_inputs: Dict[str, str] = field(default_factory=dict) # Example variable values ``` **Test Requirements:** - `test_mcp_prompt_creation` - Basic instantiation - `test_mcp_prompt_validation` - Field validation - `test_mcp_prompt_serialization` - JSON compatibility #### **Task 1.2: Create Rich Initial Prompt Metadata** - **Status:** Todo - **Priority:** HIGH - **Estimated Time:** 45-60 minutes - **Dependencies:** Task 1.1 - **Description:** Create comprehensive `data/initial_prompts.json` with diverse, high-quality prompts - **Acceptance Criteria:** 1. 8-12 diverse prompt templates 2. Coverage of all tools from MVP 1 3. Multiple prompt styles per tool (concise, detailed, specific use cases) 4. Rich descriptions suitable for semantic embedding **Content Requirements:** ```json [ { "prompt_id": "sentiment_customer_feedback_v1", "name": "Customer Feedback Sentiment Analysis", "description": "Analyzes customer feedback for business insights, focusing on actionable sentiment patterns and key concerns", "tags": ["sentiment", "customer", "business", "feedback", "analysis"], "target_tool_id": "sentiment_analyzer_v1", "template_string": "Analyze the sentiment of this customer feedback and provide a business summary:\n\nFeedback: {{customer_feedback}}\n\nPlease provide:\n1. Overall sentiment (positive/negative/neutral)\n2. Key emotional indicators\n3. Actionable business insights", "input_variables": ["customer_feedback"], "use_case": "Business customer feedback analysis", "difficulty_level": "beginner", "example_inputs": { "customer_feedback": "The product arrived late and the packaging was damaged, but the customer service team was very helpful in resolving the issue quickly." } } ] ``` **Quality Standards:** - Each prompt should be semantically distinct - Templates should be production-ready - Descriptions optimized for embedding quality - Multiple difficulty levels represented #### **Task 1.3: Extend `InMemoryKG` for Prompt Storage** - **Status:** Todo - **Priority:** HIGH - **Estimated Time:** 45-60 minutes - **Dependencies:** Task 1.2 - **Description:** Add prompt management capabilities to `InMemoryKG` - **Acceptance Criteria:** 1. Prompt loading from JSON 2. Prompt retrieval methods 3. Error handling for malformed data 4. Unit tests passing **Implementation Requirements:** ```python class InMemoryKG: def __init__(self): # ... existing code ... self.prompts: Dict[str, MCPPrompt] = {} def load_prompts_from_json(self, filepath: str) -> None: """Load MCPPrompt objects from JSON file.""" # Implementation with error handling def get_prompt_by_id(self, prompt_id: str) -> Optional[MCPPrompt]: """Retrieve prompt by ID.""" def get_prompts_for_tool(self, tool_id: str) -> List[MCPPrompt]: """Get all prompts targeting a specific tool.""" ``` **Test Coverage:** - Valid JSON loading - Invalid JSON handling - Prompt retrieval (exists/not exists) - Tool-specific prompt filtering #### **Task 1.4: Implement Prompt Vector Indexing** - **Status:** Todo - **Priority:** HIGH - **Estimated Time:** 60-75 minutes - **Dependencies:** Task 1.3 - **Description:** Extend vector indexing to include prompt embeddings - **Acceptance Criteria:** 1. Prompt embeddings generation and storage 2. Semantic prompt search functionality 3. Integration with existing vector search 4. Performance optimization **Implementation Approach:** ```python class InMemoryKG: def __init__(self): # ... existing code ... self.prompt_embeddings: List[List[float]] = [] self.prompt_ids_for_vectors: List[str] = [] def build_vector_index(self, embedder: EmbeddingService) -> None: """Build vector index for both tools and prompts.""" # ... existing tool indexing ... # Prompt indexing for prompt_id, prompt in self.prompts.items(): prompt_text = self._create_prompt_embedding_text(prompt) embedding = embedder.get_embedding(prompt_text) if embedding: self.prompt_embeddings.append(embedding) self.prompt_ids_for_vectors.append(prompt_id) def find_similar_prompts(self, query_embedding: List[float], top_k: int = 3) -> List[str]: """Find prompts similar to query using cosine similarity.""" # Similar logic to find_similar_tools def _create_prompt_embedding_text(self, prompt: MCPPrompt) -> str: """Create descriptive text for prompt embedding.""" return f"{prompt.name} - {prompt.description} - Use case: {prompt.use_case} - Tags: {', '.join(prompt.tags)}" ``` #### **Task 1.5: Update Application Initialization** - **Status:** Todo - **Priority:** MEDIUM - **Estimated Time:** 15-20 minutes - **Dependencies:** Task 1.4 - **Description:** Update `app.py` to load prompts during startup - **Acceptance Criteria:** 1. Prompts loaded during application startup 2. Vector index includes prompt embeddings 3. Error handling for missing prompt files 4. Startup logging for prompt loading ### **Sprint 1 Success Criteria** - [x] MCPPrompt ontology defined and tested - [x] Rich prompt metadata created (8-12 prompts) - [x] InMemoryKG enhanced for prompt management - [x] Vector indexing includes prompt embeddings - [x] Application startup includes prompt loading - [x] All unit tests passing - [x] No regression in MVP 1 functionality --- ## 🚀 Sprint 2: Enhance Planner to Suggest Tool+Prompt Pairs **Duration:** 2-3 hours **Priority:** HIGH **Dependencies:** Sprint 1 completion ### **Sprint 2 Objectives** - Create `PlannedStep` data structure for (Tool, Prompt) pairs - Implement intelligent prompt selection logic - Enhance `SimplePlannerAgent` for combined suggestions - Maintain backward compatibility with MVP 1 ### **Sprint 2 Tasks** #### **Task 2.1: Define `PlannedStep` Data Structure** - **Status:** Todo - **Priority:** HIGH - **Estimated Time:** 20-30 minutes - **Dependencies:** Task 1.1 - **Description:** Create structured representation for tool+prompt suggestions - **Acceptance Criteria:** 1. `PlannedStep` dataclass properly defined 2. Serialization compatibility 3. Optional confidence scoring 4. Unit tests passing **Implementation Specification:** ```python @dataclass class PlannedStep: """Represents a suggested tool with its optimal prompt.""" tool: MCPTool prompt: MCPPrompt confidence_score: float = 0.0 # Combined tool+prompt confidence reasoning: str = "" # Optional: why this combination was selected def to_dict(self) -> Dict[str, Any]: """Convert to dictionary for JSON serialization.""" return { "tool": { "id": self.tool.tool_id, "name": self.tool.name, "description": self.tool.description, "tags": self.tool.tags }, "prompt": { "id": self.prompt.prompt_id, "name": self.prompt.name, "description": self.prompt.description, "template": self.prompt.template_string, "input_variables": self.prompt.input_variables, "use_case": self.prompt.use_case }, "confidence_score": self.confidence_score, "reasoning": self.reasoning } ``` #### **Task 2.2: Implement Intelligent Prompt Selection** - **Status:** Todo - **Priority:** HIGH - **Estimated Time:** 90-120 minutes - **Dependencies:** Task 2.1 - **Description:** Create sophisticated prompt selection algorithm - **Acceptance Criteria:** 1. Multi-stage selection process (tool → prompt) 2. Semantic ranking of prompts for selected tool 3. Confidence scoring for combinations 4. Fallback strategies for edge cases **Algorithm Design:** ```python def select_best_prompt_for_tool(self, tool: MCPTool, user_query: str, query_embedding: List[float]) -> Optional[MCPPrompt]: """Select the most appropriate prompt for a tool given user context.""" # 1. Get all prompts for this tool candidate_prompts = self.kg.get_prompts_for_tool(tool.tool_id) if not candidate_prompts: return None # 2. If only one prompt, return it if len(candidate_prompts) == 1: return candidate_prompts[0] # 3. Multi-criteria selection for multiple prompts scored_prompts = [] for prompt in candidate_prompts: score = self._calculate_prompt_relevance_score(prompt, user_query, query_embedding, tool) scored_prompts.append((score, prompt)) # 4. Return highest scoring prompt scored_prompts.sort(key=lambda x: x[0], reverse=True) return scored_prompts[0][1] def _calculate_prompt_relevance_score(self, prompt: MCPPrompt, user_query: str, query_embedding: List[float], tool: MCPTool) -> float: """Calculate relevance score for prompt given user context.""" score = 0.0 # Semantic similarity to user query (60% weight) prompt_embedding = self._get_prompt_embedding(prompt.prompt_id) if prompt_embedding: semantic_similarity = self.kg._cosine_similarity(query_embedding, prompt_embedding) score += semantic_similarity * 0.6 # Use case matching (20% weight) if any(keyword in user_query.lower() for keyword in prompt.use_case.lower().split()): score += 0.2 # Tag matching (15% weight) query_words = set(user_query.lower().split()) prompt_tags = set(tag.lower() for tag in prompt.tags) tag_overlap = len(query_words.intersection(prompt_tags)) / max(len(prompt_tags), 1) score += tag_overlap * 0.15 # Difficulty preference (5% weight) - prefer beginner for unclear queries if len(user_query.split()) < 10 and prompt.difficulty_level == "beginner": score += 0.05 return score ``` #### **Task 2.3: Enhance SimplePlannerAgent** - **Status:** Todo - **Priority:** HIGH - **Estimated Time:** 60-75 minutes - **Dependencies:** Task 2.2 - **Description:** Implement comprehensive planning with tool+prompt suggestions - **Acceptance Criteria:** 1. New `plan_task_with_prompt` method 2. Backward compatibility with existing `suggest_tools` 3. Comprehensive error handling 4. Performance optimization **Enhanced Agent Implementation:** ```python class SimplePlannerAgent: def plan_task_with_prompt(self, user_query: str, top_k_suggestions: int = 3) -> List[PlannedStep]: """Plan task by suggesting both tools and prompts.""" if not user_query or not user_query.strip(): return [] query_embedding = self.embedder.get_embedding(user_query) if query_embedding is None: return [] # 1. Find relevant tools similar_tool_ids = self.kg.find_similar_tools(query_embedding, top_k=top_k_suggestions * 2) # Get more for filtering planned_steps: List[PlannedStep] = [] for tool_id in similar_tool_ids: tool = self.kg.get_tool_by_id(tool_id) if not tool: continue # 2. Find best prompt for this tool best_prompt = self.select_best_prompt_for_tool(tool, user_query, query_embedding) if not best_prompt: continue # Skip tools without prompts for MVP 2 # 3. Calculate combined confidence tool_confidence = self._calculate_tool_confidence(tool, query_embedding) prompt_confidence = self._calculate_prompt_confidence(best_prompt, user_query, query_embedding) combined_confidence = (tool_confidence + prompt_confidence) / 2 # 4. Create planned step step = PlannedStep( tool=tool, prompt=best_prompt, confidence_score=combined_confidence, reasoning=f"Selected {tool.name} with {best_prompt.name} based on semantic similarity and use case matching" ) planned_steps.append(step) if len(planned_steps) >= top_k_suggestions: break # Sort by confidence and return planned_steps.sort(key=lambda x: x.confidence_score, reverse=True) return planned_steps # Maintain backward compatibility def suggest_tools(self, user_query: str, top_k: int = 3) -> List[MCPTool]: """Legacy method for backward compatibility.""" planned_steps = self.plan_task_with_prompt(user_query, top_k) return [step.tool for step in planned_steps] ``` ### **Sprint 2 Success Criteria** - [x] PlannedStep data structure implemented - [x] Intelligent prompt selection algorithm working - [x] Enhanced SimplePlannerAgent operational - [x] Backward compatibility maintained - [x] Comprehensive unit tests passing - [x] Performance benchmarks meet targets (<500ms response time) --- ## 🚀 Sprint 3: Update Gradio UI for Rich Prompt Display **Duration:** 2-3 hours **Priority:** HIGH **Dependencies:** Sprint 2 completion ### **Sprint 3 Objectives** - Redesign UI for tool+prompt display - Implement rich prompt template visualization - Add input variable guidance - Maintain intuitive user experience ### **Sprint 3 Tasks** #### **Task 3.1: Redesign UI Layout for Rich Content** - **Status:** Todo - **Priority:** HIGH - **Estimated Time:** 45-60 minutes - **Dependencies:** Task 2.3 - **Description:** Enhance Gradio interface for comprehensive tool+prompt display - **Acceptance Criteria:** 1. Clear separation of tool and prompt information 2. Expandable/collapsible sections for detailed views 3. Professional visual hierarchy 4. Mobile-responsive design **UI Design Specification:** ```python with gr.Blocks(theme=gr.themes.Soft(), title="KGraph-MCP: AI-Powered Tool & Prompt Discovery") as app: gr.Markdown("# 🧠🛠️ KGraph-MCP: Intelligent Tool & Prompt Suggestions") gr.Markdown("Discover relevant MCP tools and get optimized prompt templates for your tasks.") with gr.Row(): with gr.Column(scale=2): query_input = gr.Textbox( label="Describe your task", placeholder="e.g., 'I need to analyze customer feedback sentiment'", lines=3 ) with gr.Row(): find_button = gr.Button("Find Tools & Prompts", variant="primary") clear_button = gr.Button("Clear", variant="secondary") with gr.Column(scale=1): gr.Markdown("### Example Queries:") gr.Markdown(""" - "I need to analyze text sentiment" - "Help me summarize a long document" - "Generate captions for my images" - "Check my code for quality issues" """) # Results section results_section = gr.Group(visible=False) with results_section: gr.Markdown("## 🎯 Suggested Tools & Prompts") # Tool+Prompt cards suggestions_accordion = gr.Accordion("Suggestions", open=True) with suggestions_accordion: suggestions_display = gr.HTML() # Error/info display status_display = gr.Markdown(visible=False) ``` #### **Task 3.2: Implement Rich Content Formatting** - **Status:** Todo - **Priority:** HIGH - **Estimated Time:** 60-75 minutes - **Dependencies:** Task 3.1 - **Description:** Create beautiful HTML formatting for tool+prompt display - **Acceptance Criteria:** 1. Professional card-based layout 2. Syntax highlighting for prompt templates 3. Clear visual hierarchy 4. Interactive elements (copy buttons, expand/collapse) **Formatting Implementation:** ```python def format_planned_steps_as_html(planned_steps: List[PlannedStep]) -> str: """Create rich HTML display for planned steps.""" if not planned_steps: return "
No suggestions found. Try a different query.
" html_parts = [] for i, step in enumerate(planned_steps, 1): confidence_bar = create_confidence_bar(step.confidence_score) card_html = f"""Tool: {step.tool.description}
Tags: {', '.join(step.tool.tags)}
{step.prompt.description}
{escape_html(step.prompt.template_string)}
{var}: {example}