[Search] Unified Search UI with Autocomplete

[Search] Unified Search UI with Autocomplete

Goal

Add a unified search experience across all SciDEX content, accessible from every page via the navigation bar. Real-time autocomplete with type-grouped results (wiki pages, hypotheses, entities, papers, analyses, gaps).

Requirements

Functional

• Global search bar: Present on every SciDEX page (via nav_html)
• Real-time autocomplete: Results appear as user types (debounced 200ms)
• Type-grouped results: Wiki, Hypotheses, Graph Nodes, Papers, Analyses, Gaps
• Result display: Title, type badge, relevance score, snippet (50 chars)
• Keyboard navigation: Up/down arrows, Enter to select, Esc to close
• Click navigation: Navigate to appropriate detail page
• Performance: <200ms API response time
• Mobile responsive: Collapsible search on small screens

Technical

• Backend: /api/search?q={query}&limit={n} endpoint
• Frontend: Vanilla JS + CSS (no framework dependencies)
• Integration: Modify nav_html() function in api.py
• Search scope:

- wiki_entities (entity_name, summary)
- hypotheses (title, description)
- knowledge_edges (source_id, target_id, relation)
- papers (title, authors)
- analyses (title, question)
- gaps (title, description)

API Design

Endpoint: /api/search

Query Parameters:

• q (required): Search query string
• limit (optional): Max results per type (default: 5)

Response Format:

{
  "query": "APOE",
  "total_results": 23,
  "results_by_type": {
    "wiki": [
      {
        "type": "wiki",
        "id": "APOE",
        "title": "APOE",
        "snippet": "Apolipoprotein E is a protein...",
        "url": "/wiki/APOE",
        "relevance": 0.95
      }
    ],
    "hypotheses": [
      {
        "type": "hypothesis",
        "id": "hyp-123",
        "title": "APOE4 increases AD risk via...",
        "snippet": "APOE4 allele carriers show increased...",
        "url": "/hypothesis/hyp-123",
        "relevance": 0.87
      }
    ],
    "entities": [...],
    "papers": [...],
    "analyses": [...],
    "gaps": [...]
  }
}

Implementation:

@app.get("/api/search")
def unified_search(q: str, limit: int = 5):
    """Unified search across all content types."""
    db = get_db()
    query_lower = q.lower()
    results = {"query": q, "total_results": 0, "results_by_type": {}}
    
    # Search wiki entities
    wiki_results = db.execute("""
        SELECT entity_name as title, summary as snippet, entity_type
        FROM wiki_entities
        WHERE LOWER(entity_name) LIKE ? OR LOWER(summary) LIKE ?
        ORDER BY 
            CASE WHEN LOWER(entity_name) = ? THEN 0
                 WHEN LOWER(entity_name) LIKE ? THEN 1
                 ELSE 2 END,
            entity_name
        LIMIT ?
    """, (f'%{query_lower}%', f'%{query_lower}%', query_lower, f'{query_lower}%', limit)).fetchall()
    
    results["results_by_type"]["wiki"] = [
        {
            "type": "wiki",
            "id": r['title'],
            "title": r['title'],
            "snippet": (r['snippet'] or '')[:80] + '...',
            "url": f"/wiki/{r['title']}",
            "relevance": calculate_relevance(r['title'], q)
        } for r in [dict(row) for row in wiki_results]
    ]
    
    # Search hypotheses
    hyp_results = db.execute("""
        SELECT id, title, description, composite_score
        FROM hypotheses
        WHERE LOWER(title) LIKE ? OR LOWER(description) LIKE ?
        ORDER BY composite_score DESC
        LIMIT ?
    """, (f'%{query_lower}%', f'%{query_lower}%', limit)).fetchall()
    
    results["results_by_type"]["hypotheses"] = [
        {
            "type": "hypothesis",
            "id": r['id'],
            "title": r['title'][:60] + '...' if len(r['title']) > 60 else r['title'],
            "snippet": (r['description'] or '')[:80] + '...',
            "url": f"/hypothesis/{r['id']}",
            "relevance": r['composite_score']
        } for r in [dict(row) for row in hyp_results]
    ]
    
    # Search KG entities
    entity_results = db.execute("""
        SELECT DISTINCT source_id as entity
        FROM knowledge_edges
        WHERE LOWER(source_id) LIKE ?
        UNION
        SELECT DISTINCT target_id as entity
        FROM knowledge_edges
        WHERE LOWER(target_id) LIKE ?
        LIMIT ?
    """, (f'%{query_lower}%', f'%{query_lower}%', limit)).fetchall()
    
    results["results_by_type"]["entities"] = [
        {
            "type": "entity",
            "id": r['entity'],
            "title": r['entity'],
            "snippet": "Knowledge graph entity",
            "url": f"/entity/{r['entity']}",
            "relevance": calculate_relevance(r['entity'], q)
        } for r in [dict(row) for row in entity_results]
    ]
    
    # Search papers
    paper_results = db.execute("""
        SELECT pmid, title, authors
        FROM papers
        WHERE LOWER(title) LIKE ? OR LOWER(authors) LIKE ?
        LIMIT ?
    """, (f'%{query_lower}%', f'%{query_lower}%', limit)).fetchall()
    
    results["results_by_type"]["papers"] = [
        {
            "type": "paper",
            "id": r['pmid'],
            "title": r['title'][:60] + '...' if len(r['title']) > 60 else r['title'],
            "snippet": f"Authors: {r['authors'][:50]}...",
            "url": f"https://pubmed.ncbi.nlm.nih.gov/{r['pmid']}",
            "relevance": 0.5
        } for r in [dict(row) for row in paper_results]
    ]
    
    # Count total
    results["total_results"] = sum(len(v) for v in results["results_by_type"].values())
    
    return results

def calculate_relevance(text: str, query: str) -> float:
    """Simple relevance scoring (0-1)."""
    text_lower = text.lower()
    query_lower = query.lower()
    if text_lower == query_lower:
        return 1.0
    elif text_lower.startswith(query_lower):
        return 0.9
    elif query_lower in text_lower:
        return 0.7
    else:
        return 0.5

Frontend Implementation

Search Component HTML

Add to nav_html() function in api.py:

def nav_html(active=""):
    """Navigation bar with integrated search."""
    search_html = """
    <div class="search-container">
        <input type="text" id="global-search" placeholder="Search SciDEX..." 
               autocomplete="off" aria-label="Search">
        <div id="search-results" class="search-results hidden"></div>
    </div>
    """
    
    # Insert search_html after nav links, before closing </nav>
    nav = f"""<nav style="display:flex;justify-content:space-between;align-items:center;...">
        <div class="nav-links">
            <a href="/">Dashboard</a>
            <!-- other links -->
        </div>
        {search_html}
    </nav>"""
    
    return nav + SEARCH_JS + SEARCH_CSS

Search JavaScript

const SEARCH_JS = """
<script>
(function() {
    const searchInput = document.getElementById('global-search');
    const resultsContainer = document.getElementById('search-results');
    let debounceTimer;
    let selectedIndex = -1;
    let currentResults = [];

    searchInput.addEventListener('input', (e) => {
        clearTimeout(debounceTimer);
        const query = e.target.value.trim();
        
        if (query.length < 2) {
            resultsContainer.classList.add('hidden');
            return;
        }
        
        debounceTimer = setTimeout(() => performSearch(query), 200);
    });

    async function performSearch(query) {
        try {
            const response = await fetch(`/api/search?q=${encodeURIComponent(query)}&limit=5`);
            const data = await response.json();
            displayResults(data);
        } catch (error) {
            console.error('Search failed:', error);
        }
    }

    function displayResults(data) {
        if (data.total_results === 0) {
            resultsContainer.innerHTML = '<div class="search-no-results">No results found</div>';
            resultsContainer.classList.remove('hidden');
            return;
        }

        let html = `<div class="search-summary">${data.total_results} results for "${data.query}"</div>`;
        
        const typeLabels = {
            'wiki': 'Wiki Pages',
            'hypotheses': 'Hypotheses',
            'entities': 'Graph Nodes',
            'papers': 'Papers',
            'analyses': 'Analyses',
            'gaps': 'Knowledge Gaps'
        };

        currentResults = [];
        
        for (const [type, results] of Object.entries(data.results_by_type)) {
            if (results.length === 0) continue;
            
            html += `<div class="search-group">
                <div class="search-group-title">${typeLabels[type] || type}</div>`;
            
            results.forEach((result, idx) => {
                const globalIdx = currentResults.length;
                currentResults.push(result);
                
                html += `<a href="${result.url}" class="search-result" data-index="${globalIdx}">
                    <div class="search-result-header">
                        <span class="search-result-title">${result.title}</span>
                        <span class="search-result-badge search-badge-${result.type}">${result.type}</span>
                    </div>
                    <div class="search-result-snippet">${result.snippet}</div>
                </a>`;
            });
            
            html += '</div>';
        }
        
        resultsContainer.innerHTML = html;
        resultsContainer.classList.remove('hidden');
        selectedIndex = -1;
    }

    // Keyboard navigation
    searchInput.addEventListener('keydown', (e) => {
        const resultElements = resultsContainer.querySelectorAll('.search-result');
        
        if (e.key === 'ArrowDown') {
            e.preventDefault();
            selectedIndex = Math.min(selectedIndex + 1, resultElements.length - 1);
            updateSelection(resultElements);
        } else if (e.key === 'ArrowUp') {
            e.preventDefault();
            selectedIndex = Math.max(selectedIndex - 1, -1);
            updateSelection(resultElements);
        } else if (e.key === 'Enter' && selectedIndex >= 0) {
            e.preventDefault();
            resultElements[selectedIndex].click();
        } else if (e.key === 'Escape') {
            resultsContainer.classList.add('hidden');
            searchInput.blur();
        }
    });

    function updateSelection(elements) {
        elements.forEach((el, idx) => {
            if (idx === selectedIndex) {
                el.classList.add('selected');
                el.scrollIntoView({block: 'nearest'});
            } else {
                el.classList.remove('selected');
            }
        });
    }

    // Close results when clicking outside
    document.addEventListener('click', (e) => {
        if (!e.target.closest('.search-container')) {
            resultsContainer.classList.add('hidden');
        }
    });
})();
</script>
"""

Search CSS

const SEARCH_CSS = """
<style>
.search-container {
    position: relative;
    width: 300px;
}

#global-search {
    width: 100%;
    padding: 0.6rem 1rem;
    background: rgba(255,255,255,0.08);
    border: 1px solid rgba(255,255,255,0.12);
    border-radius: 8px;
    color: #e0e0e0;
    font-size: 0.9rem;
    transition: all 0.2s;
}

#global-search:focus {
    outline: none;
    border-color: #4fc3f7;
    background: rgba(255,255,255,0.12);
}

.search-results {
    position: absolute;
    top: 100%;
    left: 0;
    right: 0;
    margin-top: 0.5rem;
    background: #1a1a2e;
    border: 1px solid rgba(79,195,247,0.3);
    border-radius: 8px;
    max-height: 500px;
    overflow-y: auto;
    box-shadow: 0 4px 12px rgba(0,0,0,0.5);
    z-index: 1000;
}

.search-results.hidden {
    display: none;
}

.search-summary {
    padding: 0.8rem 1rem;
    color: #888;
    font-size: 0.85rem;
    border-bottom: 1px solid rgba(255,255,255,0.08);
}

.search-group-title {
    padding: 0.6rem 1rem;
    color: #4fc3f7;
    font-size: 0.8rem;
    font-weight: 600;
    text-transform: uppercase;
    background: rgba(79,195,247,0.08);
    border-top: 1px solid rgba(255,255,255,0.05);
}

.search-result {
    display: block;
    padding: 0.8rem 1rem;
    color: #e0e0e0;
    text-decoration: none;
    border-bottom: 1px solid rgba(255,255,255,0.05);
    transition: background 0.15s;
}

.search-result:hover,
.search-result.selected {
    background: rgba(79,195,247,0.12);
}

.search-result-header {
    display: flex;
    justify-content: space-between;
    align-items: center;
    margin-bottom: 0.3rem;
}

.search-result-title {
    font-weight: 500;
    color: #e0e0e0;
}

.search-result-badge {
    padding: 0.2rem 0.5rem;
    border-radius: 4px;
    font-size: 0.7rem;
    font-weight: 600;
    text-transform: uppercase;
}

.search-badge-wiki { background: rgba(129,199,132,0.2); color: #81c784; }
.search-badge-hypothesis { background: rgba(255,213,79,0.2); color: #ffd54f; }
.search-badge-entity { background: rgba(79,195,247,0.2); color: #4fc3f7; }
.search-badge-paper { background: rgba(206,147,216,0.2); color: #ce93d8; }

.search-result-snippet {
    color: #adb5bd;
    font-size: 0.85rem;
    line-height: 1.4;
}

.search-no-results {
    padding: 2rem 1rem;
    text-align: center;
    color: #888;
}

/* Mobile responsive */
@media (max-width: 768px) {
    .search-container {
        width: 100%;
        margin-top: 0.5rem;
    }
    
    .search-results {
        max-height: 400px;
    }
}
</style>
"""

Implementation Plan

Phase 1: Backend API (2 hours)

Create /api/search endpoint with multi-table queries

Implement relevance scoring algorithm

Add result grouping by type

Test with various queries

Optimize query performance (<200ms target)

Phase 2: Frontend Integration (3 hours)

Add search component to nav_html()

Implement JavaScript autocomplete logic

Add debouncing (200ms delay)

Build results dropdown UI

Add keyboard navigation (arrows, Enter, Esc)

Phase 3: Styling & UX (1 hour)

Style search bar and results dropdown

Add type badges with colors

Make mobile responsive

Add loading states and transitions

Phase 4: Testing & Polish (1 hour)

Test across different content types

Test keyboard navigation

Test mobile responsiveness

Performance optimization

Edge case handling (no results, errors)

Total Estimated Time: 7 hours

Acceptance Criteria

Work Log

2026-04-16 13:30 PT — Slot 5 (minimax:75)

• Verified push succeeded: b6c790fab and 9ce7acefe now on origin/main
• Force-pushed to replace remote branch (was stuck on 61c5b34a9 which lacked "api.py" in message)
• Implementation verified: nav_html() has search box with type-grouped results, keyboard nav, mobile CSS
• /api/search endpoint at line 13608 with FTS5+BM25 ranking
• Task complete

2026-04-16 12:45 PT — Slot 5 (minimax:75)

• Task reopened: audit found NO_COMMITS on main — original work lost
• Previous commits 61c5b34a9 and 890551380 were not on origin/main
• Rebased on latest origin/main (396ace855)
• Amended commit message to explicitly mention api.py (pre-push hook requirement)
• api.py: Add nav search autocomplete with type-grouped results and keyboard navigation

2026-04-02 00:10 PT — Slot 4

• Task assigned: Unified search UI with autocomplete
• Created comprehensive spec with complete implementation
• API design: /api/search with relevance scoring
• Frontend: Vanilla JS + CSS autocomplete component
• Keyboard navigation: arrows, Enter, Esc
• Mobile responsive design
• Estimated 7 hours for full implementation
• Spec complete with code examples ready for integration

File: unified_search_spec.md

Modified: 2026-04-25 22:00

Size: 16.1 KB