[Hiring] Senior Fullstack Data & AI Search Engineer @Sparix Global.

Role Description

We are looking for a hands-on Data & AI Search Engineer to design and deliver a production-grade, AI-augmented enterprise search capability for a large international organisation. The engagement covers the full pipeline from raw data ingestion through to AI-generated, grounded answers surfaced via a conversational or search interface.

The right candidate combines deep Elasticsearch engineering with practical experience building Retrieval-Augmented Generation (RAG) pipelines and agentic AI workflows. This is an individual contributor role with direct impact on a critical knowledge management platform.

Key Responsibilities

• Data Engineering and Ingestion
  • Design and build scalable ingestion pipelines and connectors from enterprise sources including SharePoint, Liferay, web crawls, Data Lakes, and corporate systems into Elasticsearch or equivalent search indexes.
  • Support batch, incremental, and near-real-time indexing; implement change tracking, version management, source provenance, access permission mapping, and deletion event handling to keep the index accurate.
  • Build document conversion pipelines for PDF, Word, Excel, PowerPoint, HTML, email, and scanned content; convert to structured Markdown and vector embeddings using tools such as Marker, Docling, or equivalent frameworks.
  • Design semantic chunking strategies (chunk size, overlap, section-aware splitting, heading preservation, table handling) and implement metadata extraction, enrichment, and deduplication during ingestion.
• Retrieval and Search
  • Develop hybrid search capabilities combining BM25 keyword search, semantic vector search, metadata filtering, and contextual retrieval.
  • Build re-ranking pipelines using embedding models, cross-encoders, or custom ranking logic to improve result relevance.
  • Implement advanced retrieval techniques: query rewriting, query expansion, multi-query retrieval, parent-child retrieval, contextual document embeddings, and contextual compression.
  • Enforce security controls so users retrieve only content they are authorised to access.
• RAG Pipeline and Agentic Workflows
  • Design and build the end-to-end RAG pipeline connecting enterprise search to large language models for grounded answer generation.
  • Implement agentic workflows where the AI can invoke tools, call enterprise APIs, perform multi-step reasoning, and refine searches iteratively to answer complex queries.
  • Engineer prompt orchestration patterns: system prompts, retrieval prompts, guardrails, context assembly, response formatting, and fallback strategies for low-confidence or ambiguous queries.

Technical Requirements

• Core Search Engineering
  • Deep, hands-on Elasticsearch experience: query DSL, BM25 tuning, function_score, boosting and decay functions, multi-field matching.
  • Index and data modelling: field type selection, custom analyzers and tokenizers per content type (code, prose, structured records, multimedia).
  • Cluster operations: shard strategy, index sizing, reindexing, query latency tuning, and cluster health management.
  • Search evaluation and relevance testing: building ground-truth benchmarks, measuring precision/recall, NDCG, and iterating against them.
  • Experience with Elasticsearch, OpenSearch, Azure AI Search, or equivalent enterprise search platforms.
• Data and Ingestion Engineering
  • Proven experience building or configuring connectors for SharePoint, Liferay, databases, and Azure Data Lake including incremental sync, CDC, rate limiting, and API edge-case handling.
  • Proficiency in Python; experience with data processing frameworks and document conversion libraries.
• AI and RAG Engineering
  • Hands-on experience with embedding models, re-ranking models, cross-encoders, prompt engineering, and response grounding techniques.
  • Experience with LLM orchestration frameworks: LangChain, LlamaIndex, Haystack, or equivalent.
  • Practical experience with tool calling, agentic workflows, function calling, and multi-step retrieval.
  • Experience integrating with commercial or open-source LLMs: Azure OpenAI, OpenAI, Anthropic, Google Gemini, Meta Llama, Mistral, or similar.
• Frontend
  • Working knowledge of React or equivalent front-end technologies to support search UI integration (desirable, not mandatory).

Qualifications

• First-level university degree in Computer Science, Computer Engineering, Information Systems, or a related discipline.
• 8 years of professional experience in software or data engineering.
• Minimum 6 years of hands-on experience building enterprise search, AI-powered search, semantic search, RAG, or LLM-based applications.
• Excellent written and verbal communication skills in English.

Benefits

• We have built a high-visibility knowledge management platform for a large international organisation.
• End-to-end ownership across data engineering, retrieval, and GenAI layers.
• Fully remote, flexible working arrangement within agreed time zone coverage.
• Potential for contract extension based on delivery and business need.