Job Opportunities API

We are looking for an experienced Lead Data Engineer to lead the architecture, evolution, and delivery of our modern AWS-native data platform. This platform powers underwriting intelligence, GenAI-driven insights, and large-scale content processing, serving as the foundation for data products across the organization. You will own the full data lifecycle—from ingestion and transformation to storage, retrieval, and AI consumption—working across Python-based processing pipelines, .NET services, Aurora PostgreSQL, OpenSearch, and LangGraph-powered agentic workflows. This is a senior individual contributor role with significant technical leadership responsibilities. You will define engineering standards, drive architectural decisions, mentor engineers, and collaborate closely with product, platform, and AI teams to build reliable, scalable, and high-performing data systems. • Design, build, and evolve scalable, fault-tolerant data pipelines across ingestion, transformation, storage, and serving layers on AWS. • Own the delivery of cloud-native data solutions, including AWS Batch, Lambda, Python, .NET APIs (REST/GraphQL), Aurora PostgreSQL, and OpenSearch. • Define data architecture, schema design, indexing strategies, and data contracts to support analytics and GenAI-powered applications. • Establish and enforce data quality, testing, monitoring, and operational standards using CloudWatch, Splunk, Pentaho, and ThoughtSpot. • Partner with AI/ML and product teams to enable LangGraph-based agentic RAG solutions, ensuring reliable, high-quality, and retrieval-optimized data. • Develop and optimize data pipelines supporting LLM evaluation, prompt engineering, ground-truth datasets, and synthetic test data generation. • Review and validate AI-assisted engineering outputs, ensuring production readiness and technical excellence. • Drive technical design through architecture documents, RFCs, and engineering best practices, balancing scalability, maintainability, and business needs. • Lead deployment strategies, incident investigations, and continuous improvements to platform reliability and operational resilience. • Mentor engineers, establish data engineering standards, and provide technical leadership through design reviews and hands-on guidance. • Collaborate across engineering teams to manage data dependencies, align technical direction, and communicate delivery plans, risks, and architectural decisions.