Senior Engineer — AI Security, Systems Engineering & Developer Productivity

Core Charter & Leadership

Technical Skills Matrix

4 Key Pillars of Delivery

🧠 Proven Expertise: AI/LLM Consumption Patterns, AI Platform Integration & Secure AI Application Development

How a Staff/Senior Engineer designs, integrates, governs, and secures enterprise Large Language Model (LLM) workloads end-to-end.

• Real-Time Streaming (SSE / WebSockets / gRPC): Uses Server-Sent Events (text/event-stream) for interactive copilots and chat UIs to achieve Time-To-First-Token (TTFT) < 500ms while tokens stream progressively.
• Prefix KV-Caching & Semantic Caching: Reuses prompt prefix Key-Value (KV) caches (cutting input token cost by up to 90% and latency by 80% on shared system prompts/RAG docs) + Redis Vector Semantic Caching for repeated queries.
• Structured Outputs & Constrained Decoding: Enforces strict JSON Schema / Pydantic / Zod contracts via response_format so downstream microservices parse deterministic JSON without regex failures.
• Model Cascading & Tiered Routing: Routes 70–80% of high-volume tasks (intent routing, summarization, classification) to low-cost Small Language Models (SLMs like Claude Haiku / Gemini Flash / GPT-4o-mini) and escalates complex multi-step reasoning to frontier models.
• Async Batch APIs & Event-Driven Consumers: Uses Kafka/SQS queues + Batch Inference APIs (50% cost discount) for non-blocking workloads like repo-wide SAST vulnerability triage, PR review, and document embedding pipelines.
• Hybrid RAG vs. Long-Context vs. Fine-Tuning (LoRA): Combines BM25 keyword search + dense vector embeddings + Reciprocal Rank Fusion (RRF) + Cross-Encoder Reranking for live enterprise grounding.

• Unified AI Gateway (Kong AI / LiteLLM / Envoy / Portkey): Abstracts AWS Bedrock, Azure OpenAI, Google Vertex AI, and Anthropic behind a single OpenAI-compatible SDK interface—eliminating vendor lock-in.
• Multi-Provider Failover & Circuit Breaking: Automatically fails over across regions or model families when an upstream provider returns HTTP 429 (Rate Limit) or HTTP 503 (Overloaded).
• Model Context Protocol (MCP) & Tool Orchestration: Connects agents to internal enterprise systems (GitHub, Jira, ServiceNow, Kubernetes, PostgreSQL) via standardized JSON-RPC MCP servers with Idempotency-Key protection on state-mutating calls.
• Token FinOps, Quotas & Chargeback: Enforces multi-tenant Tokens-Per-Minute (TPM), Requests-Per-Minute (RPM), and daily USD budget caps in Redis, attributing exact prompt/completion costs per business unit.
• LLMOps Observability (OpenTelemetry GenAI): Emits W3C distributed traces capturing prompt version, model ID, temperature, token counts, TTFT, and tool latency to Datadog / LangSmith / Arize Phoenix.
• CI/CD Evaluation Gates (LLM-as-a-Judge & RAGAS): Blocks prompt or model changes in CI if Faithfulness, Answer Relevance, or Hallucination benchmarks regress on golden datasets.

• Prompt Injection Defense (Direct & Indirect — LLM01): Uses structural XML delimiters (<untrusted_data>), Dual-LLM isolation (Quarantined Reader vs. Privileged Planner), and inline guardrail classifiers (Llama Guard / Azure AI Content Safety / NeMo).
• PII / Secret Scrubbing & Zero Data Retention (LLM02): Redacts PII, PHI, API keys, and credentials pre-inference (Microsoft Presidio + TruffleHog), enforces .copilotignore, and mandates Enterprise Zero Data Retention (ZDR) & no-training clauses.
• Permission-Aware RAG (Document-Level ACLs): Enforces Role-Based / Attribute-Based Access Control (RBAC/ABAC) metadata filters inside the vector database query before context assembly so users cannot retrieve unauthorized files.
• Insecure Output Handling Prevention (LLM05): Treats all model output as untrusted user input—enforcing parameterized SQL queries, HTML sanitization (DOMPurify), and AST validation before executing generated code.
• Least-Privilege Agentic Agency & Sandboxing (LLM06): Runs code execution and MCP tools inside ephemeral gVisor / Firecracker microVMs with scoped OAuth 2.1 Non-Human Identities (NHI) and strict egress allowlists.
• Human-in-the-Loop (HITL) Step-Up Approval: Requires explicit cryptographic human confirmation before an agent executes high-blast-radius actions (prod deployments, DB mutations, external emails, wire transfers).

• Audit Shadow AI usage across Digital Technology using CASB/DNS logs.
• Establish GitHub Copilot Enterprise baseline and enforce `.copilotignore` boundaries.
• Catalog top 500 platform vulnerabilities in the Mythos pipeline.
• Baseline Java 8/11 inventory across core transactional services.

• Launch pilot of Copilot Autofix with GHAS & CodeQL on 20 critical repos.
• Build and deploy first enterprise Claude Skill with MCP for Jira & Git triage.
• Implement OpenRewrite automated recipes for Java 17/21 language features.
• Deploy Kyverno admission controllers on EKS to enforce signed container images.

• Scale Mythos remediation roadmap across all engineering tribes with EPSS scoring.
• Roll out automated Rapid Release Hotfix pipeline for sub-4-hour zero-day turnarounds.
• Instrument OpenTelemetry (OTel) metrics for developer velocity & LLM latency.
• Present board-level AI security posture report and measurable MTTR reduction.