
ISTScaling the AI observability & evals platform at Confident AI, Inc.
I build data-intensive systems and the infrastructure underneath them.
My focus is distributed systems, observability and platform engineering.
At Confident AI, I work across LLM evaluation pipelines, Kubernetes, multi-region infrastructure on AWS, Azure and GCP, and backend services in Python and TypeScript.
Previously at TurboML, Inc. and Puch AI. Long-time open-source contributor.
01Activity
7,393 contributions on GitHub in the last year.
- Active days
- 262
- Longest streak
- 41d
- Busiest week · Nov 1
- 471
- Best day · Nov 12
- 200
02Experience
Where I have worked and what I built there, most recent first.
- Confident AI, Inc. (YC W25)
- Founding Engineer · Full-timeOct 2025 – Present
- Throughput
- 240% more on online evaluations, with the peak error rate down from ~50% to under 0.1% at 5x the load.
- Migration
- US and EU production moved from Railway to AWS EKS overnight, with no downtime at cutover.
- Open source
- AWS, GCP and Azure Terraform modules published on HashiCorp's Terraform Registry.
All 24 highlights
First full-time founding engineer hire.
- Migrated online evaluations to a delegation-based asynchronous architecture: 240% more throughput and peak error rate down from ~50% to under 0.1% at 5x the previous peak load.
- Migrated US and EU production and the staging deployment pipeline from Railway to AWS (EKS) overnight, with no downtime at cutover.
- Placed servers next to their databases in each region to cut latency.
- Set up autoscaling with Karpenter and KEDA, so capacity follows real traffic and queue load.
- Moved every infrastructure change to pull requests with ArgoCD, approvals and nightly drift checks.
- Tightened security: no static cloud keys, encrypted secrets, private networking and fixed IPs for database allow-lists.
- Set up full observability with Datadog and CloudWatch alarms across every node and service.
- Built Anthropic, LangChain and OpenAI integrations for the DeepEval Python and TypeScript SDKs.
- Manage multi-region AWS, Azure and GCP infrastructure with Docker, Kubernetes, Terraform, CI/CD and ArgoCD.
- Built Terraform modules for VPC, EKS/ECS, RDS, ALB/NLB, NAT, S3, networking and IAM across multi-AZ deployments, and published AWS, GCP and Azure modules on HashiCorp's Terraform Registry.
- Designed production Kubernetes workloads, RBAC, storage, networking and secrets management with ArgoCD sync waves.
- Built asynchronous systems for alerts and monitors, and a centralised streaming gateway for the LLM Playground, Arena, Experiments and Evals.
- Extended LLM support across Bedrock, Vertex AI, Azure OpenAI, Portkey, OpenRouter, TrueFoundry and LiteLLM.
- Built cloud-agnostic storage across S3, GCP Storage, Azure Blob and MinIO with secure pre-signed URL delivery.
- Built a secure code sandbox for executing code-based evaluation metrics on serverless functions (AWS Lambda, Azure Functions).
- Built Confident AI's relay agent for private-subnet tunnelling without exposing AI Connections, with Azure AD authentication.
- Added MCP server context to AI Connections and the evaluation pipeline, with MCP OAuth 2.0 client credentials.
- Built Public Dashboards for securely sharing graphs, tables and widgets.
- Built metered billing with Stripe invoicing, platform licensing, API key rotation and expiry, and AWS Marketplace licensing.
- Built code vulnerability scanning with DeepTeam, GitHub repository integration and CI security with Harness integration.
- Built weekly release CI/CD, Helm charts, ClickHouse export pipelines, multipart S3 uploads with gzip compression and secure public pre-signed URLs.
- Added IAM role and Mantle API support for Bedrock integrations, and real-time audit log streaming to Datadog.
- Consolidated APIs and schemas for scheduled tasks, models and configuration, graphs and widgets, and SSE.
- Built multi-user edit activity.
- AWS
- Azure
- GCP
- Kubernetes
- Karpenter
- KEDA
- Helm
- Terraform
- ArgoCD
- Python
- TypeScript
- PostgreSQL
- ClickHouse
- Redis
- Datadog
Puch AI- Software Engineer · Part-timeJun 2025 – Oct 2025
- Agent
- Built the tool-calling router and integrations that power the agent.
- Data
- Moved the primary database from PostgreSQL to MongoDB with minimal downtime.
All 5 highlights
- Built the tool-calling router and integration abstractions that power the agent, in Python.
- Implemented message debouncing on top of a task queue and worker setup.
- Designed output parsing that turns LLM responses into WhatsApp-compatible message structures.
- Migrated the primary database from PostgreSQL to MongoDB with minimal downtime.
- Set up the LLMOps lifecycle: tracing, prompt management, session tracking, memory, evaluations and datasets.
- Python
- MongoDB
- PostgreSQL
- Task Queues
- LLMOps
TurboML, Inc.- Software Engineer · ContractJun 2024 – Oct 2025
- Queries
- ~40% faster with a purpose-built PostgreSQL access layer replacing the ORM.
- Latency
- ~30% lower response times with Redis caching and asynchronous processing on Kafka.
All 6 highlights
- Worked across the LLMOps platform: backend services, dashboard and Python SDK.
- Maintained the observability SDK for distributed LLM systems with OpenTelemetry instrumentation.
- Improved query performance by ~40% with a purpose-built PostgreSQL access layer replacing the ORM.
- Designed storage for metrics, prompts and span datasets on PostgreSQL and ClickHouse.
- Cut response times by ~30% with Redis caching and asynchronous processing on Kafka.
- Added API key and project namespaces to the SDK, enabling integration in under five lines of code.
- Software Engineer Intern · InternshipDec 2023 – May 2024
- Requests
- ~60% fewer redundant requests through client-side caching with TanStack Query.
All 3 highlights
- Migrated the MLOps and LLMOps dashboard from Streamlit to Next.js with richer data visualisation.
- Reduced redundant requests by ~60% through client-side caching with TanStack Query.
- Built a careers portal with resume parsing, candidate filtering and admin tooling on Hono.
- Python
- FastAPI
- TypeScript
- Next.js
- PostgreSQL
- ClickHouse
- Kafka
- Redis
03Projects & Open Source
Open-source work at Confident AI, and side projects in systems and infrastructure.
Open source
- deepevalconfident-aiThe open-source LLM evaluation framework.Built the Anthropic, LangChain and OpenAI integrations for the Python and TypeScript SDKs.19.5K216 commits
- deepteamconfident-aiA framework to red team LLMs and AI agents.Built code vulnerability scanning with DeepTeam in the Confident AI platform.3.1K20 commits
Projects
- wormhole-aiVersion control for prompts, with a playground and a prompt registry.TypeScript, Next.js
- balancer-robinLoad balancer that checks server availability and routes requests round-robin.Go
- grpcalcCalculator service over gRPC with a Go client and server.Go, gRPC
- tcp-goTCP server built with Go and goroutines.Go
04Writing
Notes on distributed systems, infrastructure and building with LLMs.
05Stack
Tools I reach for day to day.
- Languages
- Python, TypeScript, Go, SQL
- Infrastructure
- AWS, Azure, GCP, Kubernetes, Helm, Terraform, Docker, ArgoCD
- Data
- PostgreSQL, ClickHouse, Redis, Kafka, MongoDB
- Backend
- FastAPI, Node.js, Hono, gRPC, OpenTelemetry
06Contact
Building infrastructure for LLM systems. If you are working on something similar, I would like to hear about it.
- Built with
- Next.js 16, Tailwind CSS, Geist
- Deployed
- Nov 18, 2026fe4bac1
- Revision
- 2026.11