AI-Powered Test Engineer (Colombia)

About The Role

We are a :15-person health-tech B2B platform running a serverless-first AWS stack (Lambda, API Gateway, RDS, Amplify, Cognito).

We're looking for a QA Engineer / SDET who writes test code, owns test infrastructure, and is fluent in using AI tooling to author and maintain tests — as well as testing AI-powered features themselves. You'll be the person who turns "we keep shipping regressions" into a durable, automated safety net.

This is a hands-on, code-writing role. You'll work directly with a small engineering team and own QA strategy end-to-end.

What you'll do

• Design and maintain end-to-end and regression test suites in Playwright (our current framework), including using `codegen` and AI-assisted authoring to accelerate coverage.
• Use AI tooling (LLM-based test generation, self-healing selectors, AI code assistants) to reduce the cost of writing and maintaining tests — with judgment about where AI helps and where it introduces flakiness.
• Test LLM- and MCP-powered product features: build evaluation harnesses, define quality bars for non-deterministic outputs, and design guardrail/regression checks for natural-language and agent-driven workflows.
• Attack our three standing testing gaps: mock drift, untested Lambda infrastructure, and recurring regressions that ship without accompanying tests.
• Build post-deploy smoke tests and contract tests against our integration layer (HubSpot, Dwolla, Intercom, PostHog, Slack).
• Partner with engineering to enforce a Definition of Done that requires tests, and embed QA into the sprint cadence rather than bolting it on at the end.
  
  
  

Must-have

• Strong JavaScript/TypeScript; you write and maintain test code, not just test plans.
• Production experience with Playwright, Cypress, or equivalent modern E2E frameworks.
• Hands-on experience using AI tools in a real testing workflow (test generation, maintenance, or triage).
• Experience testing non-deterministic / AI-powered systems, or a clear, credible approach to how you'd do it.
• Comfort in AWS serverless environments (Lambda, API Gateway) and testing distributed integrations.
• Ability to own strategy independently in a small team.
  
  
  

Nice-to-have

• Experience with LLM evaluation frameworks or building eval/guardrail systems.
• MCP or agent-based tooling familiarity.
• Prior QA-in-a-startup experience where you built the function from scratch.
• Health-tech, fintech, or regulated-industry background.
  
  
  

Why this role

You'll own QA at a company that treats testing as an engineering problem, not a checkbox — with real autonomy, a modern stack, and a mandate to bring AI into the workflow deliberately.