CI/CD

CI/CD runs the test suite automatically on a shared server on every code change and uses the result to gate merges/releases.

Asked 'How do you integrate your automated tests into a CI/CD pipeline?' / 'Walk me through the stages of your test pipeline.'

named in the job descriptions

A failing test exits non-zero, the job fails, the PR check goes red, and branch-protection required-checks physically block the merge.

Asked 'What actually stops broken code from reaching main?' / 'What's a quality gate?'

named in the job descriptions

Repo setting that requires named checks to pass before merge is allowed; this is what converts a red build into a blocked merge.

Asked 'How is the gate actually enforced in GitHub?'

named in the job descriptions

Control when a workflow runs and the difference between running on PR vs on merge to main.

Asked 'When does your pipeline run?' / 'PR builds vs nightly builds?'

named in the job descriptions

Know when a step calls a reusable action (uses:) vs runs a shell command (run:) and the four workhorse actions.

Asked Follow-ups while whiteboarding the workflow.

asked in interviews, not named in any job description

Write a working .github/workflows/*.yml that checks out code, sets up Python, installs deps + browsers, runs pytest, and uploads the report.

Asked 'Write a GitHub Actions workflow that runs Playwright tests on every PR.'

named in the job descriptions

Understand a declarative Jenkinsfile (pipeline { agent / stages / stage('Test') { steps {...} } }) well enough to read one and add a test stage.

Asked 'Add a stage to this Jenkinsfile that runs the regression suite and publishes the report.'

named in the job descriptions

The Jenkins equivalents of GHA artifacts/secrets: post{always{}} block, junit/publishHTML for reports, withCredentials/credentials() for secrets.

Asked 'How do reports and credentials work in Jenkins vs GitHub Actions?'

named in the job descriptions

When to pick which: Jenkins = self-hosted, customizable, plugin-rich, you maintain it; GHA = GitHub-native, serverless, YAML, zero-maintenance.

Asked 'Jenkins vs GitHub Actions - when would you pick which?'

named in the job descriptions

Azure Pipelines YAML is another CI platform; same trigger/stage/step concepts.

Asked 'Any Azure DevOps exposure?' (screening).

named in the job descriptions

.gitlab-ci.yml with stages/jobs; the 'parallel' keyword makes sharding trivial; native Docker + merge-request integration.

Asked 'Have you used GitLab CI?' / 'How would you shard in GitLab?'

named in the job descriptions

Other hosted CI platforms that appear in some JDs; same pipeline concepts.

Asked Screening keyword match only.

named in the job descriptions

A gate is an automated pass/fail rule that must be green to merge/release (all required suites pass, min coverage, no critical vulns, required reviewer).

Asked 'What's a quality gate? Give examples.' / 'Own CI/CD quality gates - decide what blocks a merge.'

named in the job descriptions

Fail the build when coverage drops below a threshold via pytest --cov --cov-fail-under=N.

Asked 'How do you enforce coverage in CI?'

named in the job descriptions

Upload HTML report + traces + JUnit XML even when tests fail, using if: always()/!cancelled().

Asked 'Where do artifacts go and how do you get them on failure?'

named in the job descriptions

Know the three artifact types and that JUnit XML is what CI natively parses for pass/fail and test-result UIs.

Asked 'What reports does your pipeline produce and how do you triage a failure?'

named in the job descriptions

Aggregated historical test-result dashboards across runs (trends, flaky history, categorization).

Asked 'How do you visualize test results over time?'

named in the job descriptions

workers/pytest-xdist parallelize across CPU cores on one machine; sharding splits the suite across multiple CI machines via a job matrix.

Asked 'How would you cut a 40-minute suite to under 10?' / 'How do you run tests in parallel in CI?'

named in the job descriptions

strategy: matrix fans one job into N parallel jobs across browsers, versions, or shard indices.

Asked 'How do you run across browsers/versions in CI?'

asked in interviews, not named in any job description

Cache dependencies and browser binaries to cut minutes off every run.

Asked 'How do you speed up CI runs?' (paired with sharding answer).

asked in interviews, not named in any job description

Fan-out/fan-in: each shard emits partial results, a final needs:-dependent job merges them into one report.

Asked 'You sharded the suite - how do you get one combined report?'

asked in interviews, not named in any job description

For his Python/pytest stack sharding needs a plugin: pytest-shard (--shard-id/--num-shards, hash-based) or pytest-split (--splits/--group, duration-based).

Asked 'How do you shard a pytest suite across CI machines?' (deep follow-up exposes the runner nuance).

asked in interviews, not named in any job description

Run tests inside mcr.microsoft.com/playwright/python:<ver>-noble so browsers + system libs are baked in and CI matches local.

Asked 'How do you make CI reproducible?' / 'Why run tests in Docker?'

named in the job descriptions

Tests can execute inside K8s pods; knowing that is enough for an SDET.

Asked 'Any Kubernetes exposure?' (screening).

named in the job descriptions

Systematic diagnosis of environment-specific failures.

Asked 'A test passes locally but fails in CI - how do you debug?'

named in the job descriptions

Identify flaky tests, retry in CI (retries: 2 max) without masking, quarantine so they don't block the gate, track, root-cause, re-enable.

Asked 'How do you handle flaky tests in CI?' (asked almost universally).

named in the job descriptions

Measure flaky rate as a number; treat >2% as a reliability incident, not noise.

Asked 'How do you know your CI is trustworthy?'

named in the job descriptions

Inject secrets via GHA encrypted secrets (${{ secrets.X }}) or Jenkins credentials; they are masked in logs and passed as env vars.

Asked 'How do you handle secrets/credentials in the pipeline?'

asked in interviews, not named in any job description

Run a fast subset on PRs and the full cross-browser suite nightly via cron.

Asked 'Do you run everything on every PR? How do you balance speed and coverage?'

named in the job descriptions

Post pass/fail to Slack/Teams via webhook on failure and/or comment the report link on the PR.

Asked 'How does the team find out a build failed?'

named in the job descriptions

Provision the app + data CI needs per run: DB seeding, test-data factories, sidecar services (GHA services:/docker-compose), cleanup.

Asked 'How do you manage test data in CI when tests run on every PR?'

named in the job descriptions

DRY pipeline logic reused across repos.

Asked 'How do you avoid duplicating pipeline config across services?'

asked in interviews, not named in any job description

Spin a fresh app instance per PR, run E2E against it, tear it down; usually DevOps-built.

Asked 'How would you test against isolated per-PR environments?'

named in the job descriptions

Run only tests affected by the diff on PRs, full suite nightly.

Asked 'How do you keep PR feedback fast as the suite grows?'

asked in interviews, not named in any job description

Derive shard count from actual test volume instead of hardcoding, eliminating maintenance.

Asked Design-question follow-up on scaling the pipeline.

asked in interviews, not named in any job description

Architect a pipeline end-to-end under interview pressure.

Asked 'Architect a CI/CD pipeline for 500 daily test runs.'

named in the job descriptions

LLM outputs are non-deterministic, so CI gates them with threshold/score bands (pass-rate, score windows), not exact-match asserts.

Asked 'How would you put an AI/LLM feature under a CI quality gate?'

asked in interviews, not named in any job description

SDET owns the test stage (steps that run the suite, gates, parallelization, flakiness, reports); DevOps/SRE owns the platform (runners, cluster, deploy mechanics, secret infra).

Asked 'Where does your responsibility for the pipeline end?'

named in the job descriptions