DevOps proposals converge on the same vocabulary - CI/CD, infrastructure as code, Kubernetes, observability - which makes them almost impossible to compare on paper. The differences show up in how a vendor answers specific operational questions, because operating a system teaches things that configuring one does not.
01Ask what they would measure first
A vendor who proposes tooling before measuring your current delivery metrics is selling a template. The right first move is establishing a baseline - deployment frequency, lead time, change failure rate and time to restore - because without it nobody can prove the work improved anything.
This question also reveals whether they think in outcomes or deliverables. 'We will implement GitLab CI' is a deliverable. 'We will find out why your lead time is eleven days and fix the biggest contributor' is an outcome, and only one of them is worth paying for.
02Ask about a migration they advised against
Any experienced team has told a client not to do something - not to adopt Kubernetes, not to build a platform, not to migrate CI systems. A vendor who has never advised against work is either inexperienced or does not say no, and both are expensive over a long engagement.
The follow-up matters as much: what did they recommend instead, and what happened? A specific answer with a real trade-off indicates judgement. A vague one indicates the story was constructed for the question.
03Ask how they would handle your existing pipeline
The right instinct is to fix what exists before replacing it, because CI systems are rarely the actual bottleneck - caching, test ordering and manual approvals usually are. A vendor whose first move is migrating you to their preferred tool is optimising for their familiarity, not your delivery speed.
Migration is sometimes correct. What you want to hear is a specific reason your current tool cannot do what you need, not a general preference dressed as a recommendation.
| Question | Shallow answer | Answer with depth |
|---|---|---|
| Do we need Kubernetes? | Yes, it is the standard | Depends on service count, traffic shape, who operates it |
| How do you speed up CI? | Better runners | Cache keying, test sharding by timing, affected-target detection |
| How do you handle existing infra? | Rebuild it as code | Import incrementally, riskiest first, no rebuild |
| What is your first deliverable? | A pipeline | A measured baseline and the constraint worth fixing |
04Ask who actually does the work
The people in the pitch are frequently not the people on the project. Ask directly who will be assigned, what their background is, and whether you can speak to them before signing. A vendor who will not put you in front of the actual engineers is managing a risk that will become yours.
Also ask about continuity. Platform work depends on accumulated context, and a rotating cast rebuilds that understanding repeatedly at your expense.
- Meet the engineers who will be assigned, not just the leads.
- Ask what happens if your named engineer leaves the vendor.
- Confirm the ratio of senior to junior on your project.
- Ask how they hand over so you are not dependent on them.
05Ask what handover looks like
A platform only the vendor can operate is a dependency you will pay for indefinitely. Handover deliverables - documentation, runbooks, architecture decision records and genuine enablement for your team - should be contractual, not aspirational.
Test this by asking a simple question: if the engagement ended in three months, could your team run what they built? A vendor confident in their handover will answer that directly. One who hesitates has told you what the relationship will look like.
Topics
Priya Iyer
Staff Engineer · SyncTrix
Writes about the engineering decisions behind production systems - architecture, delivery and the trade-offs that only show up at scale.
Building something like this?
SyncTrix engineers AI, SaaS, platform and cloud systems for enterprises and high-growth teams. Tell us what you're shipping and we'll scope it with you.