25+ developers ship without a ticket. Three of us run the platform.

A new cloud foundation for a customer-facing application, with delivery teams from several organisations shipping on it. In production for three years.

100+changes a week, on average
25+contributors outside the platform team, last 3 months
200+cloud projects managed
BEFOREdaysAFTERminutesChangeTicketManual releaseProductionChangePull requestReviewMergeProductionBEFOREdaysAFTERminutesChangeTicketManual releaseProductionChangePull requestReviewMergeProduction

The situation

The organisation was building a new cloud foundation, and a custom stack on top of it, to deliver a customer-facing application. The platform was being built while teams were already working on it. Cloud resources were created by hand, one request at a time, and every new project, permission or deployment queued behind a handful of people.

Elsewhere in the same organisation

Releases go through a manual process with no automation. Even a simple configuration change takes days to reach production.

ELSEWHEREdaysON THE PLATFORMminutesChangeManual releaseprocessManualdeployProductionChangePullrequestReviewMergeappliesProductionELSEWHEREdaysON THE PLATFORMminutesChangeManual releaseprocessManualdeployProductionChangePullrequestReviewMergeappliesProduction
The path of one configuration change, elsewhere and on the platform.

Who does what

Product teams
Several dozen developers across several delivery teams, building the product’s services and frontends. They request their own environments, deployments and permissions.
Platform team
Three of us. We built the platform, review infrastructure and deployment changes, and run it.
Security team
A small team on the client side. They review every permission and access change.
Cloud governance
The client’s cloud policies, billing and auditing, delivered through the same pipelines.

Each group works in its own repository. A team proposes a change as a pull request, the people who own that area review it, and merging it is what applies it to the cloud. Nobody files a ticket, and nobody provisions anything by hand.

PROPOSESREPOSITORYREVIEWSMERGE APPLIESProductteamsCloudgovernancePlatformrepoDeploymentrepoSecurityrepoPolicypipelinesPlatformteamSecurityteamCloudprojectsRunningservicesRoles andaccessPolicy andbillingProductteamsCloudgovernancePlatformrepoDeploymentrepoSecurityrepoPolicypipelinesPlatformteamSecurityteamCloudprojectsRunningservicesRoles andaccessPolicy andbilling
Who proposes, who reviews, and what the merge applies.

What we built

  • A platform repository: one manifest per cloud project. Product teams propose new environments; the platform team approves.
  • A deployment repository: what runs in each environment, the cloud services it needs and how they connect. Product teams submit; the merge deploys.
  • A security repository: service accounts, roles and guardrails. Product teams request the permissions they need; the security team approves.
  • Pipelines for cloud governance, so policy, billing and audit configuration follow the same path.

What it changed

  • Over 100 changes a week, on average. About three in five are deployments; the rest are permission and security changes, mostly requested by product teams themselves.
  • Minutes from merge to production.
  • No security incident requiring review in three years. Risky permission requests are caught in review, before they reach the cloud.

Built to outlast its own tooling

The CI engine is being replaced, from Concourse to GitHub Actions, one project at a time. Both engines have run side by side for months, with no downtime and no issues. Not a single platform or deployment manifest had to change, because the manifests describe what should run, not how it gets deployed.

WHAT TEAMS WRITEWHAT DEPLOYS ITWHAT RUNSManifestsunchangedConcourseGitHub ActionsCloud projectsand servicesManifestsunchangedConcourseGitHub ActionsCloud projectsand services
Two engines, one set of manifests.

Delivery

Three engineers. First project in production in under six months. Running for three years.

Talk to the person who built it

If your teams wait on tickets to ship, or your platform team has become the bottleneck, we will tell you what we think in one call, before any proposal.

DanielCo-Founder

daniel@arendal.ca