Spotlight: Scaling pipelines with VMSS
One of the first bottlenecks I spotted at ParentPay Group was in our delivery pipeline. Builds were constantly queuing, slowing down releases and frustrating developers across the business. It was a problem hiding in plain sight, and I couldn’t ignore it.
Digging deeper, I discovered most teams were using a messy combination of Microsoft-hosted agents and self-hosted Azure VMs. It worked, but it wasn’t efficient. The queues weren’t just costing us time — they were inflating costs unnecessarily.
After a detailed cost analysis and effort review, I realised something game-changing: because almost every developer had a Visual Studio Enterprise subscription, we could leverage parallel self-hosted jobs at no extra cost.
My proposal? Move to Azure Virtual Machine Scale Set (VMSS) Agents — giving us scalable, on-demand agent pools that cut wait times and reduced costs.
To make it happen, I went deep into the weeds:
Microsoft don’t publish their agent images for licensing reasons — but they do publish the GitHub Actions runner agents repo, which contains the code to generate replicas.
actions
Using this foundation, I created exact replica Azure Pipeline agent images on our own infrastructure.
I used Packer — a tool I’d never touched before — to build these images. At the time, publishing to an Azure Image Gallery wasn’t supported, so I even had to step into Go to extend the workflow the way I wanted.
The result was a set of images matching Microsoft’s spec across:
- Ubuntu 20.04
- Windows Server 2019
- Windows Server 2022
Infrastructure was provisioned with Terraform for repeatability and consistency.
Finally, I wired everything together with the Azure DevOps API to create and register the VMSS agent pools.
The system didn’t just work — it thrived. Each month an automation pipeline pulled the latest changes from Microsoft’s repo, built fresh images, and published them. This effectively offloaded image maintenance to Microsoft, while still giving us control of our infrastructure.
Closing
The end result was massive: fast, scalable, cost-effective, and secure agents — ready when developers needed them, without the queues. What had once been a daily frustration became an invisible, reliable foundation that empowered delivery instead of holding it back