Have you ever sat around watching the progress of your Azure DevOps pipelines, just waiting for them to finish? Maybe you were trying to fix a bug that only happened in a specific environment? Staging or production maybe?

At one point I was the one waiting. And waiting. And waiting again. What was I doing?

I was trying to solve a bug that only occurred in our production environment. This meant that I had to wait for our build pipeline, and our publish pipeline to finish before I could test if the bug was fixed.

Normally, our CI/CD pipelines works like a charm. And they did at this point as well. But it was a large project, with several api's and corresponding build pipelines for each of them. Which meant I had to cancel some of them just to speed things up. The specific api I was working on took about 7-8 minutes just in the build pipeline.

At this point I was getting tired of having to wait so long for each try. And I remembered seeing something about caching in the tasks menu . After doing some googling I managed to come up with a working solution. Look further down for a detailed solution.

2-3 minutes was the new build time!!!

So, this did't solve my problem, but I could now spend a whole lot less time waiting for my new builds to go through. After seeing such a good result I decided to implement caching on almost all of our pipelines. Our builds are now ready much faster in case we need to quickly publish an emergency patch.

.Net .yml with caching

trigger:
- master

pool:
  name: 'Azure Pipelines'
  vmImage: 'ubuntu-latest'

variables:
  solution: '**/*.sln'
  buildPlatform: 'Any CPU'
  buildConfiguration: 'Release'

steps:
- task: UseDotNet@2
  inputs:
    packageType: 'sdk'
    version: '9.0.x'

- task: Cache@2
  inputs:
    key: 'nuget | "$(Agent.OS)" | $(Build.SourcesDirectory)/**/*.csproj'
    restoreKeys: |
      nuget | "$(Agent.OS)"
    path: $(HOME)/.nuget/packages
    
- task: DotNetCoreCLI@2
  inputs:
    command: 'publish'
    publishWebProjects: false
    projects: '**/Api.csproj'
    arguments: '--configuration $(buildConfiguration) --output $(build.artifactstagingdirectory)'
    zipAfterPublish: false
  
- task: ArchiveFiles@2
  inputs:
    rootFolderOrFile: '$(build.artifactstagingdirectory)/Api'
    includeRootFolder: false
    archiveType: 'zip'
    archiveFile: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
    replaceExistingArchive: true

- task: PublishBuildArtifacts@1
  inputs:
    PathtoPublish: '$(Build.ArtifactStagingDirectory)/$(Build.BuildId).zip'
    ArtifactName: '_api'
    publishLocation: 'Container'

This Azure DevOps pipeline is designed to build, cache dependencies, publish, archive, and upload artifacts for a .NET 9 API project whenever a commit is pushed to the master branch.

Trigger Section

The pipeline is triggered automatically whenever there is a push to the master branch. This means it will run whenever new changes are merged into master.

Agent Pool

The pipeline runs on an Azure-hosted agent with ubuntu-latest as the operating system. This provides a clean Linux-based virtual machine for building the .NET application.

Pipeline Variables

Three variables are defined:

• solution: Looks for all .sln (solution) files in the repository.

• buildPlatform: Uses 'Any CPU' for compatibility across different environments.

• buildConfiguration: Uses 'Release' mode to ensure the application is optimized for production.

Installing the .NET 9 SDK

Since the project is using .NET 9, the UseDotNet@2 task is used to install the latest patch version of .NET 9. This ensures that the correct version of the SDK is available for the build process.

Caching NuGet Dependencies

To speed up builds, the Cache@2 task is used to store NuGet packages. This prevents the pipeline from downloading all dependencies every time it runs.

• The cache key is based on:

  • The OS (Agent.OS).

  • The .csproj files in the repository (so that the cache is invalidated if dependencies change).

• The cached NuGet packages are stored in the default location: $(HOME)/.nuget/packages.
  If the dependencies haven't changed since the last build, they are restored from the cache instead of downloading them again.

Building and Publishing the API

The .NET publish command is used to compile and package the API project (Api.csproj).

• The --configuration argument ensures that the build is done in Release mode.

• The output is stored in the artifact staging directory so it can be used in later steps.

• zipAfterPublish is set to false because the pipeline manually zips the build output in a later step.

Archiving the Published API

After the .NET publish step, the pipeline creates a ZIP archive of the published API files.

• The published files are taken from the artifact staging directory.

• The ZIP file is named using the Build ID, ensuring that each build gets a unique filename.

• The existing ZIP file is replaced if this step is run again.

Uploading the ZIP as a Build Artifact

The final step in the pipeline is to upload the ZIP file so it can be used for deployment.

• The ZIP file is stored as a pipeline artifact with the name _api.

• This allows other stages (like a deployment pipeline) to download and deploy the API.

Summary of What This Pipeline Does

1. Triggers on master branch - Runs automatically when new changes are pushed.

2. Uses an Ubuntu build agent - Runs in a clean environment.

3. Installs .NET 9 SDK - Ensures the correct runtime is available.

4. Caches NuGet packages - Speeds up dependency installation.

5. Builds and publishes the API - Compiles the project in Release mode.

6. Archives the build output - Creates a ZIP file for easy deployment.

7. Uploads the ZIP as an artifact - Stores the build for later use.

Angular .yml after the Ad

Angular .yml with caching

This pipeline is designed to build and cache dependencies for an Angular application, then publish the build artifacts for deployment.

trigger:
  branches:
    include:
      - main
      - develop
  paths:
    exclude:
      - README.md

pool:
  vmImage: 'ubuntu-latest'

variables:
  node_version: '18.x' # Set the Node.js version

steps:
  - task: NodeTool@0
    displayName: 'Use Node.js $(node_version)'
    inputs:
      versionSpec: $(node_version)

  # Cache npm packages
  - task: Cache@2
    displayName: 'Cache npm dependencies'
    inputs:
      key: 'npm | "$(Agent.OS)" | package-lock.json'
      restoreKeys: |
        npm | "$(Agent.OS)"
      path: '$(Pipeline.Workspace)/.npm'

  # Install dependencies
  - script: |
      npm ci --cache $(Pipeline.Workspace)/.npm --prefer-offline
    displayName: 'Install dependencies'

  # Build Angular App
  - script: |
      npm run build -- --configuration=production
    displayName: 'Build Angular application'

  # Publish build artifacts
  - task: PublishBuildArtifacts@1
    displayName: 'Publish build artifacts'
    inputs:
      pathToPublish: 'dist'
      artifactName: 'angular-app'

Trigger Section

The pipeline automatically triggers when changes are pushed to either:

• The main branch (for production deployments).

• The develop branch (for testing and development).

Additionally, it excludes changes to README.md, meaning that commits modifying only this file will not trigger the pipeline.

Agent Pool

The pipeline runs on a Microsoft-hosted Ubuntu machine (ubuntu-latest). This ensures that every run starts in a clean, isolated environment with all required tools.

Pipeline Variables

A variable node_version is defined and set to 18.x. This ensures that the correct version of Node.js is used for building the Angular application.

Install Node.js

The NodeTool@0 task ensures that the Node.js environment is installed with the specified version (18.x). This is necessary for running npm commands.

Cache npm Dependencies

The Cache@2 task is used to store npm dependencies based on the package-lock.json file.

• Why Cache?

  • If the dependencies haven't changed, they will be restored from the cache instead of downloading them again.

  • This significantly reduces build times.

• How it Works?

  • A cache key is created using:

    • "npm | $(Agent.OS) | package-lock.json" → Ensures the cache is OS-specific and updated when dependencies change.

  • The cached dependencies are stored in $(Pipeline.Workspace)/.npm.

Install Dependencies

The npm ci command is used to install dependencies in a clean way.

• Why use npm ci instead of npm install?

  • Ensures a clean install based strictly on package-lock.json (no surprises!).

  • Improves build consistency.

  • Faster than npm install because it skips package version resolution.

• Additional Flags:

  • --cache $(Pipeline.Workspace)/.npm → Uses the cached dependencies if available.

  • --prefer-offline → Prefers cached files over downloading new ones.

Build Angular Application

The npm run build command compiles the Angular application in production mode.

• Why use --configuration=production?

  • Enables production optimizations like Ahead-of-Time (AOT) compilation, minification, and tree-shaking.

  • Results in a smaller and faster app.

Publish Build Artifacts

The PublishBuildArtifacts@1 task uploads the compiled Angular application (dist folder) as a pipeline artifact.

• Artifact Name: angular-app

• Purpose:

  • The built application can now be used in a later stage (e.g., deployment to Azure Static Web Apps or a web server).

Not a magic fix for all scenarios

There are some scenarios when caching doesn't speed up the builds:

• Added a new package

• Removed a package

• Updated the version of a package

Caching can improve build times significantly, but you still have to try it out on your own projects to see if it is a good fit for your project. Currently we are only running caching on our .net projects, because the caching significantly reduced build time. With our Angular project we didn't see that much of a difference in build times.

Hope you gained some useful knowledge through this article.

Powered by Synscribe