Showing posts with label octopus deploy. Show all posts
Showing posts with label octopus deploy. Show all posts

Monday, April 25, 2016

First steps towards an awesome build and deploy pipeline

So we moved to the new awesome scripted build from VSTS and GitHub as our main code repository. But how do we handle the build and deployments? In this post I will show all the tips and tricks we did to get it working. Please note that this is a continues improving plan, but it will get you started.

Code setup

First our code setup. In the root of our project we have a couple of folders to not mix up code with other tools. So we have the following folders defined:
  • src (for sources)
  • build (for build scripts, see versioning below)
  • tools (for external tools)

Build setup

For the build server we've created the following steps. See the specific configurations below each name.

Build Tab

Delete files
Contents: **\<Namespace>*.nupkg (this is because we don't do a clean checkout each time)

Powershell
Script Filename: build/ApplyVersionToAssemblies.ps1

NuGet Installer
Path to Solution: **\*.sln

Visual Studio Build
Solution: **\*.sln
MSBuild Arguments: /p:RunOctoPack=true

Visual Studio Test
Test Assembly: **\bin\$(BuildConfiguration)\*test*.dll; -:**\xunit.runner.visualstudio.testadapter.dll; -:**\Microsoft.VisualStudio.QualityTools.UnitTestFramework.dll;-:**\<NameSpace>.TestUtils.dll

NuGet Publisher
Path/Pattern to nupkg **\bin\**\MeteoGroup.RouteGuard*.nupkg
NuGet Server Endpoint: Octopus Deploy (see Octopus Deploy, below)

Triggers Tab

The triggers trigger only the master, feature and hotfix branches. All other branches are not automatically build. The branch prefixes for feature and hotfix are based on the GitFlow naming, so if we want to use GitFlow, the naming is at least the same.


General Tab

Build Number Format: <YourProjectName>_2.$(Year:yy)$(DayOfYear)$(rev:.r)

Version Updater

The version updating script is a nifty little thing that uses regex to set the version number of the build in your assemblies. And with the assemblies properly versioned, your OctoPack will also use the proper version and so will your deployment. This brings the awesomeness that everything is connected to each other!
It will create versions like 2.16109.1.0 (for master branch builds) and 2.16109-feature-<FEATURENAME> for feature builds. The versions should be SemVer 1 compatible due to limitations in the NuGet 2.0 protocol. Safe the powershell script below in the build directory as ApplyVersionToAssemblies.ps1


##-----------------------------------------------------------------------
## <copyright file="ApplyVersionToAssemblies.ps1">(c) Microsoft Corporation.
## This source is subject to the Microsoft Permissive License.
## See http://www.microsoft.com/resources/sharedsource/licensingbasics/sharedsourcelicenses.mspx.
## All other rights reserved.</copyright>
##-----------------------------------------------------------------------
# Look for a 0.0.0.0 pattern in the build number. 
# If found use it to version the assemblies.
#
# For example, if the 'Build number format' build process parameter 
# $(BuildDefinitionName)_$(Year:yyyy).$(Month).$(DayOfMonth)$(Rev:.r)
# then your build numbers come out like this:
# "Build HelloWorld_2013.07.19.1"
# This script would then apply version 2013.07.19.1 to your assemblies.

# Enable -Verbose option
[CmdletBinding()]

# Regular expression pattern to find the version in the build number 
# and then apply it to the assemblies
$BuildVersionRegex = "\d+\.\d+\.\d+"
$FileVersionRegex = "\d+\.\d+\.\d+\.\d+"
$VersionTagRegex = "refs\/heads\/(\w*)\/([\w-]*)"

# If this script is not running on a build server, remind user to 
# set environment variables so that this script can be debugged
if(-not ($Env:BUILD_SOURCESDIRECTORY -and $Env:BUILD_BUILDNUMBER))
{
    Write-Error "You must set the following environment variables"
    Write-Error "to test this script interactively."
    Write-Host '$Env:BUILD_SOURCESDIRECTORY - For example, enter something like:'
    Write-Host '$Env:BUILD_SOURCESDIRECTORY = "C:\code\FabrikamTFVC\HelloWorld"'
    Write-Host '$Env:BUILD_BUILDNUMBER - For example, enter something like:'
    Write-Host '$Env:BUILD_BUILDNUMBER = "Build HelloWorld_0000.00.00.0"'
    exit 1
}

# Make sure path to source code directory is available
if (-not $Env:BUILD_SOURCESDIRECTORY)
{
    Write-Error ("BUILD_SOURCESDIRECTORY environment variable is missing.")
    exit 1
}
elseif (-not (Test-Path $Env:BUILD_SOURCESDIRECTORY))
{
    Write-Error "BUILD_SOURCESDIRECTORY does not exist: $Env:BUILD_SOURCESDIRECTORY"
    exit 1
}
Write-Verbose "BUILD_SOURCESDIRECTORY: $Env:BUILD_SOURCESDIRECTORY"

# Make sure there is a build number
if (-not $Env:BUILD_BUILDNUMBER)
{
    Write-Error ("BUILD_BUILDNUMBER environment variable is missing.")
    exit 1
}
Write-Verbose "BUILD_BUILDNUMBER: $Env:BUILD_BUILDNUMBER"
Write-Verbose "BUILD_SOURCEBRANCH: $Env:BUILD_SOURCEBRANCH"

# Get and validate the version data
$VersionData = [regex]::matches($Env:BUILD_BUILDNUMBER,$BuildVersionRegex)
switch($VersionData.Count)
{
   0        
      { 
         Write-Error "Could not find version number data in BUILD_BUILDNUMBER."
         exit 1
      }
   1 {}
   default 
      { 
         Write-Warning "Found more than instance of version data in BUILD_BUILDNUMBER." 
         Write-Warning "Will assume empty version tag."
      }
}

$VersionTagData = [regex]::matches($Env:BUILD_SOURCEBRANCH,$VersionTagRegex)
switch($VersionTagData.Captures.Groups.Count)
{
   0 {}
   3 
      {
        $VersionTag = $VersionTagData.Captures.Groups[1].value + '-' + $VersionTagData.Captures.Groups[2].value
      }
   default 
      { 
         Write-Error "Invalid version tag data in BUILD_SOURCEBRANCH." 
      }
}

$NewVersion = $VersionData[0].value
Write-Verbose "Version: $NewVersion"
if($VersionTag){
    Write-Verbose "VersionTag: $VersionTag"  
}

# Apply the version to the assembly property files
$files = gci $Env:BUILD_SOURCESDIRECTORY -recurse -include "*Properties*","My Project" | 
    ?{ $_.PSIsContainer } | 
    foreach { gci -Path $_.FullName -Recurse -include AssemblyInfo.* }
if($files)
{
    Write-Verbose "Will apply $NewVersion to $($files.count) files."

    foreach ($file in $files) {
        $filecontent = Get-Content($file)
        attrib $file -r
        $FileVersion = $NewVersion + ".0"
        $filecontent -replace $FileVersionRegex, $FileVersion | Out-File $file

        if($VersionTag) {
            Add-Content $file "`n[assembly: AssemblyInformationalVersion(`"$NewVersion-$VersionTag`")]"
            Write-Verbose "$file.FullName - version tag applied"
        }
        else {
            Write-Verbose "$file.FullName - version applied"
        }
    }
}
else
{
    Write-Warning "Found no files."
}

Octopus Deploy

You've probably already heard of Octopus Deploy. Your build server builds, and Octopus Deploys.
In the previous step you've seen that we've created a service endpoint in VSTS. You can add it by clicking on the Settings button and add a service.


With the versioning in place, everything will work fine in Octopus, and because we have names like 'Feature' or 'Hotfix' in the package, you can even setup channels to quickly deploy hotfix patches to production and allow feature packages only to be deployed on your development environment. But this is something we still need to setup (maybe in a future blog post).

Because the applications in the end don't know the release version / environment name, we've created a Variable set which is called 'Default Environment' and added the following 2 keys in it:
All applications that need to do something with it, can now use the Version and Environment name (in our case we use it to log the version to LogStash).

When you want to update more that one project, the octo.exe is there to help you, with this super simple tool, you can create releases and deploy multiple projects at once. For example:

@echo off
set SERVER=http://<YOUR OCTOPUS SERVER>/
set APIKEY=<API KEY>
set PACKAGEVERSION=<PACKAGE VERSION FROM VSTS, when not Feature/Hotfix, add .0 to it>
set TO=Development
set RELEASENOTES="<RELEASE NOTES>"

octo create-release --server %SERVER% --releasenotes=%RELEASENOTES% --apiKey %APIKEY% --packageversion %PACKAGEVERSION% --project "<Project Name> Api"
octo create-release --server %SERVER% --releasenotes=%RELEASENOTES% --apiKey %APIKEY% --packageversion %PACKAGEVERSION% --project "<Project Name> Application"
octo create-release --server %SERVER% --releasenotes=%RELEASENOTES% --apiKey %APIKEY% --packageversion %PACKAGEVERSION% --project "<Project Name> Data Ingestor"
octo create-release --server %SERVER% --releasenotes=%RELEASENOTES% --apiKey %APIKEY% --packageversion %PACKAGEVERSION% --project "<Project Name> Product Worker"

octo deploy-release --server %SERVER% --apiKey %APIKEY% --releaseNumber %PACKAGEVERSION% --deployto %TO% --waitfordeployment --project "<Project Name> Api"
octo deploy-release --server %SERVER% --apiKey %APIKEY% --releaseNumber %PACKAGEVERSION% --deployto %TO% --waitfordeployment --project "<Project Name> Application"
octo deploy-release --server %SERVER% --apiKey %APIKEY% --releaseNumber %PACKAGEVERSION% --deployto %TO% --waitfordeployment --project "<Project Name> Data Ingestor"
octo deploy-release --server %SERVER% --apiKey %APIKEY% --releaseNumber %PACKAGEVERSION% --deployto %TO% --waitfordeployment --project "<Project Name> Product Worker"

You'll be amazed with the nice colored logging that comes out of this beauty. Now a 5 O'clock deployment is nothing more than a click away (but don't do it :)) !

We're still in the process of improving the flow of the deployment, but I like the progress we made so far. If you have any tips or questions, let me know in the comments below!

Happy deploying!
Luuk

Monday, September 28, 2015

Automatic update of assembly version using TFS2013

To successfully implement Octopus Deploy you need to have unique version numbers for each build. If you don't want to manually edit the assembly info this could be a real pain in the ***. With the following trick you can automatically generate version numbers using TFS Build server 2013.

What I did was create a BuildCommon.targets that automatically searches for the AssemblyInfo and updates the version number that matches the build number as generated by TFS, and check this file in your codetree. In our case the file is named: BuildCommon.targets and is placed next to the root of the solution:

<Project xmlns="http://schemas.microsoft.com/developer/msbuild/2003" ToolsVersion="4.0">

<!--
    Defining custom Targets to execute before project compilation starts.
-->
<PropertyGroup>
    <CompileDependsOn>
        CommonBuildDefineModifiedAssemblyVersion;
        $(CompileDependsOn);
    </CompileDependsOn>
</PropertyGroup>

<!--
    Creates modified version of AssemblyInfo.cs, replaces [AssemblyVersion] attribute with the one specifying actual build version (from MSBuild properties), and includes that file instead of the original AssemblyInfo.cs in the compilation.

    Works with both, .cs and .vb version of the AssemblyInfo file, meaning it supports C# and VB.Net projects simultaneously.
-->
<Target Name="CommonBuildDefineModifiedAssemblyVersion" Condition="'$(VersionAssembly)' != ''">
    <!-- Find AssemblyInfo.cs or AssemblyInfo.vb in the "Compile" Items. Remove it from "Compile" Items because we will use a modified version instead. -->
    <PropertyGroup>
        <VersionAssembly>$([System.Text.RegularExpressions.Regex]::Replace($(VersionAssembly), `[\w|\D]+_`, ``, System.Text.RegularExpressions.RegexOptions.IgnoreCase))</VersionAssembly>
    </PropertyGroup>
    <ItemGroup>
        <OriginalAssemblyInfo Include="@(Compile)" Condition="(%(Filename) == 'AssemblyInfo') And (%(Extension) == '.vb' Or %(Extension) == '.cs')" />
        <Compile Remove="**/AssemblyInfo.vb" />
        <Compile Remove="**/AssemblyInfo.cs" />
    </ItemGroup>
    <!-- Copy the original AssemblyInfo.cs/.vb to obj\ folder, i.e. $(IntermediateOutputPath). The copied filepath is saved into @(ModifiedAssemblyInfo) Item. -->
    <Copy SourceFiles="@(OriginalAssemblyInfo)"
          DestinationFiles="@(OriginalAssemblyInfo->'$(IntermediateOutputPath)%(Identity)')">
        <Output TaskParameter="DestinationFiles" ItemName="ModifiedAssemblyInfo"/>
    </Copy>
    <!-- Replace the version bit (in AssemblyVersion and AssemblyFileVersion attributes) using regular expression. Use the defined property: $(VersionAssembly). -->
    <Message Text="Setting AssemblyVersion to $(VersionAssembly)" />
    <RegexUpdateFile Files="@(ModifiedAssemblyInfo)"
                Regex="Version\(&quot;(\d+)\.(\d+)(\.(\d+)\.(\d+)|\.*)&quot;\)"
                ReplacementText="Version(&quot;$(VersionAssembly)&quot;)"
                />
    <!-- Include the modified AssemblyInfo.cs/.vb file in "Compile" items (instead of the original). -->
    <ItemGroup>
        <Compile Include="@(ModifiedAssemblyInfo)" />
    </ItemGroup>
</Target>

<UsingTask TaskName="RegexUpdateFile" TaskFactory="CodeTaskFactory" AssemblyFile="$(MSBuildToolsPath)\Microsoft.Build.Tasks.v4.0.dll">
    <ParameterGroup>
        <Files ParameterType="Microsoft.Build.Framework.ITaskItem[]" Required="true" />
        <Regex ParameterType="System.String" Required="true" />
        <ReplacementText ParameterType="System.String" Required="true" />
    </ParameterGroup>
    <Task>
        <Reference Include="System.Core" />
        <Using Namespace="System" />
        <Using Namespace="System.IO" />
        <Using Namespace="System.Text.RegularExpressions" />
        <Using Namespace="Microsoft.Build.Framework" />
        <Using Namespace="Microsoft.Build.Utilities" />
        <Code Type="Fragment" Language="cs">
            <![CDATA[
            try {
                var rx = new System.Text.RegularExpressions.Regex(this.Regex);
                for (int i = 0; i < Files.Length; ++i)
                {
                    var path = Files[i].GetMetadata("FullPath");
                    if (!File.Exists(path)) continue;

                    var txt = File.ReadAllText(path);
                    txt = rx.Replace(txt, this.ReplacementText);
                    File.WriteAllText(path, txt);
                }
                return true;
            }
            catch (Exception ex) {
                Log.LogErrorFromException(ex);
                return false;
            }
        ]]>
        </Code>
    </Task>
</UsingTask>

</Project>

Then change the build number format to:
$(BuildDefinitionName)_0.1.$(Year:yy)$(DayOfYear)$(Rev:.r)

and the MSBuild arguments:
/p:CustomAfterMicrosoftCommonTargets="$(TF_BUILD_SOURCESDIRECTORY)\src\BuildCommon.targets" /p:RunOctoPack=true /p:OctoPackPublishApiKey=API-123465 /p:OctoPackPublishPackageToHttp=http://octopus-server/nuget/packages /p:VersionAssembly=$(TF_BUILD_BUILDNUMBER)

This should result in unique assembly versions for each build.

Many thanks for the creators of these posts to help me create this:
http://www.lionhack.com/2014/02/13/msbuild-override-assembly-version/
http://blog.casavian.eu/blog/2014/04/23/increment-version-for-changed-assemblies-only-first-part/
http://blogs.msdn.com/b/visualstudio/archive/2010/04/02/msbuild-property-functions.aspx

Cheers,
Luuk