Query TFS/VSTS for past build history

April 20, 2017    VSTS DevOps TFS

Query TFS/VSTS for past build history

We’re using TFS on premise (2017, not an update) to run builds, track work, etc at my current client. This should also work on VSTS (which I hope we will get to move towards someday).

I was asked to figure out why our build times have increased over the last couple months. My first task was to get the metrics. After not finding the graph I wanted in the charts in the TFS dashboard, I turned to the TFS API.

I used this starting point from the Microsoft docs.

There was also some info from StackOverflow that was helpful.

I created a plain old .Net Console application to spit out a .csv file.

I added a few Nuget packages

  • Microsoft.TeamFoundationServer.Client
  • Microsoft.VisualStudio.Services.Client
  • Microsoft.VisualStudio.Services.InteractiveClient

Then I added some code

using System;
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text;
using System.Threading.Tasks;
using Microsoft.TeamFoundation.Build.WebApi;
using Microsoft.VisualStudio.Services.Client;
using Microsoft.VisualStudio.Services.WebApi;`

namespace GetBuildHistory
{
    class Program
    {
        static void Main(string[] args)
        {
            MainAsync().Wait();
        }

        static async Task MainAsync()
        {
            var collectionUri = "http://tfs:8080/collection/";
            var teamProjectName = "{Your_Project_Name}";
            var targetBuildName = "{Your_BUILD_Name}";

            VssConnection connection = new VssConnection(new Uri(collectionUri), new VssClientCredentials());
            var buildserver = connection.GetClient<BuildHttpClient>();
            var builds = await buildserver.GetBuildsAsync(
                statusFilter: BuildStatus.Completed,
                project: teamProjectName);
            var targetedBuilds = builds
                .Where(definition => definition.Definition.Name.Contains(targetBuildName))
                .OrderBy(b => b.FinishTime)
                .ToList();

            ProcessBuilds(targetedBuilds);
        }

        private static void ProcessBuilds(List<Build> targetedBuilds)
        {
            var pathString = [email protected]"C:\results\general_fullBuildResults.csv";
            var stringBuilder = new StringBuilder();
            stringBuilder.AppendLine("Name, Date, Passed, Build Time, Queue Time");
            foreach (var build in targetedBuilds)
            {
                var buildInfo = new BuildInfo
                {
                    Name = build.BuildNumber,
                    Passed = build.Result != null && build.Result.Value == BuildResult.Succeeded,
                    FinishTime = build.FinishTime.Value,
                    Started = build.StartTime.Value,
                    QueueStartTime = build.QueueTime.Value
                };

                stringBuilder.AppendLine(buildInfo.ToString());
            }

            File.WriteAllText(pathString, stringBuilder.ToString());
        }
    }

    internal class BuildInfo
    {
        public string Name { private get; set; }
        public bool Passed { private get; set; }
        public DateTime Started { private get; set; }
        public DateTime FinishTime { private get; set; }
        public DateTime QueueStartTime { private get; set; }

        private string InQueueTimeMinutes => this.CalculateTimeInQueue();

        /// <summary>
        /// The time the build took to run.
        /// </summary>
        public string TimeRanInMinutes => this.CalculateTimeRan();

        private string CalculateTimeRan()
        {
            var diff = this.FinishTime - this.Started;
            return (diff.TotalSeconds / 60).ToString(CultureInfo.CurrentCulture);
        }

        private string CalculateTimeInQueue()
        {
            var diff = this.Started - this.QueueStartTime;
            return (diff.TotalSeconds / 60).ToString(CultureInfo.CurrentCulture);
        }

        public override string ToString()
        {
            return $"{this.Name}, {this.Started.ToLocalTime():g}, {this.Passed}, {this.TimeRanInMinutes}, {this.InQueueTimeMinutes}";
        }
    }
}

I will probably add to this, but this works for me. It spits out a csv that I can use Excel to create an easy chart. This could morph into a TFS extension chart (which would be great, but more work and time than I have right now). We could even run this automatically and alert if the build time goes over a certain threshold. That could mean we have problems (see the DevOps Handbook Chapter 14 on the importance of metrics).

I’m authenticating through my Windows Active Directory account, so the console didn’t prompt me. Check the docs link above for more information about that.

But wait, there’s more

We can even drill into the task level of the builds and use Power BI to visualize the timings there too.

 /// <summary>
    /// Get the timeline information from the REST API
    /// </summary>
    /// <remarks>
    /// reference of the REST API: https://www.visualstudio.com/en-us/docs/integrate/api/build/builds
    /// </remarks>
    /// <param name="buildId"></param>
    private static async Task<Timeline> GetTimelineAsync(int buildId)
    {
        Console.WriteLine($"Getting timeline for {buildId}....");
        return await buildServerClient.GetBuildTimelineAsync(teamProjectName, buildId);
    }

I added this to my BuildInfo.cs object and used it to create the CSV rows.

public string GetTimelineCsvRow()
{
    // Build Name, Date, Passed, Task Name, Time in Minutes, State, Result
    var sb = new StringBuilder();
    var orderedRecords = this.Timeline.Records
        .OrderBy(record => record.StartTime).ToList();

    orderedRecords.ForEach(record =>
    {
        try
        {

        sb.AppendLine($"{this.Name}, {record.StartTime.Value.ToLocalTime():g}, " +
                    $"{this.Passed}, {record.Name}, " +
                    $"{this.CalculateTimeRan(record.StartTime.Value, record.FinishTime.Value)}, ${record.State}, ${record.Result}, ${record.Url}");

        }
        catch (Exception e)
        {
            Console.WriteLine(e);
            sb.AppendLine("Exception");
        }
    });

    return sb.ToString();
}

Here’s an example of the .csv output. I’ve removed some steps to condense it.

Build Name Start Time Passed Task Name Time in Minutes State Result
1.0.662.0 6/27/2017 8:30 PM True Build 40.336 $Completed $Succeeded
1.0.662.0 6/27/2017 8:30 PM True Get sources 0.901833333 $Completed $Succeeded
1.0.662.0 6/27/2017 8:31 PM True Add Git Remote 0.020283333 $Completed $Succeeded
1.0.662.0 6/27/2017 8:31 PM True Run git pull 0.008566667 $Completed $Succeeded
1.0.662.0 6/27/2017 8:31 PM True npm install --cache-min 604800 1.1625 $Completed $Succeeded
1.0.662.0 6/27/2017 8:32 PM True NuGet restore Project.sln 0.280716667 $Completed $Succeeded
1.0.662.0 6/27/2017 8:32 PM True Build Project.sln 0.4289 $Completed $Succeeded
1.0.662.0 6/27/2017 8:33 PM True grunt build 1.751333333 $Completed $Succeeded
1.0.662.0 6/27/2017 8:34 PM True C# Tests 0.4266 $Completed $Succeeded
1.0.662.0 6/27/2017 8:35 PM True JS Unit Tests 1.151566667 $Completed $Succeeded
1.0.662.0 6/27/2017 8:36 PM True Publish Test Results **/*.trx 1.2375 $Completed $Succeeded
1.0.662.0 6/27/2017 8:38 PM True Build ProjectInstaller.sln 0.25755 $Completed $Succeeded
1.0.662.0 6/27/2017 8:40 PM True Copy files from buildScripts 0.781766667 $Completed $Succeeded
1.0.662.0 6/27/2017 8:41 PM True Run UI Install on AGENT-01 AGENT-02 AGENT-03 AGENT-04 1.622166667 $Completed
1.0.662.0 6/27/2017 8:42 PM True Run Tests 15.54276667 $Completed $Succeeded
1.0.662.0 6/27/2017 9:05 PM True Run Cleanup on AGENT-01 AGENT-02 AGENT-03 AGENT-04 1.322283333 $Completed
1.0.662.0 6/27/2017 9:06 PM True Publish symbols path: \Symbols 2.599833333 $Completed $Succeeded
1.0.662.0 6/27/2017 9:10 PM True Publish Artifact: Web (Full) 2017.06.27.2030.1 0.052666667 $Completed $Succeeded
1.0.662.0 6/27/2017 9:10 PM True Post Job Cleanup 0.0029 $Completed $Succeeded
1.0.662.0 6/27/2017 9:10 PM True Finalize build 0.003383333 $Completed $Succeeded
1.0.662.0 6/27/2017 9:10 PM True Label sources 0.003383333 $Completed $Succeeded



comments powered by Disqus