SDLC Telemetry

Legacy of Agile

Over the last decade, organizations have seen a steep rise in the need to rapidly deliver high quality software solutions to meet everchanging market demands and manage fierce competition. In tandem, IT companies supporting these organizations, have switched to agile methodology to keep up with the growing demand for accelerated software delivery.

Understanding Agile

The agile model, a combination of incremental and iterative processes, originated to speed up software delivery, ensure customer satisfaction, and offer the flexibility to adapt and improve continuously. Under the agile model, business requirements were broken down into smaller chunks, enabling the development team to deliver software in shorter iterations, far more regularly. As agile methods gained ground, new automation technologies and streamlining processes began to crop up, allowing development teams to integrate smaller chunks of code continuously.

However, these smaller chunks of code had to be tested continuously so that high-quality code could be integrated several times into the trunk. While this increased the software development efficiency, it burdened the downstream testing team and operations team with continuous testing and deployment. In turn, the delivery of the overall software product got delayed, thereby defeating the agile methodology's entire purpose.

The journey from Agile to DevOps

Soon, organizations realized that the entire software development lifecycle should be treated as a single process, and most of it should be automated. The development and operation teams working in silos were merged, leading to the emergence of DevOps.

DevOps is the amalgamation of practices, cultural philosophies, and tools that enables an organization to deliver applications/services at high velocity and improve products consistently. Organizations that have moved away from traditional software development processes towards DevOps reap the benefits of faster go-to-market and increased customer satisfaction.

Elements of DevOps: CI/CD

Continuous integration is a software development practice where developers merge code changes regularly into a central repository which is then used to run automated builds and tests. The idea behind continuous integration is to identify and address bugs faster, improve quality, and reduce the time to market.

Continuous delivery is a software development practice wherein code changes are automatically built, tested, and integrated for release, taking continuous integration a step further by deploying code changes to a testing/production environment after the build phase.

Continuous deployment is another widespread DevOps practice wherein each new build is automatically deployed whenever any significant changes are made to the code.

Need for metrics

While many organizations have adopted agile and DevOps, measuring and managing software development has never been easy without defining metrics, particularly as the process becomes more complex and decentralized. In larger organizations where multiple, geographically diverse teams work on the same application, it becomes challenging to identify process delays and dependencies.

Measuring the effectiveness of DevOps with DORA Metrics

DORA metrics play an important role in measuring the team’s performance. A result of years of research conducted by the DevOps Research and Assessments (DORA) team led to the following four  DORA metrics: deployment frequency, lead time for changes, time to restore service, and change failure rate.

DORA metrics help leaders measure software delivery throughput (time to market) and quality (reliability).These metrics enable informed, data-driven decisions leading to continuous improvements that can be further linked to measurable digital transformation goals.

Challenges of Agile and DevOps

While the integration of DevOps and agile have enabled organizations to stay ahead of the market, the new approach to software development is not without its downsides. It is particularly challenging when teams start to operate at speed and scale. Some of the key challenges faced are:

• Absence of a single source of truth for better visibility
• Delayed feedback loops prolong course correction
• A diverse set of tools used; each tool operates in silos with its own reporting
• Manual data collection and metric calculation is cumbersome, error-prone, and not real-time

Telemetry: Automated monitoring and measuring for SDLC excellence

It is possible to move beyond these challenges with continuous automated data collection, measuring, and monitoring early in SDLC, right from story and code to deployment. This left shift in the software development cycle will make it much easier for DevOps teams to work seamlessly, identify bottlenecks, and take corrective action early.

Agile and DevOps are ongoing processes that require consistent feedback to improve. All software development lifecycle (SDLC) stages like requirement gathering, coding, building, testing, deployment, and release can be improved by continuously measuring and monitoring key metrics early in the SDLC.

Telemetry is the automatic recording and transmission of data from remote or inaccessible sources to an IT system in a different location for monitoring and analysis. It facilitates constant monitoring and presentation of charts to stakeholders in real-time to enable quick identification of issues. It allows teams to fix problems early preventing them from turning into complications that may be expensive to resolve later. With telemetry, it is even possible to determine key business and technology metrics patterns and create alerts if anomalies happen.

Leading vs. lagging indicators

Leading indicators are like inputs and measure activities necessary to achieve goals. Though they may be difficult to measure, they are easy to influence directly for achieving better results. Lagging indicators are like outputs, and they measure the actual results. Though they are easy to measure, it isn't easy to improve upon them directly.

DORA metrics that focus on time to market and quality fall under the category of lag metrics. However, multiple factors impact them indirectly. Lead metrics can measure these factors, and if improved upon continuously, they will directly impact the lag metrics of velocity and quality.

In the context of the application, organizations must identify fundamental performance indicators and define the lead metrics. The table below is a good starting point to define lead metrics. While it is not exhaustive, it can be used for creating baselines and observing the trends. By determining the trend's direction, we can understand if teams are moving on the right path.

Automate data collection, measurement, and presentation

Organizations that want to get started should identify the performance indicators to be measured, i.e., define the lead metrics. As a next step, they should identify various SDLC tools where data can be collected. Collected data needs to be transformed and stored. Some of the data collected can be presented directly while the rest needs to be calculated. Post calculation, the metrics data should be visually presented as charts for teams to evaluate and derive insights about progress and bottlenecks impeding them from ascending towards the lag metrics.

The above process of data collection, calculation, and presentation should be entirely automated so that teams can get continuous insights for the defined metrics. Also, telemetry should be plugged in early on in the SDLC so that teams have access to insights and can take corrective action.

The architecture diagram below represents a simple 3-tier microservices-based implementation leveraging existing visualization platforms like Grafana.

Why telemetry?

• The consolidated view of SDLC data from the telemetry dashboard supports data-driven decision making. The telemetry dashboard can also provide enterprise-wide robust program governance to improve SDLC quality and reliability.
• The telemetry dashboard can offer deep real-time insights across the software development lifecycle for faster feedback loops and help stakeholders make informed decisions for better releases.
• Organizations that invest in telemetry early in the SDLC save weeks of effort reporting DevOps and SDLC metrics, leading to faster turnaround time, improved decision-making, and greater transparency.

Closing Thoughts

The left shift of telemetry in the Agile DevOps journey is truly remarkable. Access to telemetric data in the Pre-prod environment has enabled the DevOps team to automate feedback and monitor applications in real-time. As a result, teams can easily visualize and resolve problems earlier as they arise, ensuring accelerated software delivery and improved time-to-market.