💬 EN

#

---

Is there any data that Kusto Query Language (KQL) can inherently query that I could use as a stand-in for metrics I might otherwise normally pull out of Azure DevOps (e.g. build pipeline pass/fail rates)

Kusto Query Language (KQL) can inherently query data from various Azure services, including some that can serve as alternatives to Azure DevOps metrics. Here are some options you could use as stand-ins for DevOps metrics:

1. Azure Monitor Logs: KQL can query Azure Monitor logs, which contain telemetry data from various Azure services¹. You can use this to analyze application performance, which could serve as a proxy for build quality.
2. Application Insights: KQL can be used to query telemetry data from Application Insights¹. This can provide insights into application performance and usage, which could indirectly reflect the quality of your builds and deployments.
3. Azure Data Explorer: If you’re ingesting your build pipeline data into Azure Data Explorer, you can use KQL to directly query this data²¹. This would allow you to analyze build pass/fail rates and other CI/CD metrics.
4. Log Analytics: If you’ve configured your Azure DevOps to send logs to a Log Analytics workspace, you can use KQL to query this data directly³. This would give you access to DevOps-specific metrics without needing to use the Azure DevOps API.
5. Azure Sentinel: If you’re using Azure Sentinel for security monitoring, you can use KQL to query security-related data¹. This could provide insights into the security aspects of your builds and deployments.

While these options don’t provide a direct one-to-one replacement for Azure DevOps metrics, they can offer valuable insights into your application’s performance, usage, and security, which are often reflective of the quality of your build and deployment processes.

---

How do I set up Azure devOps to send logs to a Log Analytics Workspace?

To set up Azure DevOps to send logs to a Log Analytics Workspace, you need to follow these steps:

1. Enable Azure DevOps audit log streaming to Azure Monitor logs.
2. Create a Log Analytics workspace if you don’t already have one. You can do this in the Azure portal by searching for “Log Analytics workspaces” and selecting “Create”⁴.
3. Configure Azure DevOps to stream audit logs to your Log Analytics workspace. This can be done through the Azure DevOps organization settings⁵.
4. Once configured, Azure DevOps will start sending audit logs to the specified Log Analytics workspace.
5. The logs will be stored in a table called “AzureDevOpsAuditing” within your Log Analytics workspace⁵.
6. You can then query these logs using Kusto Query Language (KQL) in the Log Analytics workspace. For example, to retrieve the top 10 audit log events from the last 3 days, you can use the following query⁵:

   AzureDevOpsAuditing
   | project TimeGenerated, ActivityId, ActorDisplayName, CategoryDisplayName, Details, OperationName, ProjectName,
   ScopeDisplayName, ScopeType, SourceSystem, Type, UserAgent
   | top 10 by TimeGenerated
   

By setting up this integration, you’ll be able to analyze Azure DevOps audit logs alongside other logs in your Azure environment, enabling better monitoring and troubleshooting capabilities.

---

I’ll turning this on help me see ADO pipeline executions, ADO repo branch creation, and such, over in Azure Monitor? Or only more surface-level things like users logging into ADO’s web portal?

Enabling Azure DevOps audit log streaming to Azure Monitor logs will provide you with visibility into a wide range of activities within your Azure DevOps environment, including pipeline executions and repository branch creation, not just surface-level actions like user logins.

When you set up this integration, you’ll be able to see detailed information about:

1. Pipeline Executions: You can track build and release pipeline runs, including success/failure rates and execution details⁶.
2. Repository Activities: Branch creation, deletion, and other source control management actions will be logged⁷.
3. Project and Repository Changes: Any modifications to projects or repositories will be recorded⁷.
4. Azure Pipelines Changes: Updates or modifications to your pipeline configurations will be captured⁷.
5. Service Connection Changes: Any alterations to service connections will be logged⁷.
6. Permission and Access Control Modifications: Changes to user permissions or access controls will be recorded⁷.

The logs will be stored in a table called “AzureDevOpsAuditing” within your Log Analytics workspace⁸⁷. You can then use Kusto Query Language (KQL) to analyze these logs. For example, to retrieve the top 10 audit log events, you could use a query like this:

AzureDevOpsAuditing
| project TimeGenerated, ActivityId, ActorDisplayName, CategoryDisplayName, Details, OperationName, ProjectName,
ScopeDisplayName, ScopeType, SourceSystem, Type, UserAgent
| top 10 by TimeGenerated

This integration provides a comprehensive view of activities within your Azure DevOps environment, allowing you to monitor, analyze, and set up alerts for various events, including pipeline executions and repository actions⁷⁹.

---

This worked great. How about dumping similar data out of GitHub into Azure Monitor?

Yes, you can stream GitHub audit log data into Azure Monitor, which provides similar functionality to what we discussed for Azure DevOps. Here’s how you can set it up:

1. GitHub Enterprise Cloud supports streaming audit logs to Azure Blob Storage. This feature allows you to send audit and Git events data from GitHub to an external data management system¹⁰.
2. To set up streaming to Azure Blob Storage:
   • Create a storage account and container in Microsoft Azure.
   • Generate a SAS (Shared Access Signature) token URL for the container.
   • In GitHub, navigate to your enterprise settings, then to Audit log > Log streaming.
   • Select “Azure Blob Storage” as the streaming destination.
   • Enter the SAS URL and save the configuration¹⁰.
3. Once set up, GitHub will stream audit log events to your Azure Blob Storage container. These events include a wide range of activities such as repository creation, permission changes, and user management actions¹⁰¹¹.
4. To make this data available in Azure Monitor:
   • Create a Log Analytics workspace if you don’t already have one.
   • Set up an ingestion process to move data from Blob Storage to Log Analytics. This could be done using Azure Functions or Azure Logic Apps.
5. Once the data is in Log Analytics, you can use Kusto Query Language (KQL) to analyze the GitHub audit logs alongside other data in Azure Monitor¹².
6. You can then create custom dashboards, set up alerts, and perform detailed analysis of your GitHub activity within the Azure Monitor environment.

This integration allows you to centralize your logging and monitoring, combining GitHub audit data with other Azure services for comprehensive visibility across your entire infrastructure¹³.

---

It looks like audit logs can also go from GitHub to Azure Event Hubs or to Splunk. And logs can stream from ADO to Azure Event Grid. What data flow architecture do you recommend if I want both version control systems’ audit logs to end up in both an Azure Log Analytics Workspace and also in Splunk? I feel like maybe an event-based model might be a good idea somehow, but maybe that’s premature optimization compared to just setting up a few redundant connectors.

To efficiently stream audit logs from both GitHub and Azure DevOps (ADO) to Azure Log Analytics Workspace and Splunk, I recommend the following data flow architecture:

GitHub Audit Logs

1. Stream GitHub audit logs to Azure Event Hubs.
2. Use Azure Functions to process the data from Event Hubs and send it to:
   • Azure Log Analytics Workspace
   • Splunk (via HTTP Event Collector)

Azure DevOps Audit Logs

1. Stream ADO audit logs to Azure Event Grid.
2. Use Azure Functions to process the data from Event Grid and send it to:
   • Azure Log Analytics Workspace
   • Splunk (via HTTP Event Collector)

Benefits of this architecture

1. Centralized event processing: Using Azure Event Hubs and Event Grid as central hubs allows for efficient distribution of data to multiple destinations.
2. Scalability: Event-based architecture can handle large volumes of audit log data efficiently.
3. Flexibility: Azure Functions provide the ability to transform and enrich the data before sending it to the final destinations.
4. Cost-effective: This approach minimizes data transfer costs and reduces the need for multiple direct integrations.
5. Real-time processing: Events can be processed and forwarded to both destinations in near real-time.
6. Reliability: Azure’s event-based services offer built-in retry mechanisms and dead-lettering for failed events.

While setting up redundant connectors directly from the sources to both destinations might seem simpler initially, the proposed event-based architecture offers better scalability, flexibility, and cost-effectiveness in the long run, especially when dealing with large volumes of audit log data from multiple sources¹⁴¹⁵¹⁶.

---

It seems like all that Azure Functions runtime might really add up, cost wise, compared to direct connectors that probably happen more at Microsoft’s expense and software maintenance headache under the hood.

While Azure Functions can indeed incur significant costs for heavy usage, it’s not necessarily more expensive than using direct connectors. The cost comparison depends on several factors:

1. Usage patterns: Azure Functions are billed based on execution time and number of executions. For infrequent or low-volume integrations, Functions can be very cost-effective, especially with the free grant of 1 million executions and 400,000 GB-s per month¹⁷.
2. Complexity: Functions allow for custom code execution, which can be more efficient for complex integrations that may require multiple connector actions.
3. Connector pricing: Some enterprise connectors have their own pricing, which can add up for high-volume scenarios¹⁸.
4. Scalability: Functions automatically scale based on demand, which can be more cost-effective for variable workloads compared to always-on solutions¹⁹.
5. Premium features: The Azure Functions Premium plan offers enhanced performance and VNET access, which might be necessary for certain enterprise scenarios¹⁷.

For a fair comparison, you’d need to consider:

1. The specific integration requirements
2. Expected volume and frequency of executions
3. Any additional features needed (like VNET access)
4. The pricing of specific connectors vs. custom code execution

In some cases, direct connectors might be more cost-effective, especially for simple, high-volume integrations. However, Functions offer greater flexibility and can be more economical for complex or variable workloads. It’s essential to analyze your specific use case to determine the most cost-effective solution.

---

I think this counts as “simple, high-volume integrations.”

You’re absolutely right, and I appreciate you pointing that out. For simple, high-volume integrations like streaming audit logs from GitHub and Azure DevOps to Azure Log Analytics Workspace and Splunk, direct connectors are indeed likely to be more cost-effective and simpler to maintain. Let’s revise the recommendation based on this insight:

Revised Recommendation

1. GitHub Audit Logs:
   • Use the native GitHub integration to stream logs directly to Azure Event Hubs
   • Configure Azure Event Hubs to forward events to Azure Log Analytics Workspace
   • Set up GitHub’s native Splunk integration for direct streaming to Splunk
2. Azure DevOps Audit Logs:
   • Use the built-in Azure DevOps feature to stream logs directly to Azure Log Analytics Workspace
   • Configure Azure DevOps to stream logs to Azure Event Grid
   • Set up an Azure Event Grid subscription to forward events to Splunk (using Splunk’s Azure Event Grid integration)

This approach leverages existing connectors and integrations, reducing the need for custom code and additional Azure services. It’s likely to be more cost-effective and easier to maintain, as it relies on Microsoft and GitHub’s managed services.

Key benefits of this revised approach:

• Lower operational costs
• Reduced complexity
• Less maintenance overhead
• Utilizes native integrations, which are typically optimized for performance and reliability

This solution aligns better with the “simple, high-volume” nature of audit log streaming, making it a more suitable choice for your scenario.

---