What KPIs prove the VS Code update improved team productivity without increasing risk?

Potential change-freeze windows

• Thursday, October 2nd: Mahatma Gandhi Jayanti and coincides with Dussehra.
• Saturday, October 11th - Sunday, October 12th: October 11th is a regional holiday in many states, Saturday and October 12th is a Bank Holiday across all of India for Dussehra.
• Monday, October 20th: Diwali/Deepavali

If you’re trying to roll out Argo CD but don’t want the whole org playing with it on day one, the trick is to combine Argo CD’s RBAC + AppProjects with policy enforcement (OPA/Gatekeeper/Kyverno) and a little bit of feature-flag thinking at the pipeline level.

When you flip on Vertex AI for your org, support teams are suddenly going to get tickets that look very different from the usual login not working stuff. Instead of deep-diving ML internals, the goal is to train support to recognize patterns, give clear responses, and escalate to the right folks fast.

Productivity KPIs (did the update make teams faster?)

• Deployment Frequency- How often teams are shipping new builds compared to before. If pods and apps can roll faster with new pipelines, this should go up.
• Lead Time for Changes- Time from code commit to deployed on OpenShift. If the new update improved CI/CD integrations, this number should shrink.
• Mean Time to Recovery (MTTR)- How quickly a failed deployment rolls back or recovers. A modern OpenShift update (esp. if you got GitOps/ArgoCD features) should make this near-instant.
• Self-Service Actions- No of support tickets for cluster operations (namespace creation, RBAC requests, image builds). If the update added automation/self-service, those tickets should drop.
• Build/Deploy Success Rate- % of builds that succeed first time. A cleaner CI/CD integration after upgrade = fewer broken builds.

Risk/Operational KPIs (prove you didn’t add fragility)

• Change Failure Rate- % of deployments that lead to hotfix/rollback. If productivity went up but this stays flat (or drops), you’ve proven no extra risk.
• Cluster Availability- Measured via SLO/uptime of the control plane and workloads. No degradation means the update didn’t destabilize infra.
• Security Findings- No of image scans or compliance checks failing pre vs post-upgrade. You don’t want to speed things up by cutting corners on CVEs.
• Policy Violations- Count of denied builds/deploys by OPA/Gatekeeper/ACM. If policy adherence stays consistent, risk hasn’t increased.
• Resource Efficiency- CPU/memory per workload vs. baseline. Helps prove you didn’t buy productivity at the cost of runaway infra spend.

You must utilize Audit Streaming to send Azure DevOps logs to an intermediary service, such as Azure Event Hubs or a Log Analytics workspace, in order to export them to a SIEM with least-privilege scopes.  Giving your DevOps organization direct, high-level access is avoided with this method.  After that, you will allow the SIEM tool to use the intermediary service rather than Azure DevOps directly.

To restrict Sourcegraph features to a pilot group, you need to use a combination of native Sourcegraph policy controls and external feature flag management. This allows for granular, user-level control that applies both to Sourcegraph's built-in features and to your codebase, which Sourcegraph indexes.

• Method: Use a feature flag provider

This is the most flexible approach, as it decouples the feature release from code deployment and provides fine-grained control based on user attributes. A provider like LaunchDarkly, Harness, or Flagsmith is recommended.
Setup process:

• Define user groups and attributes. In your feature flag provider, create a new group for your Sourcegraph pilot users. Define attributes that your Sourcegraph instance can query, such as a user ID, email address, or group membership.
• Instrument your code. Use the feature flag provider's Software Development Kit (SDK) to wrap the Sourcegraph features you want to restrict in conditional logic.
• Deploy your code. Deploy the new code with the feature flags in a disabled state. This makes the feature invisible to all users.
• Target the pilot group. In your feature flag provider's admin panel, create a rule to enable the feature flag specifically for the pilot user group. This allows you to turn the feature on or off for the pilot group instantly, without a code redeployment.