A decentralized identity rollout can pass every protocol test and still make verification slower, exclude users without a compatible wallet, or disclose more data than the old flow. Product managers should fund it as a measured replacement for a specific verification task, not as an identity-platform upgrade. The launch decision should depend on completed tasks, disclosure, and recovery—not the number of credentials issued.
Credential issuance is not evidence that users can finish the task
Start with one task that already has a baseline: for example, proving eligibility to access a restricted feature. Define success as an eligible person reaching that feature with a valid decision, within an agreed time, without support intervention. Count everyone assigned to the new flow, including people who never install a wallet. Excluding them would make wallet adoption look like verification success.
I would file Decentralized Identity and Privacy for Software Developers under background reading rather than acceptance evidence, because architectural possibilities cannot tell a product manager how many assigned users finish this particular task. The primary metric should be eligible completions per assigned user. Pair it with median and p95 time from “prove eligibility” to a usable decision, abandonment by step, and the share of attempts requiring a fallback. A fast presentation screen is irrelevant if people spend minutes acquiring the credential beforehand.
Instrument the entire journey: assignment, wallet handoff, consent screen, presentation returned, verification decision, and access granted. An OpenTelemetry span can connect stages inside your service, while Prometheus counters and Grafana panels can show aggregate outcomes. Keep the span’s identifier scoped to one attempt; do not propagate it into a credential or reuse a DID as an analytics identifier. Publish counts separately for first-time and returning attempts, and for supported wallet and device combinations, because a good overall rate can hide a broken route.
As an initial acceptance bar to tune against the existing flow, require at least 95% of eligible assigned users to reach a decision without contacting support. That figure is a planning choice, not an observed result. Report its numerator, denominator, exclusions, and confidence interval alongside it. Run Playwright checks for deterministic screen transitions and k6 for load, but do not substitute either tool’s scripted completion rate for human completion: scripts already have compatible wallets and know which button to press.
I would not make “credentials issued” a launch KPI, because an issuer can improve it without improving access, privacy, or recovery. Track issuance as a diagnostic stage instead: if issuance rises while completed verifications stay flat, the next sprint belongs to handoff and presentation, not issuer throughput.
Privacy needs failure tests with observable outcomes
“We store less data” is too vague to test. Specify the information the verifier needs—perhaps an eligibility result rather than a date of birth—and list every place a rejected presentation or debugging session might leave a copy. Then make privacy an acceptance criterion: the verifier receives no unnecessary claim, application logs contain no credential payload, and two unrelated presentations cannot be joined through a product-generated identifier.
Use synthetic credentials with distinctive canary values in automated checks. Present them successfully, trigger a validation error, time out the request, and ask support to investigate. Search application logs, OpenTelemetry attributes, error reporting, analytics exports, and support tickets for the canary. A clean search does not prove absolute privacy, since an untested destination may still exist; a hit identifies a concrete defect with an owner. Review wallet-facing text as well, because a technically minimal disclosure is not meaningful consent if the screen obscures what will be sent.
Protocol choices affect what these tests can assert. W3C Verifiable Credentials Data Model v2.0 describes the credential model; OpenID4VCI 1.0 and OpenID4VP 1.0 address issuance and presentation; SD-JWT, specified in RFC 9901, supports selective disclosure when the chosen credential profile uses it. None of those names alone establishes that your implementation discloses the minimum: test the actual request and returned claims. With OpenID4VP, check the intended verifier, the nonce, and the requested claims; never log the raw presentation while diagnosing a response_mode=direct_post failure.
Set a provisional rule of zero canary appearances across the destinations included in a seeded test run. That is a test gate to refine as integrations change, not a measured claim of zero real-world leakage. Give the gate an explicit boundary: which services were searched, how long logs were retained, and whether third-party error tools were included. Also inspect invalid and revoked credentials, since exception paths often receive less scrutiny than successful ones.
A privacy metric needs a denominator too. Report the proportion of tested requests that asked for an unnecessary claim, and the proportion of seeded failure paths that retained a canary. If a team cannot say how many paths it exercised, “no leaks found” describes its search effort more than its product.
A randomized comparison beats a protocol-only launch review
Compare two ways to complete the same eligibility task. An existing account with an eligibility record and WebAuthn sign-in wins when most users already have accounts and the service is the only verifier: it avoids wallet setup and reuses established recovery, at the cost of keeping an eligibility record in the service. An OpenID4VP credential presentation wins when eligible users already hold a usable credential and need to present it to multiple independent verifiers: it can reduce repeated collection, at the cost of wallet compatibility work, new support cases, and dependence on issuers.
Neither option wins by declaration. Randomly assign eligible traffic before the user sees a route, preserve a safe fallback, and analyze results by original assignment rather than by the route people eventually complete. Otherwise, users who struggle with the wallet disappear into the baseline group and make the new option look better. If random assignment is not permitted, use a staged rollout with comparable cohorts and state the selection bias; do not call its difference a causal effect.
Use Decentralized Identity and Privacy for Software Developers to align the team’s vocabulary, then put a written experiment contract ahead of implementation, because a shared vocabulary does not settle which outcome merits a release. The contract should name the eligible population, assignment point, fallback behavior, outcome window, and stop conditions. Security rejection rates and support contacts belong beside completion, since a design that completes more requests by accepting invalid proofs is not an improvement.
The following Node.js example runs as written. Its counts and latencies are invented to illustrate a gate; they are not pilot measurements. The completion gap is calculated from all assigned users, while p95 is an input that a real pilot would calculate from attempt timings.
const arms = [
{ name: "account", assigned: 120, completed: 108, p95ms: 830 },
{ name: "wallet", assigned: 120, completed: 104, p95ms: 1900 }
];
const [base, wallet] = arms;
const gap = (base.completed / base.assigned - wallet.completed / wallet.assigned) * 100;
console.log(`Completion gap: ${gap.toFixed(1)} percentage points`);
console.log(`Wallet p95: ${wallet.p95ms} ms`);
if (gap > 2 || wallet.p95ms > 2000) process.exitCode = 1;
The two-percentage-point completion tolerance is a starting limit to recalibrate against baseline variation and the value of reduced data collection. The 2,000-millisecond p95 limit is likewise a draft user-experience budget, not a universal standard. With these illustrative inputs, the completion gate fails even though the wallet latency gate passes; that is a reason to investigate abandonment before expanding traffic. In production, keep an audit of metric definitions and missing events, because telemetry loss can otherwise masquerade as user abandonment.
Interoperability has a denominator, not a checkbox
“Supports OpenID4VP” is not a useful release statement unless it names tested combinations. Build a matrix of issuer, credential format, wallet version, device, and verifier configuration, then count valid presentations accepted out of valid presentations attempted in each supported combination. Record invalid presentations rejected separately. A verifier that accepts everything can post an impressive acceptance rate while failing its security purpose.
Keep the first matrix deliberately small. For example, choose two issuers and two wallets as an illustrative scope limit, then add combinations only when the first set has reliable end-to-end evidence. W3C DID Core v1.0 may matter if your selected method uses DIDs, but adding a DID method solely to claim broader interoperability creates another resolution and recovery path to maintain. Test the credential format you intend to ship rather than assuming all conforming implementations exchange every format.
Distinguish incompatibility from ordinary user failure. Record whether the wallet could open the request, whether the user approved it, whether a presentation arrived, and whether signature, status, audience, or nonce validation failed. Keep those event names coarse enough to avoid retaining claims. This breakdown tells a PM whether the next increment is wallet support, issuer coordination, verifier fixes, or clearer user instructions.
Prometheus documentation publishes a default global scrape_interval of 1m. That published default is fine for many aggregate dashboards, but it does not define how long an individual verification took; calculate duration from attempt events before aggregation. If an OpenTelemetry trace crosses a component controlled by another party, review what attributes travel with it, because observability can defeat the privacy goal it was meant to validate.
Put accessibility in the matrix rather than treating it as polish. Test the wallet handoff and fallback with keyboard navigation and screen readers against WCAG 2.2 criteria. Count people who cannot complete the task through the offered route, because a working cryptographic exchange is not a working product for someone who cannot reach the approval control.
A narrow release gate makes the roadmap honest
Scope the first release around one claim, one verification task, named wallet-and-issuer combinations, and an existing fallback. Fund telemetry and canary tests in the same increment as protocol integration, because adding them after launch loses the baseline and lets privacy regressions enter unnoticed. Reserve explicit time for rejection-path testing, support scripts, and recovery: users will encounter expired credentials and changed devices even if a happy-path demo never does.
Write separate go/no-go decisions for task completion, unnecessary disclosure, invalid-proof rejection, and supported-combination coverage. Do not average them into a single “identity health” score, because strong latency cannot compensate for a leaked credential or an accepted invalid proof. A failed gate should point to a smaller next piece of work, not automatically authorize a larger platform rebuild.
First, ask the team to instrument the existing eligibility flow for one complete measurement window and write down its assignment denominator, completion rate, p95 duration, support contacts, and retained data. Then choose the single credential route that can be tested against it. If the baseline cannot be measured, scope that repair before scoping decentralized identity; otherwise, the rollout has no defensible definition of working.
