ADR: DES Data Provider Fetch API Architecture and Requirements

Purpose

This document defines the Plan 2 change where the shared parallel data-provider fetch function is implemented behind des-data-provider instead of being embedded independently in BF, PP, and PRC.

The new API gives decision services one generated REST client and one generated DTO model for fetching CDS, CRO, and experiment data. The implementation still calls the existing downstream providers, but orchestration, source-level failure classification, logging correlation, metrics, and executor ownership move into des-data-provider.

Scope

In scope:

Add a new aggregate data-provider endpoint in des-data-provider.
Represent the existing dataProviderFetchService.fetch(...) contract as OpenAPI.
Reuse the existing shared schema objects from des-shared-lib/src/main/resources/openapi-schema/data-provider/des_data_provider.yml:
#/components/schemas/CustomerData
#/components/schemas/ExperimentData
Return HTTP 200 for downstream CDS, CRO, and experiment failures, with failure details populated per requested source.
Return HTTP 400 for invalid aggregate API requests before any source call is attempted.
Propagate ce-id from BF, PP, and PRC into the new data-provider endpoint and log it as request-id in des-data-provider and pp/bf/prc models.
Generate BF, PP, and PRC request/response DTOs and REST clients from the OpenAPI contract instead of copying DTO classes by hand.

Key Design Decision

Invalid aggregate request input should return HTTP 400.

Examples:

accountId is missing, zero, or negative.
sources is missing or empty.
sources contains EXPERIMENT but experimentId is missing or blank.

These are client contract violations. Returning 200 with a source failure would make the request look accepted even though no valid source call can be made.

Downstream provider failures should return HTTP 200 with failure populated for each affected source.

Examples:

CDS returns 404 for a valid account id that is not found.
CRO returns 404 for a valid account id that is not found.
Experiment returns 404 for a valid account id and experiment id that has no experiment data.
CDS, CRO, or experiment times out.
CDS, CRO, or experiment returns 500, 503, 504, or another unexpected error.

This keeps the aggregate response source-keyed and lets BF, PP, and PRC preserve their existing local mapping behavior.

Proposed OpenAPI Contract

The contract below should be checked in as a YAML schema for code generation. The existing CustomerData and ExperimentData schema definitions must stay aligned with des-shared-lib/src/main/resources/openapi-schema/data-provider/des_data_provider.yml.

openapi: 3.1.0
info:
  title: des-data-provider aggregate fetch API
  version: 1.0.0
paths:
  /v1/data-provider/fetch:
    post:
      operationId: fetchDataProviderSources
      summary: Fetch one or more data-provider sources
      tags:
        - Data Provider
      parameters:
        - name: ce-id
          in: header
          required: true
          description: CloudEvent id propagated by the caller and logged by des-data-provider as request-id.
          schema:
            type: string
      requestBody:
        required: true
        content:
          application/json:
            schema:
              $ref: "#/components/schemas/DataProviderFetchRequest"
      responses:
        "200":
          description: Source-keyed fetch result. Per-source downstream failures are represented in the response body.
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/DataProviderFetchResult"
        "400":
          description: Invalid aggregate request. No source fetch is attempted.
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/ErrorResponse"
        "500":
          description: Unexpected aggregate API failure before a source-keyed result can be created.
          content:
            application/json:
              schema:
                $ref: "#/components/schemas/ErrorResponse"
components:
  schemas:
    DataProviderFetchRequest:
      type: object
      required:
        - accountId
        - sources
      properties:
        accountId:
          type: integer
          format: int64
          minimum: 1
        sources:
          type: array
          minItems: 1
          uniqueItems: true
          items:
            $ref: "#/components/schemas/DataProviderSource"
        experimentId:
          type: string
          description: Required when sources contains EXPERIMENT.
      allOf:
        - if:
            properties:
              sources:
                contains:
                  const: EXPERIMENT
          then:
            required:
              - experimentId

    DataProviderFetchResult:
      type: object
      required:
        - results
      properties:
        results:
          type: object
          additionalProperties: false
          properties:
            CDS:
              $ref: "#/components/schemas/CustomerDataSourceResult"
            CRO:
              $ref: "#/components/schemas/CustomerDataSourceResult"
            EXPERIMENT:
              $ref: "#/components/schemas/ExperimentDataSourceResult"
          description: Contains only the requested sources.

    CustomerDataSourceResult:
      type: object
      properties:
        value:
          anyOf:
            - $ref: "#/components/schemas/CustomerData"
            - type: "null"
        failure:
          anyOf:
            - $ref: "#/components/schemas/DataProviderFailure"
            - type: "null"

    ExperimentDataSourceResult:
      type: object
      properties:
        value:
          anyOf:
            - $ref: "#/components/schemas/ExperimentData"
            - type: "null"
        failure:
          anyOf:
            - $ref: "#/components/schemas/DataProviderFailure"
            - type: "null"

    DataProviderFailure:
      type: object
      required:
        - source
        - cause
        - message
      properties:
        source:
          $ref: "#/components/schemas/DataProviderSource"
        cause:
          $ref: "#/components/schemas/DataProviderFailureCause"
        message:
          type: string

    DataProviderSource:
      type: string
      enum:
        - CDS
        - CRO
        - EXPERIMENT

    DataProviderFailureCause:
      type: string
      enum:
        - ACCOUNT_ID_NOT_FOUND
        - TIMEOUT
        - OTHERS

    CustomerData:
      type: object
      properties:
        accountId:
          type: integer
          format: int64
        localeCode:
          type: string
        countryCode:
          type: string
        jurisdictionalCode:
          type: string
        programme:
          type: string
        initiatives:
          type: array
          items:
            type: string

    ExperimentData:
      type: object
      properties:
        id:
          type: string
        variant:
          type: string

    ErrorResponse:
      type: object
      required:
        - message
      properties:
        message:
          type: string

Failure Classification

des-data-provider must classify per-source failures as follows:

Downstream outcome	Aggregate HTTP status	Source failure cause
`404` from CDS, CRO, or experiment	`200`	`ACCOUNT_ID_NOT_FOUND`
REST client read timeout	`200`	`TIMEOUT`
`500`, `503`, `504`, or any non-404 HTTP error	`200`	`OTHERS`
Unexpected exception	`200`	`OTHERS`
Executor rejection	`200`	`OTHERS`
Invalid aggregate request	`400`	Not applicable

BF, PP, and PRC must also handle aggregate endpoint-level failures that prevent a 200 response. This includes Quarkus REST client timeout while calling /v1/data-provider/fetch, connection failures, and unexpected 5xx responses from des-data-provider.

Logging and Correlation

BF, PP, and PRC must:

Read the inbound CloudEvent ce-id.
Include ce-id as request-id in local logs.
Send the same ce-id header to message-id.

des-data-provider must:

Require or validate the message-id header at the aggregate endpoint boundary.
Log the received message-id as request-id.
Include request-id in logs for source scheduling, success, source failure classification, and aggregate response completion.

BF Happy Path

sequenceDiagram
    autonumber
    participant FRED as fred
    participant BF as bf-reward-decision
    participant DP as des-data-provider
    participant CDS as CDS
    participant CRO as CRO

    FRED->>BF: CloudEvent with ce-id=request-id
    BF->>BF: Log request-id
    BF->>DP: POST /v1/data-provider/fetch
ce-id=request-id
sources=[CDS,CRO]
    DP->>DP: Log request-id and validate request
    par Fetch CDS
        DP->>CDS: GET CDS by accountId
        CDS-->>DP: CustomerData(programme, initiatives)
    and Fetch CRO
        DP->>CRO: GET CRO by accountId
        CRO-->>DP: CustomerData(account/location fields)
    end
    DP-->>BF: 200 DataProviderFetchResult
    BF->>BF: Merge CRO account/location with CDS programme/initiatives
    BF-->>FRED: BF decision response

Request:

{
  "accountId": 123456,
  "sources": ["CDS", "CRO"]
}

Response:

{
  "results": {
    "CDS": {
      "value": {
        "accountId": 123456,
        "programme": "Gaming",
        "initiatives": ["free-bet"]
      },
      "failure": null
    },
    "CRO": {
      "value": {
        "accountId": 123456,
        "localeCode": "en-GB",
        "countryCode": "GB",
        "jurisdictionalCode": "UK"
      },
      "failure": null
    }
  }
}

BF Failure Path

sequenceDiagram
    autonumber
    participant BF as "bf-reward-decision"
    participant DP as "des-data-provider"
    participant CDS as CDS
    participant CRO as CRO

    BF->>DP: POST /v1/data-provider/fetch
ce-id=request-id
sources=CDS,CRO
    par Fetch CDS
        DP->>CDS: GET CDS by accountId
        CDS-->>DP: 404 Not Found
    and Fetch CRO
        DP->>CRO: GET CRO by accountId
        CRO--x DP: Read timeout
    end
    DP->>DP: Classify CDS=ACCOUNT_ID_NOT_FOUND and CRO=TIMEOUT
    DP-->>BF: 200 DataProviderFetchResult with failures
    BF->>BF: Map CDS account-not-found locally and fail decision on CRO timeout

Response:

{
  "results": {
    "CDS": {
      "value": null,
      "failure": {
        "source": "CDS",
        "cause": "ACCOUNT_ID_NOT_FOUND",
        "message": "No CDS data found for accountId 123456"
      }
    },
    "CRO": {
      "value": null,
      "failure": {
        "source": "CRO",
        "cause": "TIMEOUT",
        "message": "Timed out calling CRO for accountId 123456"
      }
    }
  }
}

PP Happy Path

sequenceDiagram
    autonumber
    participant FRED as fred
    participant PP as pp-reward-decision
    participant DP as des-data-provider
    participant CDS as CDS

    FRED->>PP: CloudEvent with ce-id=request-id
    PP->>PP: Log request-id
    PP->>DP: POST /v1/data-provider/fetch
ce-id=request-id
sources=[CDS]
    DP->>DP: Log request-id and validate request
    DP->>CDS: GET CDS by accountId
    CDS-->>DP: CustomerData
    DP-->>PP: 200 DataProviderFetchResult
    PP->>PP: Use CDS CustomerData in local mapping
    PP-->>FRED: PP decision response

Request:

{
  "accountId": 123456,
  "sources": ["CDS"]
}

Response:

{
  "results": {
    "CDS": {
      "value": {
        "accountId": 123456,
        "programme": "Gaming",
        "initiatives": ["price-promise"]
      },
      "failure": null
    }
  }
}

PP Failure Path

sequenceDiagram
    autonumber
    participant PP as pp-reward-decision
    participant DP as des-data-provider
    participant CDS as CDS

    PP->>DP: POST /v1/data-provider/fetch
ce-id=request-id
sources=[CDS]
    DP->>CDS: GET CDS by accountId
    CDS-->>DP: 503 Service Unavailable
    DP->>DP: Classify CDS=OTHERS
    DP-->>PP: 200 DataProviderFetchResult with failure
    PP->>PP: Throw according to current PP behavior

Response:

{
  "results": {
    "CDS": {
      "value": null,
      "failure": {
        "source": "CDS",
        "cause": "OTHERS",
        "message": "CDS returned 503 for accountId 123456"
      }
    }
  }
}

PRC Happy Path

sequenceDiagram
    autonumber
    participant FRED as fred
    participant PRC as prc-decision
    participant DP as des-data-provider
    participant CDS as CDS

    FRED->>PRC: CloudEvent with ce-id=request-id
    PRC->>PRC: Log request-id
    PRC->>DP: POST /v1/data-provider/fetch
ce-id=request-id
sources=[CDS]
    DP->>CDS: GET CDS by accountId
    CDS-->>DP: CustomerData
    DP-->>PRC: 200 DataProviderFetchResult
    PRC->>PRC: Map CustomerData to CdsResponse
    PRC-->>FRED: PRC decision response

Request:

{
  "accountId": 123456,
  "sources": ["CDS"]
}

Response:

{
  "results": {
    "CDS": {
      "value": {
        "accountId": 123456,
        "programme": "Gaming",
        "initiatives": ["progress-reward"]
      },
      "failure": null
    }
  }
}

PRC Failure Path

sequenceDiagram
    autonumber
    participant PRC as prc-decision
    participant DP as des-data-provider
    participant CDS as CDS

    PRC->>DP: POST /v1/data-provider/fetch
ce-id=request-id
sources=[CDS]
    DP->>CDS: GET CDS by accountId
    CDS-->>DP: 404 Not Found
    DP->>DP: Classify CDS=ACCOUNT_ID_NOT_FOUND
    DP-->>PRC: 200 DataProviderFetchResult with failure
    PRC->>PRC: Map ACCOUNT_ID_NOT_FOUND to CdsResponse.empty()

Response:

{
  "results": {
    "CDS": {
      "value": null,
      "failure": {
        "source": "CDS",
        "cause": "ACCOUNT_ID_NOT_FOUND",
        "message": "No CDS data found for accountId 123456"
      }
    }
  }
}

BF Experiment Path

BF experiment calls use the same aggregate endpoint and request only EXPERIMENT.

sequenceDiagram
    autonumber
    participant BF as bf-reward-decision
    participant DP as des-data-provider
    participant EXP as Experiment Provider

    BF->>DP: POST /v1/data-provider/fetch
ce-id=request-id
sources=[EXPERIMENT]
    DP->>DP: Validate accountId and experimentId
    DP->>EXP: GET experiment by accountId and experimentId
    EXP-->>DP: ExperimentData
    DP-->>BF: 200 DataProviderFetchResult
    BF->>BF: Apply BF experiment mapping locally

Request:

{
  "accountId": 123456,
  "experimentId": "bf-reward-v2",
  "sources": ["EXPERIMENT"]
}

Success response:

{
  "results": {
    "EXPERIMENT": {
      "value": {
        "id": "bf-reward-v2",
        "variant": "treatment"
      },
      "failure": null
    }
  }
}

Invalid request response:

HTTP/1.1 400 Bad Request
content-type: application/json

{
  "message": "experimentId is required when sources contains EXPERIMENT"
}

Valid request with no experiment data:

{
  "results": {
    "EXPERIMENT": {
      "value": null,
      "failure": {
        "source": "EXPERIMENT",
        "cause": "ACCOUNT_ID_NOT_FOUND",
        "message": "No experiment bf-reward-v2 found for accountId 123456"
      }
    }
  }
}

Client Responsibilities

BF must:

Request CDS and CRO for standard reward decisions.
Request EXPERIMENT with experimentId for experiment decisions.
Map ACCOUNT_ID_NOT_FOUND according to existing BF behavior.
Fail the decision on TIMEOUT and OTHERS unless an existing BF-specific fallback is explicitly retained.
Treat a requested source with neither value nor failure as a contract bug and throw IllegalStateException.
Handle Quarkus REST client timeout or endpoint-level failures while calling the aggregate endpoint.

PP must:

Request only CDS.
Return the fetched CustomerData on success.
Return new CustomerData() for CDS ACCOUNT_ID_NOT_FOUND.
Throw according to current PP behavior for TIMEOUT, OTHERS, aggregate endpoint timeout, or aggregate endpoint failure.
Treat missing value with no failure as IllegalStateException.

PRC must:

Request only CDS.
Map CDS success to CdsResponse.
Map CDS ACCOUNT_ID_NOT_FOUND to CdsResponse.empty().
Map TIMEOUT, OTHERS, aggregate endpoint timeout, or aggregate endpoint failure to PrcDecisionExceptions.
Treat missing value with no failure through PRC's current exception contract.

Code Generation Requirement

The new OpenAPI schema is the source of truth for:

Aggregate request DTO.
Aggregate response DTO.
DataProviderSource.
DataProviderFailure.
DataProviderFailureCause.
Aggregate REST client interface.

BF, PP, and PRC must use generated DTOs and client interfaces from the OpenAPI schema. They must not copy and paste these classes into each module.

Acceptance Criteria

des-data-provider exposes /v1/fetch.
The aggregate endpoint accepts ce-id and logs it as request-id.
BF, PP, and PRC propagate inbound ce-id to the aggregate endpoint.
Invalid aggregate requests return 400.
Valid aggregate requests return 200, including per-source downstream failures.
CDS and CRO may be fetched in parallel for BF.
PP and PRC request only CDS.
Experiment requests validate experimentId.
Source failures include source, cause, and message.
Clients are generated from the OpenAPI schema and do not copy DTOs manually.
Clients still handle Quarkus REST client timeout when calling the aggregate endpoint.