Feedback EvaluationAsync analysis · Transcript → LLM → PDFfeedback.carecollaborative.cloud

Feedback Evaluation Architecture

End to end

User Lifecycle — Feedback Evaluation

Every internal route, protocol, and service boundary for this use case. Phases 1–3 and 7 are shared platform infrastructure; the middle phases are specific to this use case.

Phase 1
All Use Cases

Discovery & Frontend Delivery

User BrowserCloudflare DNSDNS over HTTPS

Domain resolves via Cloudflare DNS with DDoS protection and geo-routing to nearest edge PoP

Cloudflare CDNCloudflare PagesHTTPS · TLS 1.3

Static Next.js/React frontend served from Cloudflare Pages at the edge — zero origin round-trip for assets

Cloudflare PagesUser BrowserHTTP/3 · Brotli

Compressed SPA bundle delivered to browser with service worker caching for offline resilience

Phase 2
All Use Cases

Authentication & SSO

BrowserRuntime APIHTTPS · TLS 1.3

Frontend initiates auth flow → Cloudflare LB routes to Runtime API Server on Operator-managed VM

Runtime APIIdentity ProviderOAuth2 / OIDC / SAML

Runtime redirects to configured IdP (Auth0, Azure AD, Okta) for SSO authentication

Identity ProviderRuntime APIOAuth2 Callback

IdP returns authorization code → Runtime exchanges for tokens, creates session, assigns RBAC roles via OPA/Rego

Runtime APIPostgreSQLTLS · Private Link

Session token and user profile persisted to main PostgreSQL database on Azure Flexible Server

Phase 3
All Use Cases

Workspace & Agent Provisioning

BrowserRuntime APIREST · WebSocket

User creates or joins a workspace → API Server validates RBAC permissions and initiates provisioning

Runtime APITemporal ServergRPC · Internal

API starts a Temporal workflow in the org-specific namespace for durable execution with per-org isolation

Temporal ServerWorker PoolActivity Dispatch

Temporal schedules activities on available workers — agent environment provisioned with SSH, process management, health checks

WorkerTemporal DBHeartbeat · TLS

Worker sends heartbeat signals to Temporal. On failure, Temporal retries on next available worker — zero data loss

Phase 4C
Feedback Evaluation

Feedback Session Recording

Evaluator DashboardRuntime APIREST

Evaluator initiates a feedback session. WebRTC room created for recording the live evaluator-participant interaction

SessionTranscript CaptureWebRTC · Captions

Live captions and transcription captured during the feedback session. Evaluator provides verbal feedback to participant

Session EndTranscript ExportInternal

Complete transcript exported with speaker attribution, timestamps, and session metadata. No STT→LLM→TTS agent pipeline — this is human-to-human

Phase 5C
Feedback Evaluation

Async Transcript Analysis

EvaluatorUpload TranscriptHTTPS · Multipart

Evaluator uploads the transcript from the feedback session. System validates minimum content length for meaningful analysis

TranscriptAI Governance → LLMHTTPS · TLS 1.3

Dual parallel analysis: (1) Participant performance scored using the chosen grading template with configurable competencies, (2) Evaluator feedback quality assessed for clarity, specificity, and actionability

LLMEvidence MapperJSON Response

Analysis results mapped to evaluation form fields with confidence scores and direct evidence quotes from transcript

Phase 6C
Feedback Evaluation

Form Auto-Fill & Report Generation · WIP

EvaluatorTemplate UploadHTTPS · Multipart

Evaluator uploads PDF/DOCX evaluation form template. Document Intelligence extracts form structure, fields, and rating scales

Form Template + AnalysisAI Governance → LLMHTTPS · TLS 1.3

LLM maps competency scores and evidence to extracted form fields — generating appropriate values for each field with citations

Mapped FieldsPDF GeneratorInternal

System fills the original PDF form with mapped values (text fields, radio buttons, checkboxes). Evaluator downloads the completed evaluation form

Phase 7
All Use Cases

Data Persistence & Audit

Runtime APIPostgreSQL (Main)TLS · SQLC

User data, workspace state, agent config, AI budgets, and RBAC roles stored in the main database

Temporal ServerPostgreSQL (Temporal)TLS · Internal

Workflow execution history, activity state, and timer schedules persisted for durable replay

Temporal ServerPostgreSQL (Visibility)TLS · Internal

Search indices for workflow discovery — enables querying by status, type, org, and custom attributes

Runtime APIAudit LogsAppend-Only

Every user action logged to immutable cryptographic audit chain — tamper-evident, full retention

All ServicesPrometheus + OTelMetrics · Traces

Prometheus scrapes metrics from all Runtime instances. OpenTelemetry traces span across API, Temporal, and worker boundaries

Feedback Evaluation — Capability Summary

Internal Services
API Server (REST / DRPC / WS)
Temporal (durable orchestration)
AI Governance (MITM proxy)
Evidence Mapper (form-field mapping)
PDF Generator (form auto-fill · WIP)
External Services (Agnostic)
Cloudflare (CDN / LB / Pages)
Identity Providers (SSO)
LLM Providers (pluggable)
Document Intelligence (form extraction)