Sources Sought Notice – Project Argos Agentic AI Proof of Concept (PPC)

Page 1 of 4 DRAFT - Statement of Work (SOW) – SOURCES SOUGHT NOTICE ATTACHMENT MARKET RESEARCH PURPOSE ONLY 2.1 DRAFT - STATEMENT OF WORK To achieve the purpose stated above, the Contractor shall perform the tasks below but is not limited to the following: Task 1 - Project Initiation, Requirements, and Regulatory Traceability • Conduct kickoff with & Testing Division and other Office of Standards, Certification, and Analysis (OSCA) staff and contracting stakeholders as appropriate. • Develop a requirements traceability matrix mapping 45 CFR 170.404(b)(2), § 170.215(a), § 170.315(g)(10), § 170.556, § 170.580, and secondary documentation requirements to pipeline tests and evidence artifacts. • Define the initial monitoring universe, including all active CHPL listings certified to § 170.315(g)(10), unless the Government identifies a specific subset of products certified to (g)(10) for initial prioritization. The Government recognizes that changes to the initial cohort list may require additional time and funding to be provided to the contractor. • Define issue taxonomy, severity model, human review checkpoints, and decision rights for suspected non-conformities and compliance concerns. • Deliver project management plan, integrated schedule, and stakeholder engagement plan. Specific Point of Contacts will be provided by ONC and then explicitly listed in the stakeholder plan. Task 2 – Administrative Task A monthly status report will be provided to show the progress of the contracted work. This report shall include: • Progress of each task including timeline of events. • Report any blockers that are prohibiting or delaying progress. • Expected completion dates for remaining work to be done. Task 3 - Data Source Discovery and Ingestion • Inventory public data sources, including CHPL product and certification data, Service Base URL List attributes, developer-published FHIR Bundles, Lantern data or APIs, CMS National Plan and Provider Enumeration System (NPPES) or other public organization reference data as approved, and relevant ONC Certification Program resources. -- 1 of 4 -- Page 2 of 4 • Build ingestion connectors for authoritative public sources and preserve raw source artifacts, timestamps, request metadata, response headers, and hashes for reproducibility. • Normalize developer, product, version, certification criterion, CHPL ID, service base URL list, organization, endpoint, and validation result records into a common data model. • Implement data lineage and source-of-truth precedence rules, including how the system resolves conflicts between CHPL metadata, developer-published artifacts, and endpoint observations. • Produce an ingestion data dictionary and source inventory. Task 4 - Agentic Pipeline Architecture and Governance • Design the end-to-end pipeline architecture, which may include ingestion agents, validation agents, endpoint observation agents, evidence assembly agents, triage agents, reporting services, and human review work queues. • Use deterministic validators for legal or standards-based pass/fail determinations wherever possible. Artificial Intelligence (AI)/agentic components may summarize, classify, reconcile, and draft, but shall not make final compliance determinations without human review. • Develop production ready AI agent tools capable of completing required tasks. Software development should include commercial best practices, including elements like prompts, tool policies, guardrails, retry logic, rate limits, error handling, and audit logs for agentic components. • Document model selection, evaluation methods, known limitations, fallback procedures, and version control for prompts, models, validators, and rules. • Implement a secure deployment pattern aligned with Government hosting and security requirements. Task 5 - 45 CFR 170.404(b)(2) Publication and Conformance Testing • Detect whether each applicable CHPL listing includes a publicly accessible service base URL list or Bundle link. • Fetch developer-published endpoint lists and classify common availability failures, including URL not found, timeout, redirects that obscure the published location, malformed responses, inaccessible content, or non-machine-readable content. • Validate that service base URLs are represented as FHIR R4 Endpoint resources and related organization details as FHIR R4 Organization resources. • Validate that Endpoint and Organization resources are collected into a FHIR Bundle and that Organization resources include name, location, facility identifier, and references to applicable endpoint resources. -- 2 of 4 -- Page 3 of 4 • Capture evidence of quarterly review or update status where discoverable from metadata, version history, timestamps, published content, or other Government- approved signals. • Create test cases for centrally managed and locally deployed API Information Source scenarios. Task 6 - Real-World Endpoint Observation • Move from list publication checks to observation of deployed endpoints by attempting safe, non-authenticated endpoint discovery where appropriate, including metadata and capability statement retrieval. • Assess whether endpoints respond with FHIR capability information sufficient to support monitoring of availability and standardization. • Monitor endpoint drift over time, including added, removed, moved, unreachable, or materially changed endpoints and organization mappings. • Compare observed endpoint behavior with developer-published artifacts and CHPL metadata to flag implementation gaps. • Avoid collection, request, storage, or use of patient data or protected health information unless separately authorized in writing by the Government. Task 7 - Evidence Case Generation and Triage • Build an evidence case package for each suspected issue, including source metadata, raw and normalized artifacts, validation outputs, screenshots where necessary, request/response logs, timestamps, hashes, severity, and rule traceabili…