CY 2025 Real World Testing Results Report for Kodjin

Deviations from Plan: A deviation from the original RWT Plan 2025 has been identified in relation to the resubmission of the Plan Result. The initial plan did not account for the possibility of resubmission; however, circumstances required that the Plan Result be revised and resubmitted to reflect updated information. This deviation has been noted and does not affect the overall scope or objectives of the RWT Plan.

Metrics and Outcomes

The testing was conducted throughout the calendar year 2025 across multiple healthcare settings. Data collection was facilitated through Kodjin FHIR server services, leveraging tools like Prometheus, Grafana, Loki stack, and specialized Kodjin FHIR server metrics designed to capture relevant performance and operational information.

Measure 1

Measurement / Metric: 

FHIR REST API Interactions by SMART on FHIR Applications.

Associated Criteria: 

• 170.315(g)(10) Standardized API for patient and population services.

Measure Description:

Use of the FHIR API by SMART on FHIR applications demonstrates the system’s ability to respond to requests for a single patient’s data in accordance with the specification and §170.315(g)(10) requirements.

Sampling Frame: 

All FHIR API requests initiated by authenticated SMART on FHIR applications in de-identified production and mirrored production-like environments.

Denominator: 

Total number of FHIR API requests from SMART on FHIR applications.

Numerator: 

Number of successful requests.

Data source: 

Testing is based on de-identified production server logs and internal testing based on a mirrored production-like environment.

Testing Context and Approach:

Internal testing was conducted using an open-source SMART on FHIR application (https://github.com/smart-on-fhir/patient-browser) deployed against a mirrored production-like environment that replicates the configuration, data structures, and access controls of the live production Kodjin FHIR server. 

Testing was performed on a quarterly cadence to verify continued conformance with §170.315(g)(10) requirements.

Each quarterly test cycle included the following workflow elements:

• • Authentication and authorization flows using SMART on FHIR launch sequences
• • Execution of FHIR REST API read and search interactions against the patient-level data resources required by the certification criteria
• • Verification of correct HTTP response codes, response payloads, and conformance to the applicable FHIR profiles
• • Review of server-side logs to confirm requests were received, processed, and responded to correctly.

Testing was performed by internal technical staff with knowledge of the FHIR specification and the certified system’s configuration.

Metrics with Quantitative Detail

Because no end-user SMART on FHIR application activity was recorded in production during the reporting period, real-world production metrics are zero:

• • Denominator (production): 0 requests
• • Numerator (production): 0 successful requests
• • Production success rate: N/A (no requests recorded).

Internal testing results from quarterly test cycles conducted during the reporting period:

• • Total Kodjin FHIR API requests executed: 641
• • Successful requests: 641
• • Total test cycles conducted: 4 
• • Failed requests: 0
• • Internal test success rate: 100%.

Scope and Representativeness

The internal test environment is a mirrored production-like environment maintained to closely reflect the production Kodjin FHIR server in terms of software version, configuration, security settings, and data model. The following design choices support the representativeness of internal testing:

• • The open-source SMART on FHIR application used for testing is a publicly available reference client that exercises standard SMART App Launch and FHIR REST API interactions consistent with how third-party SMART applications would interact with the server in production.
• • Test scenarios cover the patient-level FHIR resource interactions required under §170.315(g)(10), including read and search operations on resources such as Patient, AllergyIntolerance, Condition, MedicationRequest, Observation, and others defined in the applicable USCDI and US Core profiles.
• • The mirrored environment uses synthetic patient data, generated using https://synthea.org/,  that reflects the structure and complexity of real patient records, enabling realistic simulation of production workloads.
• • While the internal test environment cannot fully replicate all production variables, the controlled quarterly testing provides consistent, repeatable evidence of API conformance.

Efforts Toward Real-World Adoption

Although no customers utilized SMART on FHIR applications against the production Kodjin FHIR server during the reporting period, the following activities were undertaken or are planned to support real-world adoption:

• • Developer documentation and API onboarding materials have been made available to customers and third-party developers, including but limited to:
  • ○ Service Base URLs
  • ○ Standardized API for patient and population services
  • ○ Security and authorization flow 
• • The organization is monitoring customer interest and inquiries related to SMART on FHIR integration and third-party application access.
• • Staff responsible for customer implementations have been informed of the availability of SMART on FHIR capabilities to facilitate awareness during customer onboarding and expansion discussions.

Remediation Steps Taken: 

No remediation was required as performance met expected outcomes.

Non-Conformities Reported to ONC-ACB: 

No non-conformities were identified or reported to the ONC-ACB during the 2025 testing period for this measure.

Observations: 

Similar to 2024, it was noted that this functionality is not yet widely used by our clients. The organization will continue quarterly testing to maintain readiness and will monitor whether increased customer demand or regulatory timelines accelerate adoption in future reporting periods.

Challenges Encountered (if applicable): 

No challenges were discovered during testing.

Measure 2

Measurement / Metric: 

FHIR REST API Interactions utilizing bulk-export specification.

Associated Criteria: 

• 170.315 (b)(10) Electronic Health Information Export.

Measure Description: 

This measure demonstrates the ability to respond to requests for multiple patients’ data as a group according to the HL7 FHIR Bulk Data Access (Flat FHIR) specification and §170.315(b)(10) requirements for exporting electronic health information.

The assessment reviewed the successful creation of export files, the completeness and accuracy of the exported EHI data, and the system’s reliability during export processes. 

Sampling Frame: 

All bulk export operations initiated through the FHIR API (both system-level and patient-level exports).

Denominator: 

Total number of EHI export requests initiated by users or applications during the evaluation period.

Numerator: 

Total number of successful EHI export requests completed without errors or data loss.

Data collection: 

Testing is based on de-identified production server logs and internal testing based on a mirrored production-like environment. Additionally, logs from Minio were used from production and mirrored production-like environments, where exported data is collected. 

Outcomes: 

• • Denominator: 96
• • Numerator: 96

Exported data were successfully saved and requested by the users or applications.

Remediation Steps Taken: 

No remediation was required as performance met expected outcomes.

Non-Conformities Reported to ONC-ACB: 

No non-conformities were identified or reported to the ONC-ACB during the 2025 testing period for this measure.

Observations: 

Throughout the measurement period, users did not report any issues related to this functionality. The results demonstrated consistent performance and successful execution of export requests in accordance with the required criteria.

Challenges Encountered (if applicable): 

No challenges were discovered during testing.

Measure 3

Measurement / Metric: 

Authentication flow with Patient Selection.

Associated Criteria: 

• 170.315(g)(7) Application access – patient selection.

Measure Description:

This measure evaluates the certified Health IT module’s ability to provide an API and supporting documentation that enables external applications to request a unique patient identifier from the certified Health IT module that can be used to request additional patient data. 

Three metrics are tracked:

1. 1. Number of requests for a patient ID or token
2. 2. Number of requests that provided sufficient information to generate a valid response
3. 3. Number of follow-up requests made using the provided patient ID or token.

Sampling Frame: 

All API requests to patient selection endpoints and subsequent data retrieval requests using patient-scoped tokens.

Denominator: 

Total number of authentication attempts with patient selection initiated by applications during the evaluation period.

Numerator: 

Total number of successful authentication and patient selection completed.

Data Source: 

De-identified authentication logs, token issuance records, and API request logs.

Outcomes:

Metric 1 – Total Requests for Patient ID or Token:

• • Denominator: 4745
• • Numerator: 4745

Metric 2 – Requests Providing Sufficient Information:

• • Denominator: 1095
• • Numerator: 1095

Metric 3 – Follow-up Requests Using Patient ID/Token:

• • Denominator: 58035
• • Numerator: 58035

Remediation Steps Taken: No remediation was required as performance met expected outcomes.

Non-Conformities Reported to ONC-ACB: No non-conformities were identified or reported to the ONC-ACB during the 2025 testing period for this measure.

Observations: 

Throughout the measurement period, users did not report any issues related to this functionality. Log analysis and measurement data indicate that not all users who received a token subsequently submitted requests for patient information.

Challenges Encountered (if applicable): 

No challenges were discovered during testing.

Attestation

This Real World Testing results report is complete with all required elements, including measures addressing all certification criteria and care settings. The information presented is up to date and fully meets the health IT developer’s Real World Testing requirements.

Authorized Representative Name: Svitlana Vedmed

Authorized Representative Email: [email protected]

Authorized Representative Signature: Svitlana Vedmed

Date: 02/03/2026