CY 2024 Real World Testing results report for Kodjin

General Information

Summary of Testing Methods and Key Findings

The Real-World Testing (RWT) methods were designed to evaluate the performance, interoperability, and usability of the certified Kodjin FHIR Server in real-world settings.These methods focused on ensuring the product meets the requirements of the ONC Health IT Certification Program.

The findings indicate that the Kodjin FHIR Server performed as expected under real-world conditions, meeting ONC certification standards for interoperability and usability. Identified areas for improvement  will guide future updates to enhance system performance and the end-user experience.

Standards Version Advancement Process (SVAP) Updates

Indicate as to whether optional standards, via SVAP and/or USCDI, are leveraged as part of the certification of your health IT product(s).

• [ ] Yes, I have products certified with voluntary SVAP or USCDI standards. (If yes, please complete the table below).
• [X] No, none of my products include these voluntary standards.

Care Settings

The Kodjin FHIR Server is not limited to specific care settings; it can be used across various healthcare practices.

Metrics and Outcomes

The testing was conducted throughout the calendar year 2024 across multiple healthcare settings. Data collection was facilitated through Kodjin FHIR server services, leveraging tools like Prometheus, Grafana, Loki stack, and specialized Kodjin FHIR server metrics designed to capture relevant performance and operational information.

Measure 1

Measurement / Metric:

Count of FHIR API requests by SMART apps and error rate

Measure description: 

Usage of the  FHIR API by SMART apps demonstrates compliance with (i)(A) Responding to requests for a single patient’s data according to the standard adopted in § 170.215(a)(1) and implementation specification adopted in § 170.215(a)(2)

Associated Criteria:

• 170.315(g)(10) Standardized API for patient and population services

Outcomes:

Review of audit logs of requests by smart apps showed that users did not use smart apps during the gathering period.

Observations:

It was noted that this functionality is not yet widely used by our clients.

Challenges Encountered (if applicable):

No challenges were discovered during testing.

Measure 2

Measurement / Metric:

Count of the bulk export requests and error rate.

Measure description: 

Using the bulk export operation demonstrates compliance with (i)(B) Responding to requests for multiple patients’ data as a group according to the standard adopted in § 170.215(a)(1).

Associated Criteria:

• 170.315(g)(10) Standardized API for patient and population services.

Outcomes:

The measured error rate was 0%, indicating that all requests were successfully processed. 

Observations:

Throughout the measurement period, users did not report any issues related to this functionality.

Challenges Encountered (if applicable):

No challenges were discovered during testing.

Measure 3

Measurement / Metric:

Authentication flow with Patient Selection.

Measure description: 

1. Number of requests for a patient ID or token
2. Number of requests that provided sufficient information to generate a valid response
3. Number of follow-up requests made using the provided patient ID or token.

Associated Criteria:

• 170.315(g)(7) Application access — patient selection.

Outcomes:

1. 5270
2. 512
3. 28756

Observations:

Throughout the measurement period, users did not report any issues related to this functionality.

Challenges Encountered (if applicable):

No challenges were discovered during testing.

Attestation

This Real World Testing results report is complete with all required elements, including measures addressing all certification criteria and care settings. The information presented is up to date and fully meets the health IT developer’s Real World Testing requirements.

Authorized Representative Name: Svitlana Vedmed

Authorized Representative Email: [email protected]

Authorized Representative Signature:

Date: 31/12/2024