How to Leverage FHIR Bulk API for Data Extraction

09.21.2022

While the healthcare industry generates about 30% of the global data, there’s a lack of ready access and easy exchange of said data. This creates a fundamental barrier in transforming healthcare into a data-driven enterprise. The Health Level Seven Fast Healthcare Interoperability Resources (HL7 FHIR) standard aims to solve this issue. The FHIR standard is crucial in facilitating data standardization and interoperability to provide an easier way for data to be accessed, shared, and analyzed.

Since its first publication, FHIR has become widely adopted, in part thanks to the 21st Century Cures Act (US), which requires healthcare entities to have an application programming interface (API), providing easy access to all data elements of a patient’s electronic health record. As a result, FHIR is helping healthcare stakeholders solve the biggest challenges of health data interoperability and boost data-driven innovation.

One way FHIR helps streamline health data exchange is with the FHIR Bulk API, which allows you to extract large volumes of data to support various use cases across healthcare, research, and public health domains. In this article, we’ll take a closer look at the HL7 FHIR Bulk Data API, how it works, and why it makes such a difference. We’ve also put together a Bulk FHIR API Postman collection featuring examples of how to export data in the Kodjin FHIR Server. You can use this collection to test bulk export functionality yourself.

What Is the FHIR Bulk API?

The FHIR Bulk API provides a convenient way to download standardized health datasets from EHRs on a large scale. It allows certified health information technology owners to enable authorized clients to request and extract population datasets from cloud-based FHIR servers.

Unlike the regular FHIR REST API, bulk data export makes retrieving data possible in a single request rather than hundreds. In addition, the export request can be configured to define which data types should be exported and how to filter them.

To lower the load on system performance, the FHIR Bulk Data API specification enables asynchronous request processing, which also leverages a system-to-system SMART backend services authentication and authorization framework for security.

Exported datasets are stored in files and can be downloaded by the client when ready. The amount of time it takes to process a request depends on the size of the dataset and the number of FHIR resources. The data is in newline delimited JSON (NDJSON) format, which takes less space and is more convenient for large amounts of data than a regular searchset response.

How does the FHIR Bulk API work?

As previously mentioned, FHIR Bulk follows an asynchronous request pattern, meaning that once the export query has been sent, the server will indicate that the process has started. A client can check the status by polling from the Content-Location header returned by the server. The process is initiated by a kick-off request to one of the available export endpoints:

  • …/Patient/$export—exports data on all patients (works with all resources in Patient Compartment)
  • …/Group/[group id]/$export—export all data for patients in a specified group
  • …/$export—full system-scale export of all datasets

The extraction results can be filtered using query parameters described in the picture below. In addition, servers can limit what data is retrieved to comply with specific policies or regulations.

Key FHIR Bulk API query parameter

How to configure a kick-off request

Example of _typeFilter parameter use

A _typeFilter parameter can be used to filter resources that meet the specified criteria. The criteria is a search string with the same interpretation as if it was appended to the base URL and submitted to the FHIR search using REST API. 

Any valid search string can be used, including ones with modifiers and prefixes.

The example below demonstrates how to configure a kick-off request to export only patients from a group who had a reaction to immunization. In this case, _typeFilter contains a search immunization query with :missing=false modifier.

Example Request:
curl --location --request GET 'https://kodjin-staging.edenlab.dev//fhir/Group/0a60d2a2-38ce-49f6-ac45-42347193af50/$export?_type=Immunization&_typeFilter=Immunization%3Freaction-date:missing%3Dfalse' \
--header 'content-type: application/json' \
--header 'prefer: respond-async' \
--data-raw ''

Example of POST request use

Some requests may contain lots of filter parameters. In this case, it is convenient to use a POST request and supply filter parameters in the FHIR Parameters Resource in the body. The example below demonstrates how to export resources filtered by practitioner ID. The request contains the _typeFilter parameter for each resource type.

Example Request:
curl --location --request POST 'https://kodjin-staging.edenlab.dev//fhir/$export' \
--header 'content-type: application/json' \
--header 'prefer: respond-async' \
--data-raw '{"resourceType" : "Parameters",
  "parameter" : [
     {"name":"_since",
    "valueInstant": "2022-01-01T00:00:00Z"},
    {"name":"_type",
     "valueString": "Observation, Condition, Procedure, Immunization"},
    {"name":"_typeFilter",
     "valueString": "Observation?performer=Practitioner/9bac339d-ac3b-4715-bf9a-1dab1dec7fa2"},
    {"name":"_typeFilter",
     "valueString": "Condition?asserter=Practitioner/9bac339d-ac3b-4715-bf9a-1dab1dec7fa2"},
    {"name":"_typeFilter",
     "valueString": "Procedure?performer=Practitioner/9bac339d-ac3b-4715-bf9a-1dab1dec7fa2"},
    {"name":"_typeFilter",
     "valueString": "Immunization?performer=Practitioner/9bac339d-ac3b-4715-bf9a-1dab1dec7fa2"}
]
}'

Example of _elements parameter use

In some cases, you will need only a short set of fields for analysis instead of the entire resource. The example below demonstrates how to export only condition and observation codes using an _elements parameter.

Example Request:
curl --location --request GET 'https://kodjin-staging.edenlab.dev//fhir/$export?_type=Observation,Condition&_since=2022-07-13T00:00:00Z&_elements=code' \
--header 'content-type: application/json' \
--header 'prefer: respond-async' \
--data-raw ''

If the request were successful, the server would return an HTTP header with a content location, which you can check to see the request’s status. Once the extraction is finished, the completion response will contain a JSON manifest with the locations of the extracted data in files in the NDJSON format (some servers support the use of other formats as well). Each file contains one type of FHIR resource. 

The final step is to send a ‘GET’ file request to download the bulk data. The complete data extraction process is illustrated below.

Security is paramount to the sensitive nature of healthcare data. 

Therefore, the FHIR specification recommends implementing a SMART backend services authorization to secure bulk data requests.

This authorization framework leverages short-lived access tokens to verify requests. 

Here’s how it works:

  1. A user generates a token signed with its private key and a client ID to send a request.
  2. The FHIR server validates the request using the user’s public key.
  3. If successful, the server issues a temporary access token that’s used to send data extraction requests.

Once the token expires, the process needs to be repeated. This also allows user access control, as servers can be configured to restrict data access associated with a particular client ID.

Takeaway

In the US, to comply with the 21st Century Act and its requirements to provide easy clinical data access, the adoption of the FHIR Bulk Data API is expected to become even more widespread across the healthcare ecosystem. 

Additionally, there is substantial community interest in implementing a bulk-import FHIR data ($import) operation. There are currently two proposals published by the SMART on FHIR team that are still in their prototyping phase. You can test them out and provide feedback to help accelerate the development. 

We hope you’ve found this article useful. For more practical examples of how to export data with Bulk FHIR API, please refer to our Postman collection

If you want to implement FHIR in your system, need help transferring or extracting FHIR data, or want to integrate your system with an EHR, our team can help. Contact us to discuss your project!

Let’s chat

We would be glad to share more details about our enterprise-level solutions and other cases based on the FHIR standard








    Schedule demo

    We will share all details about our enterprise-level solutions and other cases based on the FHIR standard








      Thank you!

      KODJIN WHITEPAPER

      Please leave your email to get Kodjin White Paper.
      By downloading files from this site you agree to the Policy





        Thank you!

        Thank you!