Medical Certificate Scanner UI Components | Android Document Scanner

Introduction

The Scanbot SDK provides the ability to find and extract content from German Medical Certificates (MC / AU-Bescheinigung forms).

The following fields can be extracted:

• Checkboxes:
  • Initial Certificate (Erstbescheinigung)
  • Renewed Certificate (Folgebescheinigung)
  • Work Accident (Arbeitsunfall, Arbeitsunfallfolgen, Berufskrankheit)
  • Assigned To Accident Insurance Doctor (dem Durchgangsarzt zugewiesen)
• Date Fields:
  • Incapable Since (arbeitsunfähig seit)
  • Incapable Until (voraussichtlich arbeitsunfähig bis ...)
  • Diagnosed On (festgestellt am)
• Patient information:
  • Insured Person Age - Adult or Child
  • Insurance Provider
  • First Name
  • Last Name
  • Address
  • Diagnosis
  • Health Insurance Number
  • Insured Person Number
  • Status
  • Place Of Operation Number
  • Doctor Number

The Medical Certificate recognizer is available as an RTU UI and as a classic component (types of components are explained here).

Integration

Take a look at our Example Apps to see how to integrate the Medical Certificate recognizer.

• Ready-To-Use UI: ready-to-use-ui-demo
• Classic UI Components: mc-scanner

Add Feature as a Dependency

MedicalCertificateRecognizer is available with SDK Package 4. You have to add the following dependencies for it:

implementation("io.scanbot:sdk-package-4:$latestSdkVersion")
implementation("io.scanbot:sdk-mc-assets:$latestSdkVersion")

Some Medical Certificates contain additional information in the barcode. For such cases you also have to add the dependency with Barcode Recognizer assets:

implementation("io.scanbot:sdk-barcode-assets:$latestSdkVersion")

caution

Please do not use multiple scanners at the same time. For example, do not combine generic document scanner, health insurance scanner, text data scanner, etc. at the same time! Each scanner instance requires a lot of memory, GPU, and processor resources. Using multiple scanners will lead to performance issues for the entire application.

Initialize the SDK

The Medical Certificate Recognizer is based on the OCR feature of the Scanbot SDK. Please check the docs for more details.

In order to use the Medical Certificate Recognizer you need to prepare the English OCR language file. Place the eng.traineddata file in the assets sub-folder assets/ocr_blobs/ of your app.

Then on initialization of the SDK call the prepareOCRLanguagesBlobs(true) method:

import io.scanbot.sdk.ScanbotSDKInitializer

...

ScanbotSDKInitializer()
    .prepareOCRLanguagesBlobs(true)
    // ...
    .initialize(this)

caution

Unfortunately, we have noticed that all devices using a Cortex A53 processor DO NOT SUPPORT GPU acceleration. If you encounter any problems, please disable GPU acceleration for these devices.

ScanbotSDKInitializer()
        .allowGpuAcceleration(false)

Ready-To-Use UI Component

Ready-To-Use UI Component (activity) that is responsible for recognizing Medical Certificates is MedicalCertificateRecognizerActivity.

Starting and configuring RTU Medical Certificate recognizer

First of all, you have to add the SDK package and feature dependencies as described here.

Initialize the SDK as described here. More information about the SDK license initialization can be found here.

To start the RTU UI Medical Certificate recognizer you only have to start a new activity and be ready to process its result later.

val resultLauncher: ActivityResultLauncher<MedicalCertificateRecognizerConfiguration>

...

resultLauncher = registerForActivityResult(MedicalCertificateRecognizerActivity.ResultContract()) { resultEntity: MedicalCertificateRecognizerActivity.Result ->
    if (resultEntity.resultOk) {
        // Here you can handle and present recognized Medical Certificate information (patient info, the cropped image of the Medical Certificate, all checkboxes and dates)
    }
}

...

myButton.setOnClickListener {
    val configuration = MedicalCertificateRecognizerConfiguration()
    resultLauncher.launch(configuration)
}

myButton.setOnClickListener {
    val configuration = MedicalCertificateRecognizerConfiguration()
    val intent = MedicalCertificateRecognizerActivity.newIntent(this@MainActivity, configuration)
    startActivityForResult(intent, MEDICAL_CERTIFICATE_REQUEST_CODE_CONSTANT)
}

if (requestCode == MEDICAL_CERTIFICATE_REQUEST_CODE_CONSTANT) {
    val result: MedicalCertificateRecognizerActivity.Result = MedicalCertificateRecognizerActivity.extractResult(resultCode, data)
    if (result.resultOk) {
        // Here you can handle and present recognized Medical Certificate information (patient info, the cropped image of the Medical Certificate, all checkboxes and dates)
    }
}

An instance of MedicalCertificateRecognizerConfiguration is required for starting the RTU UI activity. It allows configuration changes through methods it exposes:

val configuration = MedicalCertificateRecognizerConfiguration()
configuration.setRecognizePatientInfo(true)
configuration.setReturnCroppedDocumentImage(true)

configuration.setFlashEnabled(false)
configuration.setCancelButtonTitle("Stop")

info

All parameters in MedicalCertificateRecognizerConfiguration are optional.

API references for all of these methods can be found on this page.

Handling the result

Below is a simple example of presenting the result: MedicalCertificateRecognizerResult fields, shown in a Toast notification.

val result: MedicalCertificateRecognizerResult

val medicalCertificateContent: String = StringBuilder()
    .append("Type: ").append(if (result.checkboxes
                    ?.find { checkBox -> checkBox.type == McInfoBoxSubtype.McBoxInitialCertificate }
                    ?.hasContents == true)
        "Initial"
    else if (result.checkboxes
                    ?.find { checkBox -> checkBox.type == McInfoBoxSubtype.McBoxRenewedCertificate }
                    ?.hasContents == true)
        "Renewed"
    else
        "Unknown").append("\n")
    .append("Work Accident: ").append(if (result.checkboxes
                    ?.find { checkBox -> checkBox.type == CheckBoxType.McBoxWorkAccident }
                    ?.hasContents == true)
        "Yes"
    else "No").append("\n")
    .append("Accident Consultant: ").append(
            if (result.checkboxes
                            ?.find { checkBox -> checkBox.type == CheckBoxType.McBoxAssignedToAccidentInsuranceDoctor }
                            ?.hasContents == true)
                "Yes"
            else "No"
    ).append("\n")
    .append("Start Date: ").append(
            result.dates?.find { dateRecord -> dateRecord.type == DateRecordType.DateRecordIncapableOfWorkSince }?.dateString
    ).append("\n")
    .append("End Date: ").append(
            result.dates?.find { dateRecord -> dateRecord.type == DateRecordType.DateRecordIncapableOfWorkUntil }?.dateString
    ).append("\n")
    .append("Issue Date: ").append(
            result.dates?.find { dateRecord -> dateRecord.type == DateRecordType.DateRecordDiagnosedOn }?.dateString
    )
    .append("\n")
    .append("Form type: ${result.mcFormType.name}")
    .append("\n")
    .append(result.patientInfoBox.patientInfoFields.joinToString(separator = "\n", prefix = "\n") { "${it.patientInfoFieldType.name}: ${it.value}" })
    .toString()

Toast.makeText(this@MainActivity, medicalCertificateContent, Toast.LENGTH_LONG).show()

Full API references for the result class are available here.

Classic component

Try our Medical Certificate Recognizer Example App or check the following step by step integration instructions.

MedicalCertificateRecognizer can be used in conjunction with ScanbotCameraXView or separately. Let's have a look at an example with ScanbotCameraXView.

Add feature dependencies and initialize the SDK

First of all, you have to add the SDK package and feature dependencies as described here.

Initialize the SDK as described here. More information about the SDK license initialization can be found here.

Add ScanbotCameraXView to layout

<io.scanbot.sdk.ui.camera.ScanbotCameraXView
    android:id="@+id/camera_view"
    android:layout_width="match_parent"
    android:layout_height="match_parent" />

Get MedicalCertificateRecognizer instance from ScanbotSDK and attach it to ScanbotCameraXView

val scanbotSDK = ScanbotSDK(this)
val medicalCertificateRecognizer: MedicalCertificateRecognizer = scanbotSDK.createMedicalCertificateRecognizer()

//Attach `FrameHandler`, that will detect a Medical Certificate document in the camera frames
val frameHandler: MedicalCertificateFrameHandler = MedicalCertificateFrameHandler.attach(cameraView, medicalCertificateRecognizer)

// Attach `AutoSnappingController`, that will trigger the snap as soon as `FrameHandler` detects a Medical Certificate document in the preview frame successfully
val autoSnappingController = MedicalCertificateAutoSnappingController.attach(cameraView, frameHandler)

Add a snapping result listener for IScanbotSdkView

cameraView.addPictureCallback(object : PictureCallback() {
    override fun onPictureTaken(image: ByteArray, captureInfo: CaptureInfo) {
        processPictureTaken(image, captureInfo.imageOrientation)
    }
})

To detect full Medical Certificate information you have to perform the Certificate recognition on the snapped image.

private fun processPictureTaken(image: ByteArray, imageOrientation: Int) {
    // Here we get the full image from the camera.
    // Implement a suitable async(!) detection and image handling here.

    // Run Medical Certificate recognition on the snapped image:
    val resultInfo = medicalCertificateRecognizer.recognizeMcJpeg(image,
        0,
        shouldCropDocument = true,
        returnCroppedDocument = true,
        recognizePatientInfo = true,
        recognizeBarcode = true)

    if (resultInfo != null && resultInfo.recognitionSuccessful) {
        // Here you can handle and present recognized Medical Certificate information (patient info, the cropped image of the Medical Certificate, all checkboxes and dates)
    } else {
        // If recognition was not successful - show a warning and try again
        runOnUiThread {
            val toast = Toast.makeText(this, "No Medical Certificate content was recognized!", Toast.LENGTH_LONG)
            toast.setGravity(Gravity.CENTER, 0, 0)
            toast.show()
        }
    }
}

If resultInfo is not null and resultInfo.recognitionSuccessful flag is true then Medical Certificate content was successfully recognized.

As the result you will get a MedicalCertificateRecognizerResult object that contains all extracted data:

• checkboxes: List<CheckBox> - list of the checkboxes with content types, states and confidence values
• dates: List<DateRecord> - list of the DateRecord objects with recognized dates, date types, recognitionConfidenceValues and validationConfidenceValues
• patientInfoBox - zone of patient info:
  • patientInfoFields: List<McPatientInfoField> - list of the McPatientInfoField objects with information about the patient, recognitionConfidenceValues and validationConfidenceValues
• mcFormType: McFormType - type of the Medical Certificate

Add a Finder Overlay

In addition, it is also possible to add a "Finder Overlay". This feature allows you to predefine a Medical Certificate document area over the ScanbotCameraXView screen. By using this overlay the Medical Certificate recognizer can skip the time-consuming step "Search for Medical Certificate area" and perform the recognition directly in the specified "Finder Overlay" area. By using this approach the Medical Certificate scanner recognizes the document and extracts the content much faster.

Details about applying finder view logic in the layout and in the code can be found here.

Want to scan longer than one minute?

Generate your free "no-strings-attached" Trial License and properly test the Scanbot SDK.

Get your free Trial License

What do you think of this documentation?

What can we do to improve it? Please be as detailed as you like.

---