Skip to main content

Xamarin.Forms | Xamarin Document Scanner

Integration with Xamarin.Forms#

SDK Initialization#

Initialize SDK#

The Scanbot SDK must be initialized before usage. Make sure to run the initialization as early as possible. We recommend implementing the initialization in the Application class of the Android app, and in the AppDelegate class of the iOS app.

Android#

Add the following code to your Application class:

using ScanbotSDK.Xamarin.Forms;
[Application(LargeHeap = true)]public class MainApplication : Application{    const string licenseKey = null;    ...    public override void OnCreate()    {        ...        // You can pass null as licenseKey for trial mode. See the "License" section for more details.        SBSDKInitializer.Initialize(this, licenseKey, new SBSDKConfiguration { EnableLogging = true });        ....    }    ...}

Furthermore, it is highly recommended to add the flag LargeHeap = true to the Application attribute on the MainApplication since image processing is very memory-intensive.

iOS#

Add the following code to your AppDelegate class:

using ScanbotSDK.Xamarin.Forms;
[Register("AppDelegate")]public class AppDelegate : UIApplicationDelegate{    const string licenseKey = null;    ...    public override bool FinishedLaunching(UIApplication application, NSDictionary launchOptions)    {        // You can pass null as licenseKey for trial mode. See the "License" section for more details.        SBSDKInitializer.Initialize(application, licenseKey, new SBSDKConfiguration { EnableLogging = true });        ....    }    ...}

API Reference#

The main entry point of the Xamarin.Forms wrapper is the ScanbotSDK.Xamarin.Forms.SBSDK class:

public static class SBSDK{    /// Singleton implementation of SDK functions.    public static IScanbotOperations Operations => DependencyService.Get<IScanbotOperations>();
    /// Singleton implementation of ready-to-use UI launcher and page management.    public static IReadyToUseUi UI => DependencyService.Get<IReadyToUseUi>();
    ...}
  • SBSDK.Operations - functions for raw image processing as well as page processing.
  • SBSDK.UI - functions that launch the Scanbot SDK ready-to-use UI screens for document scanning, cropping, MRZ scanning, barcode scanning, etc.

Pages#

The Scanbot SDK provides a ready-to-use UI for document scanning and cropping. Both components use the notion of a 'page' as a data model for the scanning and cropping activities. A page is represented by ScanbotSDK.Xamarin.Forms.IScannedPage:

public interface IScannedPage : INotifyPropertyChanged{    /// Gets the document image. May be null if document detection has not yet run on the original image.    ImageSource Document { get; }
    /// Gets the original image, as taken by the camera.    ImageSource Original { get; }
    /// Gets the document image in a size useful for showing on screen. May be null if document detection has not yet run on the original image.    ImageSource DocumentPreview { get; }
    /// Gets the document image in a size useful for showing on screen.    ImageSource OriginalPreview { get; }
    /// Gets the document preview if available, otherwise the original preview. Useful when data-binding.    ImageSource AvailablePreview { get; }
    /// Gets the four points of the polygon that bounds the detected document. May be empty if document detection has not yet run on the original image.    Point[] Polygon { get; set; }
    /// Gets the filter.    ImageFilter Filter { get; }
    /// Gets the detection status. May be <see cref="DocumentDetectionStatus.Unknown"/> if document detection has not yet run on the original image.    DocumentDetectionStatus DetectionStatus { get; }
    /// Sets the document image of this page.    Task<ImageSource> SetDocumentAsync(ImageSource document);
    /// Sets the filter to apply on the unfiltered document image. The page will raise change notifications for the <see cref="Document"/> and related properties when done.    Task<ImageSource> SetFilterAsync(ImageFilter filter);
    /// Returns a preview of the given filter as applied to the document image.    Task<ImageSource> GetFilteredDocumentPreviewAsync(ImageFilter filter);
    /// Rotates the images of this page counter-clockwise in 90 degree increments. The page will raise change notifications to all image properties when done.    Task RotateAsync(int rotations);
    /// Remove all image files associated with this page. After the task completes,    /// the images in this object will no longer be valid.    Task RemoveAsync();
    /// Detects and crops the document in the original image. After completion the <see cref="DetectionStatus"/> property    /// will contain the status of the detection and, if successful, the <see cref="Document"/> property will    /// contain the image of the cropped document.    Task<DocumentDetectionStatus> DetectDocumentAsync();}

Pages are stored in an internal page file storage, where the pageId serves as a name prefix for the stored image files. Operations that modify pages work in-place. That is, for example, RotatePage() overwrites the page's image files with their rotated versions. This behavior differs from the behavior of raw image functions like RotateImage() which always create a new file.

IScannedPage implements INotifyPropertyChanged and can be directly used as a BindingContext.

Image and Page Processing#

public interface IScanbotOperations{    /// Returns information about the installed OCR languages.    OcrConfigs OcrConfigs { get; }
    /// Returns whether the installed license is valid. If the SDK was initialized without a license,    /// trial mode will be enabled and the license will remain valid for about a minute.    bool IsLicenseValid { get; }
    /// Applies an image filter to an ImageSource asynchronously.    Task<ImageSource> ApplyImageFilterAsync(ImageSource image, ImageFilter filter, ResultOptions resultOptions = null);
    /// Rotates an image asynchronously.    Task<ImageSource> RotateImageAsync(ImageSource image, double degrees, ResultOptions resultOptions = null);
    /// Creates a PDF file out of one or more ImageSource objects. All images must be instances of FileImageSource.    /// These are images created with ImageSource.FromFile and images stored as files by the SDK    /// (which is the default behavior, corresponds to ResultStorage.File).    ///    /// The generated file will be deleted when calling CleanUp. If you want to keep the file, move it to a folder that you have control over.    Task<Uri> CreatePdfAsync(IEnumerable<ImageSource> images, PDFPageSize pageSize);
    /// Writes the given images into a TIFF file. If more than one image is given, the function creates a multi-page TIFF.    /// The function can optionally create a 1-bit encoded TIFF.    ///    /// The generated file will be deleted when calling CleanUp. If you want to keep the file, move it to a folder that you have control over.    Task<Uri> WriteTiffAsync(IEnumerable<ImageSource> images, TiffOptions options = null);
    /// Detects a document in an image.    Task<DocumentDetectionResult> DetectDocumentAsync(ImageSource image, ResultOptions resultOptions = null);
    /// Recognizes a Payform in an image.    Task<PayFormRecognitionResult> RecognizePayformAsync(ImageSource image);
    /// Performs OCR on an image.    Task<string> PerformOcrAsync(IEnumerable<ImageSource> images, IEnumerable<string> languages, string pdfOutputPath = null);
    /// Creates a ScannedPage object from an ImageSource. Used when you need to pass an ImageSource to CroppingPage.    Task<IScannedPage> CreateScannedPageAsync(ImageSource source);
    /// Returns all scanned pages currently in persistent storage.    Task<IEnumerable<IScannedPage>> GetAllPagesAsync();
    /// Removes all internally generated files from methods of this interface, as well as all page image files.    Task CleanUp();}

Storage Encryption#

Scanbot SDK provides the ability to store the generated image files (JPG, PNG) and PDF files encrypted. This feature provides an additional level of security to the default secure storage locations of the native SDKs.

By default the file encryption is disabled. If you wish to enable encryption, you have to pass the following config parameters on SDK initialization:

new SBSDKConfiguration{    Encryption = new SBSDKEncryption    {        Mode = EncryptionMode.AES256,        Password = "SomeSecretPa$$w0rdForFileEncryption"    }};

By activating the storage encryption the native Scanbot SDKs will use the builtin AES 128 or AES 256 encryption. All generated image files (JPG, PNG) including the preview image files, as well as the exported PDF files will be encrypted in memory and stored as encrypted data files on the flash storage of the device.

The Scanbot SDK derives the AES key from the given password, an internal salt value, and the internal number of iterations using the PBKDF2 function.

When applying image operations like cropping, rotation or image filters, Scanbot SDK will decrypt the image file in memory, apply the changes, encrypt and store it again.

If encryption is enabled, IScannedPage properties Document, Original, DocumentPreview, OriginalPreview and AvailablePreview will be encrypted.

You should still pass these images to all ScanbotSDK functions, as they deal with encryption internally, and the paths to files are valid, however, you cannot display encrypted ImageSource objects.

You need to use the await DecryptedOriginal() or await DecryptedDocument() methods of IScannedPage to display images in your app.

Persistence of Page Objects#

Scanbot SDK does not persist page objects, only the image files (.jpg or .png). The management of page objects and the persistence of the page metadata is left to the app. Depending on the use case, it may be necessary to persist the data of the page objects in the app (e.g. as JSON in a local SQLite database). If this is the case, the following must be considered.

The Page objects contain absolute paths to the image files, i.e. the Document image of a ScannedPage might be located at:

file:///var/mobile/Containers/Data/Application/54286F82-A8F7-4850-A684-8D3487726A4D/Library/Application%20Support/net.doo.ScanbotSDK/SBSDK_ImageStorage_Default/PageFileStorage/JPEG/documents/60426F47-4119-48F8-ADA9-F7E60D583CB4_preview.jpg

The page image files are stored in corresponding storage locations. By default, the Scanbot SDK Android uses the internal files directory of the app, whereas the Scanbot SDK iOS uses the "Application Support" folder. Those storage folders are secure and are not affected by an app update.

However, on iOS, the absolute file path of the "Application Support" folder is not reliable. For each fresh install of an app, a new UUID is generated, in the following format: /var/mobile/Containers/Data/Application/{UUID}. That is where application data is stored. When updating the app, the uuid may change. For example:

Before an app update:

file:///var/mobile/Containers/Data/Application/54286F82-A8F7-4850-A684-8D3487726A4D/Library/Application%20Support/net.doo.ScanbotSDK/SBSDK_ImageStorage_Default/PageFileStorage/JPEG/documents/60426F47-4119-48F8-ADA9-F7E60D583CB4.jpg

After the app update:

file:///var/mobile/Containers/Data/Application/C9181555-252A-4665-892F-793008ED0EDD/Library/Application%20Support/net.doo.ScanbotSDK/SBSDK_ImageStorage_Default/PageFileStorage/JPEG/documents/60426F47-4119-48F8-ADA9-F7E60D583CB4.jpg

Please note that the actual image files are still there, only the absolute file paths change.

To get to the new absolute paths, the API method RefreshImageUris(IEnumerable<IScannedPage>) has been introduced in the ScanbotSDK.Xamarin.Forms SDK version 3.2.0. Please use this method to refresh all image file URIs from affected pages:

List<IScannedPage> loadedPages = // load pages from databasevar refreshedPages = await SBSDK.Operations.RefreshImageUris(loadedPages);

It is highly recommended to use this method whenever you have to (re-)load Page objects from the database of your app, regardless of whether there was an app update or not.

Document Scanner#

Launches the Document Scanner. The scanned pages will be returned asynchronously.

Show in example app

alt

Usage#

var config = new DocumentScannerConfiguration();var result = await SBSDK.UI.LaunchDocumentScannerAsync(config);if (result.Status == OperationResult.Ok){    // result.Pages contains the scanned pages}

Cropping UI#

Launches the Cropping UI on the given page. A page object can be initially created using the Document Scanner or SBSDK.Operations.CreateScannedPageAsync().

Show in example app

alt

Usage#

var page = ...var config = new CroppingScreenConfiguration();var result = await SBSDK.UI.LaunchCroppingScreenAsync(page, config);if (result.Status == OperationResult.Ok){    // `page` has been updated}

Machine-readable Zone Scanner#

Launches the MRZ scanner. The scanned data will be returned asynchronously.

Show in example app

alt

Usage#

var config = new MrzScannerConfiguration();config.FinderWidthRelativeToDeviceWidth = 0.9;config.FinderHeightRelativeToDeviceWidth = 0.18;var result = await SBSDK.UI.LaunchMrzScannerAsync(config);if (result.Status == OperationResult.Ok){    // result.Fields contain the recognized MRZ fields}

Barcode/QR-code Scanner#

Launches the barcode/QR-code scanner. The scanned code will be returned asynchronously.

Show in example app

alt

Usage#

var config = new BarcodeScannerConfiguration();var result = await SBSDK.UI.LaunchBarcodeScannerAsync(config);if (result.Status == OperationResult.Ok){    // result.Text contains the decoded value of the scanned machine code}

Generic Document Recognizer#

Launches the Generic Document Recognizer. The scanned documents will be returned asynchronously.

Show in example app

The Generic Document Recognizer is based on the OCR feature and thus requires the proper installation of the OCR language files deu.traineddata and eng.traineddata (aka. blob files). For more details on how to set up OCR language files please refer to the OCR section.

Usage#

var config = new GenericDocumentRecognizerConfiguration{    DocumentType = GenericDocumentType.DeIdCard,    TopBarButtonsColor = Color.Green,    // Further customization parameters...};var result = await SBSDK.UI.LaunchGenericDocumentRecognizerAsync(configuration);if (result.Status == OperationResult.Ok){    // result contains the documents that have been recognized}

Optical Character Recognition#

The Scanbot SDK provides simple and convenient APIs to run Optical Character Recognition (OCR) on images.

As result you can get:

  • a searchable PDF document with the recognized text layer (aka. sandwiched PDF document)
  • recognized text as plain text
  • bounding boxes of all recognized paragraphs, lines and words
  • text results and confidence values for each bounding box

The Scanbot OCR feature is based on the Tesseract OCR engine with some modifications and enhancements. The OCR engine supports a wide variety of languages. For each desired language a corresponding OCR training data file (.traineddata) must be provided. Furthermore, the special data file osd.traineddata is required (used for orientation and script detection). The Scanbot SDK package contains no language data files to keep the SDK small in size. You have to download and include the desired language files in your app.

Preconditions to achieve a good OCR result#

Conditions while scanning

A perfect document for OCR is flat, straight, in the highest possible resolution and does not contain large shadows, folds, or any other objects that could distract the recognizer. Our UI and algorithms do their best to help you meet these requirements. But as in photography, you can never fully get the image information back that was lost during the shot.

Languages

You can use multiple languages for OCR. But since the recognition of characters and words is a very complicated process, increasing the number of languages lowers the overall precision. With more languages, there are more results where the detected word could match. We suggest using as few languages as possible. Make sure that the language you are trying to detect is supported by the SDK and added to the project.

Size and position

Put the document on a flat surface. Take the photo from straight above in parallel to the document to make sure that the perspective correction does not need to be applied much. The document should fill most of the camera frame while still showing all of the text that needs to be recognized. This results in more pixels for each character that needs to be detected and hence, more detail. Skewed pages decrease the recognition quality.

Light and shadows

More ambient light is always better. The camera takes the shot at a lower ISO value, which results in less grainy photos. You should make sure that there are no visible shadows. If you have large shadows, it is better to take the shot at an angle instead. We also do not recommend using the flashlight - from this low distance it creates a light spot at the center of the document which decreases the recognition quality.

Focus

The document needs to be properly focused so that the characters are sharp and clear. The autofocus of the camera works well if you meet the minimum required distance for the lens to be able to focus. This usually starts at 5-10cm.

Typefaces

The OCR trained data is optimized for common serif and sans-serif font types. Decorative or script fonts drastically decrease the quality of recognition.

Download and Provide OCR Language Files#

You can find a list of all supported OCR languages and download links on this Tesseract wiki page.

⚠️️️ Please choose and download the proper version of the language data files:

Download the desired language files as well as the osd.traineddata file and place them in the Assets sub-folder SBSDKLanguageData/ of your Android app or in the Resources sub-folder ScanbotSDKOCRData.bundle/ of your iOS app.

Example for Android:

Droid/Assets/SBSDKLanguageData/eng.traineddata  // english language fileDroid/Assets/SBSDKLanguageData/deu.traineddata  // german language fileDroid/Assets/SBSDKLanguageData/osd.traineddata  // required special data file

Example for iOS:

iOS/Resources/ScanbotSDKOCRData.bundle/eng.traineddata  // english language fileiOS/Resources/ScanbotSDKOCRData.bundle/deu.traineddata  // german language fileiOS/Resources/ScanbotSDKOCRData.bundle/osd.traineddata  // required special data file

OCR API#

IEnumerable<ImageSource> documentImageSources = ...
var languages = new[] { "en" }; // or specify more languages like { "en", "de", ... }var result = await SBSDK.Operations.PerformOcrAsync(documentImageSources, languages);
// See the recognized plain text of all images/pages via result.Text,// or get more details of each image/page via result.Pages:foreach (var ocrPage in result.Pages){    //ocrPage.Paragraphs ...    //ocrPage.Lines ...    //ocrPage.Words ...    // example for words:    foreach (var wordBlock in ocrPage.Words)    {        //wordBlock.Text ...        //wordBlock.BoundingBox.X ...        //wordBlock.BoundingBox.Y ...        //wordBlock.BoundingBox.Width ...        //wordBlock.BoundingBox.Height ...    }}

Detect barcodes from still image#

The Scanbot SDK detects barcodes from an existing image (ImageSource).

List<Barcode> result = await SBSDK.Operations.DetectBarcodesFrom(image);

Workflows#

A Workflow represents a set of multiple scanning steps. You can combine Document Scanning with QR code detection or MRZ recognition on an ID card image, for example. Workflow Steps can be run on a captured still image or a video frame, so a step can either be a live-detection or a still-image capturing step. You can validate each step result, present an error message to the user if the validation fails and restart a step.

Usage#

var workflow = SBSDK.UI.CreateWorkflow();workflow.AddScanBarcodeStep(    "QR code and Document 1/2",    "Please scan a QR code",    new[] { BarcodeFormat.QrCode });
workflow.AddScanDocumentPageStep(    "QR code and Document 2/2",    "Please scan a document."    );var config = new WorkflowScannerConfiguration{    ...};var result = await SBSDK.UI.LaunchWorkflowScannerAsync(workflow, config);if (result.Status == OperationResult.Ok){    // result.Results is a collection of results, one from each step in the workflow.}

Estimate Blurriness#

Estimates image blurriness. Less is sharper, more is blurred.

Usage#

var image = CurrentPage.Document;var blur = await SBSDK.Operations.EstimateBlurriness(image);

In broad terms, consider blur values as follows:

  • 0.0-0.3: This image is not blurry at all
  • 0.3-0.6: Somewhat blurry, should be okay
  • 0.6-1.0: I am skeptical of the usefulness of the image

However, this is not as easy as it seems. If a scanned document has a predominantly white background, it will be considered a very blurred image.

TIFF Creation#

Task<Uri> WriteTiffAsync(IEnumerable<ImageSource> images, TiffOptions options = null)

Writes the given images into a TIFF file. All images must be instances of FileImageSource. These are images created with ImageSource.FromFile and images stored as files by the Scanbot Xamarin SDK. If more than one image is given, the function creates a multi-page TIFF. The function can optionally create a 1-bit encoded (binarized) TIFF. The generated TIFF file will be deleted when calling CleanUp. If you want to keep the file, move it to a folder that you have control over.

Typically, the final hi-res document image files of the scanned Pages (Page.Document) are used to generate the TIFF file.

Usage#

// list of scanned pagesList<IScannedPage> ScannedPages = ...
// collect the document images of scanned pages List<ImageSource> DocumentImages = new List<ImageSource>();foreach (var page in ScannedPages){    DocumentImages.Add(page.Document);}
// and create a binarized (1-bit encoded) TIFF from the document imagesvar tiffFileUri = await SBSDK.Operations.WriteTiffAsync(    DocumentImages,    new TiffOptions { OneBitEncoded = true, Dpi = 300, Compression = TiffCompressionOptions.CompressionCcittT6 });

Input args:

  • images: Input images as a list of ImageSources in proper order (image element 1 => page 1, etc).
  • options: optional TiffOptions:
    • OneBitEncoded : Optional boolean flag. If true, the input images will be binarized and the output TIFF file will be saved with one bit per pixel. If false, the input images will be stored as-is. The default value is false.
    • Dpi : Optional integer value for Dots Per Inches. The default value is 200 dpi.
    • Compression : Optional TIFF compression type. The default value depends on the OneBitEncoded flag. CompressionCcittT6 (CCITT Fax 4) is the default value for binarized (OneBitEncoded=true) images. For color images (OneBitEncoded=false) the default value is CompressionAdobeDeflate (ZIP).

Result:

  • tiffFileUri: File URI of the TIFF result file ('file:///...').

Supported TIFF Compression Types#

  • CompressionNone : No compression.
  • CompressionCcittT6 : "CCITT Fax 4" compression type. Most common and recommended type for binarized (1-bit) black and white images to achieve a small TIFF file size.
  • CompressionAdobeDeflate : "ZIP" compression type. Most common type for color images.
  • CompressionCcittrle
  • CompressionCcittfax3
  • CompressionCcittT4
  • CompressionCcittfax4
  • CompressionCcittrlew
  • CompressionLzw
  • CompressionPackbits
  • CompressionDeflate
caution

Please note that the following compression types are only compatible for binarized images (1-bit encoded black & white images): CompressionCcittrle, CompressionCcittfax3, CompressionCcittT4, CompressionCcittfax4, CompressionCcittT6, CompressionCcittrlew.

PDF Creation#

Task<Uri> CreatePdfAsync(IEnumerable<ImageSource> images, PDFPageSize pageSize)

Creates a PDF file out of one or more ImageSource objects. All images must be instances of FileImageSource. These are images created with ImageSource.FromFile and images stored as files by the Scanbot Xamarin SDK. If more than one image is given, the function creates a multi-page PDF. The generated PDF file will be deleted when calling CleanUp. If you want to keep the file, move it to a folder that you have control over.

Typically, the final hi-res document image files of the scanned Pages (Page.Document) are used to generate the PDF file.

Usage#

// list of scanned pagesList<IScannedPage> ScannedPages = ...
// collect the document images of scanned pages List<ImageSource> DocumentImages = new List<ImageSource>();foreach (var page in ScannedPages){    DocumentImages.Add(page.Document);}
// and create the PDF file from the document imagesvar pdfFileUri = await SBSDK.Operations.CreatePdfAsync(DocumentImages, PDFPageSize.FixedA4);

Input args:

  • images: Input images as a list of ImageSources in proper order (image element 1 => page 1, etc).
  • pageSize: PDFPageSize enum value to specify the output page size. See below.

Result:

  • pdfFileUri: File URI of the PDF result file ('file:///...').

Supported PDF Page Sizes#

  • A4: The page has the aspect ratio of the image, but is fitted into A4 size. Whether portrait or landscape depends on the image's aspect ratio.
  • FixedA4: The page has A4 size. The image is fitted and centered within the page. Whether portrait or landscape depends on the image's aspect ratio.
  • USLetter: The page has the aspect ratio of the image, but is fitted into US letter size. Whether portrait or landscape depends on the image's aspect ratio.
  • FixedUsLetter: The page has US letter size. The image is fitted and centered within the page. Whether portrait or landscape depends on the image's aspect ratio.
  • Auto: For each page the best matching format (A4 or US letter) is used. Whether portrait or landscape depends on the image's aspect ratio.
  • AutoLocale: Each page of the result PDF will be of US letter or A4 size depending on the current locale. Whether portrait or landscape depends on the image's aspect ratio.
  • FromImage: Each page is as large as its image at 72 dpi.
PDF with OCR

If you need to generate a searchable PDF file with a text layer, please refer to the "OCR" section.