Skip to main content

Using Multiple Objects Detector | Android Document Scanner

The Scanbot SDK provides the ability to detect multiple non-overlapping rectangular objects (like a bunch of business cards) with a single snap. As a result of detection, the user gets a list of Polygon instances, each representing coordinates of a detected object.

Additionally, MultipleObjectsDetector.Params can be used to filter results by aspect ratio for detected objects. See Detection params

Try our Multiple Objects Detector Example App or check the following step by step integration instructions.

Step 1 - Add Multiple Objects Detector Feature as a Dependency#

MultipleObjectsDetector is available with the SDK Package 2. You have to add the following dependency for it:

api "io.scanbot:sdk-package-2:$latestVersion"

It can be used both in conjunction with ScanbotCameraView (live detection, e.g. for preview) and by itself for detection on a Bitmap.

Step 2 - Add ScanbotCameraView and MultiplePolygonsView to layout#

<io.scanbot.sdk.camera.ScanbotCameraView    android:id="@+id/camera_view"    android:layout_width="match_parent"    android:layout_height="match_parent"    >
    <io.scanbot.sdk.ui.multipleobjects.MultiplePolygonsView        android:id="@+id/polygon_view"        android:layout_width="match_parent"        android:layout_height="match_parent"        app:polygonFillColor="#55009688"        app:polygonStrokeColor="#ff80cbc4"        app:polygonStrokeWidth="8dp"        /></io.scanbot.sdk.camera.ScanbotCameraView>

MultiplePolygonsView utilizes the same set of customizations as PolygonView. See Customizing drawn polygon

Step 3 - get MultipleObjectsDetector instance from ScanbotSDK and attach it to ScanbotCameraView#

val scanbotSdk = ScanbotSDK(this)val multipleObjectsDetector = scanbotSdk.createMultipleObjectsDetector()val multipleObjectsFrameHandler = MultipleObjectsFrameHandler.attach(cameraView, multipleObjectsDetector)

Step 4 - Add result handler for MultipleObjectsFrameHandler:#

multipleObjectsFrameHandler.addResultHandler(polygonView.multipleObjectDetectorHandler)

Step 5 - Feed snapped picture to MultipleObjectsDetector, process results#

First, decode image ByteArray obtained from camera's callback. To do this, simply use our ImageProcessor:

override fun onPictureTaken(image: ByteArray, captureInfo: CaptureInfo) {    val imageProcessor = ScanbotSDK(this).imageProcessor()    val resultBitmap = imageProcessor.processJpeg(image, RotateOperation(captureInfo.imageOrientation))}

Next, we can get detected polygons from MultipleObjectDetector:

val multipleObjectsDetector = ScanbotSDK(this).createMultipleObjectsDetector()val polygons = multipleObjectsDetector.detectOnBitmap(resultBitmap)

As an example of further application, we can use given polygons to crop detected objects as separate pages:

import io.scanbot.sdk.persistence.Pageimport io.scanbot.sdk.process.ImageFilterTypeimport io.scanbot.sdk.core.contourdetector.DetectionResult
    ...        val pageProcessor = scanbotSdk.createPageProcessor()    val pageFileStorage = scanbotSdk.createPageFileStorage()
    val detectedObjectsPages: List<Page> = polygons.map { polygon ->        val pageId = pageFileStorage.add(resultBitmap)        val page = Page(pageId, emptyList(), DetectionResult.OK, ImageFilterType.NONE)        pageProcessor.cropAndRotate(page, 0, polygon.polygonF)    }

Detection params#

MultipleObjectsDetector allows customization using the MultipleObjectsDetector.Params class, instance of which should be set via MultipleObjectsDetector.setParams method. It accepts two values - minAspectRatio and maxAspectRatio, which together represent a range. MultipleObjectsDetector will qualify only objects within that range and ignore all others.

multipleObjectsDetector.setParams(MultipleObjectsDetector.Params(1.6f, 1.8f))

The aspect ratio is calculated as width/height. This means that we can distinguish between different desired orientations: aspect ratio in a range from 1 (square) to 2 (one side is twice as long as the other) qualifies as landscape-oriented objects, whereas ratio 0,5 .. 1 qualifies as portrait-oriented objects.

This can all be explained using a real-life example. One of the business card size standards is 8,9cm X 5,1cm (3.5" x 2"), which gives landscape-aligned cards an aspect ratio of 8,9 / 5,1 = 1,745, and portrait-aligned cards an aspect ratio of 5,1 / 8,9 = 0,573. Setting minAspectRatio and maxAspectRatio to 1,6 and 1,9 respectively will allow you to detect landscape-aligned business cards. Using range 0,4 .. 0,7 is for portrait-aligned cards, and 0,4 .. 1,9 will allow both and everything in between.