Automatically rotate documents, images and pages with OCR and AI

Automatically rotate documents, images and pages with OCR and AI

If you are in the business of large scale document processing, you are probably facing problems related to bad data quality. Bad data quality can come in many forms. In case of document processing these can be things like bad image quality, wrong file formats and rotated documents. Klippa specialises in smart document processing and is able to solve these issues for you. In this blog we will explain how Klippa can help to automatically correct document rotations at large scale. This optimizes processing efficiency and reduces processing costs.

An example use case

So let’s assume you work for a company that is processing financial documents at large scale. For example receipts and invoices for loyalty purposes. This is a common use case in a.o. cashback automation. You have a data entry team that has to check receipts in an interface and extract certain data or perform certain approvals. Manually checking documents is already a time consuming task for normal documents, let alone for bad quality documents. If you are processing 100.000 documents a month, and 10% of the documents are rotated, manually rotating 10.000 documents a month can be a time consuming and annoying task. The yearly cost of rotating 120.000 documents will easily be €20.000 in just labor. Luckily, automation can easily reduce these costs by 90%, saving you €18.000 a year. A great business case! 

Below is an example of the type of data you can expect from customers:

Automatically rotate images and documents

How to automatically correct document & image orientation?

As you can see there are many different types of problems with these documents. In this blog we will focus on automated rotation of receipts and invoices, but this goes for any document type. If you are interested in other document processing solutions like automated document sorting, document classification, image to text, searchable PDF conversion, image cropping, image quality detection, image splitting or image stitching. Read our relevant articles on these topics. 

So let’s focus on rotating the incorrect images above automatically to the correct orientation. A simple approach that many people would think of first, is just to check for the height and width of the documents and rotate them to vertical orientation, so that the height is larger than the width. This sounds simple and effective, but sadly it is error prone. Receipts and invoices come in many different shapes and sizes. Sometimes rectangles, sometimes squares. This approach can cause documents that actually are in the right rotation to be turned into the wrong rotation. It can also cause you to rotate documents to the 180 degrees opposite, so upside down. Luckily there is another solution, doing it based on the text content of a document. To get there our software takes 3 important steps:

1. First we optimise the image quality. We can do this by cropping the receipts pictures, correcting perspective and improving the contrast. This already gives us better readable images, which is relevant for the second step. You can see an example result of the first step below:

Automated receipt cropping

2. In the second step we convert the documents and images to text using OCR. If the document is a PDF, it will first be converted into an image and then into text. This creates a searchable document and reveals what the text orientation is. Of course nobody reads a sentence from top to bottom, but mostly from left to right and in some cases from right to left. On some documents you will have text in multiple orientations. In these cases we will perform a text count and choose the rotation that most text is in.

3. Now that we know the text orientation we can almost rotate the document. The document should be rotated so that you can read left to right for most languages, but for some languages from right to left. This is a determining factor in the rotation. So now we first use a machine learning classifier to determine the country of origin and language of the document. Once this is done the image or document can be rotated by our software and stored in the desired format. In many cases this is the original file format, so for images that would often be a JPEG, but we can also convert it into a format of choice, for example a PDF. The result looks something like this:

Converted receipt images

Now that you have good quality images in the correct orientations, you might already have what you need. But we can even take it one or two steps further: we can actually give you the OCR results in a TXT format, but we can even give you the result in a structured format like JSON. Below you can see a simplified example of those two additional steps:

Klippa Receipt OCR example

What Klippa can do for you

As you can see it is a bit of a technical process and it involves computer vision, OCR and document conversion techniques. Luckily you don’t have to build these tools yourself, because Klippa offers it’s technologies via APIs, SDKs and interfaces so you can implement it into your existing workflow within hours or days. Automatically rotating images, documents and pages is just the tip of the iceberg. If you have any document related challenges that you need help with, just reach out to us. We are happy to see how we can help you reach your goals. 

Works with AZEXO page builder