Create OCR App using Salesforce Einstein OCR API

Nowadays organizations are going for digital automation for most of their repetitive work like manual records entries from printed forms. This manual entry requirement can be for the application form, insurance forms, doctor prescription forms, examination forms, digitized business cards, and many more. This post will explain the OCR App using Salesforce Einstein to extract text from images and populate them in Salesforce objects.

For extracting text from images we have many API service available which gives almost 95% accuracy. Refer to my other post related to this service.

• Extract License Plate Number from Image In Salesforce
• Extract Text From Image using Google Cloud Vision

Salesforce announced OCR (Einstein Optical Character Recognition) service in Apr,2019. This API is now available for use.

Einstein Optical Character Recognition (OCR) leverages computer vision to analyze documents and extract relevant information, making repetitive tasks like data entry more efficient.

Let us integrate Einstein OCR in Salesforce for extracting form data. Below steps will be required to integrate it.

1. Create an account in Einstein Platform Services
2. Create a Private Key and Generate Token
3. Call Einstein OCR API from Apex
4. Extract image data in the Case object

1. Create an account in Einstein Platform Services

We have to consume Einstein OCR API so first create an API account. Create an account at https://api.einstein.ai/signup. This will send an email to your provided email. Confirm email to start working on OCR.

In the registration process, it will ask you to download a key file. Download that file, it will be used to generate tokens. The file will be saved as einstein_platform.pem. Upload this file in the Salesforce File object. Below is a screenshot of the file record of key file.

You can also follow the steps which are mentioned at Einstein Vision and Language

2. Create a Private Key and Generate Token

For integrating external API from Apex we need an API token which will require to authenticate requests. Einstein OCR API requires a valid JWT Token. This token will be generated from the above-mentioned key file.

The token can be generated online from https://api.einstein.ai/token as well but this will not work when we use API in Salesforce. We have to generate tokens at runtime before calling OCR API. We will use API https://api.einstein.ai/v2/oauth2/token for generating tokens from the apex.

Apex Class for generating Token

EinsteinController.getAccessToken() should be called to generate a token from the apex code before calling API.

3. Call Einstein OCR API from Apex

We have an API Token and API URL https://api.einstein.ai/v2/vision/ocr to extract texts from images. Let us call API from the apex with the required request data.

Request Details:

1. sampleLocation : This is the image URL. We can get a downloadable URL for our uploaded image. Refer Extract License Plate Number from Image In Salesforce for creating a downloadable URL for any uploaded image.
2. modelId : This parameter define which type of text need to be extracted from image like tabular data or business card. Value for this parameter can be OCRModel (for unstructured data) and tabulatev2 (for tabular data)

Einstein OCR API can be called using multipart/form-data and request parameter will be passed in body as blob.

blob formBlob = EncodingUtil.base64Decode(form64);
string contentLength = string.valueOf(formBlob.size());
req.setBodyAsBlob(formBlob);

Apex Code for calling API

Note: Add API URL (https://api.einstein.ai) in the remote site setting or you can use a named credential to avoid this.

4. Extract image data in the Case object

Now we are ready with consuming API service to extract images. For this post, I have created one sample image form where some field information is present. Using the above Einstein OCR API we will extract data from the image and put that in the Case object.

We need to extract field data from above-mentioned image. Similar to this we can have different forms or business cards. To extract information from above mentioned image we have to map this information in one mapping object.

Object Creation:

Create one custom object OCRTemplateMapping__c with the below fields.

Below are sample record for above mentioned image.

If you need to use other form then put these X,Y coordinates accordingly. You can check X and Y coordinates from https://yangcha.github.io/iview/iview.html . For my sample image X, Y coordinates will be like the below image.

As we have to extract image data and put that in case object so add below fields in Case object.

Now we ready with object creation. Let us write Apex code that will extract proper image data and put that in case record.

Flow to call above Apex Action:

Create a flow which call above apex method. You can refer post (Extract License Plate Number from Image In Salesforce) for flow and content url creation.

Button to call Flow:

Add one action button in case object which will call above flow.

Demo App:

References:

Einstein Vision and Language

https://developer.salesforce.com/docs/atlas.en-us.apexcode.meta/apexcode/apex_intro_what_is_apex.htm

Related Posts

1. Named Entity Recognition using Salesforce Einstein API
2. Extract Driver License Detail from Image using Einstein API
3. Extract Text From Image using Google Cloud Vision