Nowadays organizations are going for digital automation for most of their repetitive work like manual records entries from printed forms. This manual entry requirement can be for the application form, insurance forms, doctor prescription forms, examination forms, digitized business cards, and many more. This post will explain the OCR App using Salesforce Einstein to extract text from images and populate them in Salesforce objects.
For extracting text from images we have many API service available which gives almost 95% accuracy. Refer to my other post related to this service.
- Extract License Plate Number from Image In Salesforce
- Extract Text From Image using Google Cloud Vision
Salesforce announced OCR (Einstein Optical Character Recognition) service in Apr,2019. This API is now available for use.
Einstein Optical Character Recognition (OCR) leverages computer vision to analyze documents and extract relevant information, making repetitive tasks like data entry more efficient.
Let us integrate Einstein OCR in Salesforce for extracting form data. Below steps will be required to integrate it.
- Create an account in Einstein Platform Services
- Create a Private Key and Generate Token
- Call Einstein OCR API from Apex
- Extract image data in the Case object
1. Create an account in Einstein Platform Services
We have to consume Einstein OCR API so first create an API account. Create an account at https://api.einstein.ai/signup. This will send an email to your provided email. Confirm email to start working on OCR.
In the registration process, it will ask you to download a key file. Download that file, it will be used to generate tokens. The file will be saved as einstein_platform.pem. Upload this file in the Salesforce File object. Below is a screenshot of the file record of key file.
You can also follow the steps which are mentioned at Einstein Vision and Language
2. Create a Private Key and Generate Token
For integrating external API from Apex we need an API token which will require to authenticate requests. Einstein OCR API requires a valid JWT Token. This token will be generated from the above-mentioned key file.
The token can be generated online from https://api.einstein.ai/token as well but this will not work when we use API in Salesforce. We have to generate tokens at runtime before calling OCR API. We will use API https://api.einstein.ai/v2/oauth2/token for generating tokens from the apex.
Apex Class for generating Token
EinsteinController.getAccessToken() should be called to generate a token from the apex code before calling API.
3. Call Einstein OCR API from Apex
We have an API Token and API URL https://api.einstein.ai/v2/vision/ocr to extract texts from images. Let us call API from the apex with the required request data.
- sampleLocation : This is the image URL. We can get a downloadable URL for our uploaded image. Refer Extract License Plate Number from Image In Salesforce for creating a downloadable URL for any uploaded image.
- modelId : This parameter define which type of text need to be extracted from image like tabular data or business card. Value for this parameter can be OCRModel (for unstructured data) and tabulatev2 (for tabular data)
Einstein OCR API can be called using multipart/form-data and request parameter will be passed in body as blob.
blob formBlob = EncodingUtil.base64Decode(form64); string contentLength = string.valueOf(formBlob.size()); req.setBodyAsBlob(formBlob);
Apex Code for calling API
Note: Add API URL (https://api.einstein.ai) in the remote site setting or you can use a named credential to avoid this.
4. Extract image data in the Case object
Now we are ready with consuming API service to extract images. For this post, I have created one sample image form where some field information is present. Using the above Einstein OCR API we will extract data from the image and put that in the Case object.
We need to extract field data from above-mentioned image. Similar to this we can have different forms or business cards. To extract information from above mentioned image we have to map this information in one mapping object.
Create one custom object OCRTemplateMapping__c with the below fields.
|Field Name||Data Type||Size|
Below are sample record for above mentioned image.
If you need to use other form then put these X,Y coordinates accordingly. You can check X and Y coordinates from https://yangcha.github.io/iview/iview.html . For my sample image X, Y coordinates will be like the below image.
As we have to extract image data and put that in case object so add below fields in Case object.
|Field Name||Data Type||Size|
Now we ready with object creation. Let us write Apex code that will extract proper image data and put that in case record.
Flow to call above Apex Action:
Create a flow which call above apex method. You can refer post (Extract License Plate Number from Image In Salesforce) for flow and content url creation.
Button to call Flow:
Add one action button in case object which will call above flow.