Extract Driver License Detail from Image using Einstein API

Einstein Vision API help to extract text from different image formats. We can extract information from any tabular data, form data or business card’s image. This post will give step by step information about extracting driver license detail from Image using Einstein API in Salesforce.

This post will be applicable for extracting below other type of form image data extraction.

• Driver’s License
• Form 1040
• Form W-2
• Passport
• Pay Stub
• Permanent Resident Card

This process will help us in getting data extracted immediately and reduce lot of time to process that information. Some time it can help govt agency like passport office, transport office in validating detail faster and can help in taking decision on time.

Steps to integrate Salesforce and Einstein Vision API

1. Create account in Einstein Platform Services
2. Create Private Key and Generate Token
3. Call Einstein OCR API from Apex
4. Extract image data in Case object
5. Test Code

1. Create account in Einstein Platform Services

Setup account at https://api.einstein.ai/signup. See step 1 from Create OCR App using Salesforce Einstein OCR API for more detail

2. Create Private Key and Generate Token

Create private key and token to call API. Check step 2 from Create OCR App using Salesforce Einstein OCR API for more detail

3. Call Einstein OCR API from Apex

After setting up API and authentication detail, we are now ready with API call which will extract detail from uploaded image in case record. For this blog, I am using case object but you can use any other object based on requirement. To extract tabular or business card data we have to use x,y co-coordinates to get specific data (refer our previous post for this). To extract form data, Einstein API gives named entities like first_name, last_name etc from image, no need to check coordinates to extract information. Once we extracted data, we can update detail in case object.

Let us create apex code which are required for recognizing and extracting driver license detail.

Code File Detail:

• EinsteinController.apxc: This class will get access token to call Einstein API. I have used salesforcecodex@gmail.com as I have registered using this email. Change this email in your code based on user created by you.
• EinsteinFormOCR.apxc: This class will extract named entities from drive license image. We have to use Form and DL as type in parameter.
• FormContentExtractor.apxc: This class will get image from case object. Try to upload image less then 100kb so that we can use it in Apex. For bigger size image will take longer time to process image so it will not work in all scenario.
• FormResponse.apxc: This is response wrapper class.

Once we call FormContentExtractor.getImageText method, we will get output in below format.

address=123 SAMPLE STREET PHOENIX AZ, date_of_birth=09/13/1999, document_discriminator=1234567890136, endorsements=NONE, expiration_date=09/13/2021, first_name=JANE, formtype=driver LICENSE, hair=BRO, height=5'06", issue_date=09/13/2020,

We have to updated these data in case object where image data is extracted. As these data can be different for different image type. We need some kind of mapping information to store these data in case object. So let us create a mapping object for this data and salesforce object where data need to be updated.

We can create object with name Form Template Mapping to hold mapping information. Below fields will be added in this object.

Here is some sample mapping data for driver license form image.

Now we have mapped our data and object field, let us write code to updated these fields based on response and mapping.

Line# 24 to 35 checking what response we received from image text extraction and based on that response what mapping we have in form data mapping record. If we have matching record then we will update corresponding field value.

As this class’s method getFormData is created as InvocableMethod so that we can execute this from flow.

Let us create flow to call method getFormData which will be available as action Extract Driver License Detail in flow.

Steps for flow

1. Create a recordId input variable in flow. This will get record on which page flow will be added
2. Call action Extract Driver License Detail
3. Show message for action result

If you are not able to accomplished above flow step then here is link to see flow meta xml document.

Add Flow as action button on Case object:

Add above flow as action button and then add that button to page layout.

5. Test Code

Related Einstein OCR Posts

1. Named Entity Recognition using Salesforce Einstein API
2. Create OCR App using Salesforce Einstein OCR API
3. Extract Text From Image using Google Cloud Vision