Recently I got request from one of my blog member for extracting text from image which is uploaded in Salesforce record’s attachment. So i have analyzed some APIs which has similar capability.
There are various tool and APIs available for extracting text from images. Here are some of them which are popular in market
For this blog, i have used Google Cloud Vision to extract text from image. This OCR (Optical Character Recognition) API has special feature to recognize logo as well.
The Google Cloud Vision API enables developers to understand the content of an image by encapsulating powerful machine learning models in an easy to use REST API. You can build metadata on your image catalog, moderate offensive content, or enable new marketing scenarios through image sentiment analysis.
Let us see steps required to extract text from image.
- Enable Google Cloud Vision API
- Create Apex class to extract text from image
- Create lightning component to extract image
- Add lightning component as button on record page
1. Enable Google Cloud Vision API
First we have to enable Google Cloud Vision API and create API key for authentication purpose. Below steps can be performed to get API key.
- Create user in Google cloud at https://console.developers.google.com . We can also use any existing gmail user.
- Create project in Google Console
- Once project is created, go to API library at https://console.developers.google.com/apis/library. Select Google Cloud Vision API and enable it.
- Create Credential (API Key) for API to be used.
- Add https://vision.googleapis.com in Salesforce’s remote site setting
2. Create Apex class to extract text from image
We have to create apex class which will use Google Cloud Vision API and extract text from image. You can check API request and response detail at https://cloud.google.com/vision/docs/reference/rest.
ImageService apex class will extract text. We should create separate request and response class for passing and getting data as JSON.
3. Create lightning component to extract image
In step 2, We have created image service class which will extract text from given image’s blob data. In this step we will get image file content and pass content to above mentioned class.
ImageServiceController class will be used to get all image files on which processing will be done for extracting text.
Let us create lightning component to consume above ImageServiceController. When image is successful extracted, this component show ‘Text extracted from image. Check file record detail section’.
4. Add lightning component as button on record page
Add lightning component as button or component in your record page. If you have added that as button, when you click button it will call component and extract text from image. After you got message Text extracted from image. Check file record detail section, you can go to image file and check extracted texts.
Demo
References:
https://cloud.google.com/vision/docs/reference/rest
15 Comments
Thank You Abhinav. Let me know, if you need our help.
I am getting following error:
This API method requires billing to be enabled. Please enable billing on project #659325175485 by visiting https://console.developers.google.com/billing/enable?project=659325175485 then retry. If you enabled billing for this project recently, wait a few minutes for the action to propagate to our systems and retry
Hello Nilesh,
Looks like billing is not set for your vision project. Please check this video for that support https://www.youtube.com/watch?v=uINleRduCWM.
You can also log support ticket in google console to resolve this issue.
Thank You,
Dhanik
Hello Dhanik,
Great Post and Very Useful
One thing I need to know is how we can automatically parse the response to create a Lead Record or Contact Record. The Details are stored as the text file so how to appropriately parse the data to have accurate data?
Your help is much appreciated
Thanks in Advance
Hello Vineet,
You can use BoundingPoly (text location coordinates) for each text section. Check link https://cloud.google.com/vision/docs/handwritinghttps://cloud.google.com/vision/docs/handwriting
Thank You,
Dhanik
Or I mean to say how to identify the data like this is the name and this is address etc
Hello Vineet,
You can use BoundingPoly (text location coordinates) for each text section.
Thank You,
Dhanik
Pingback: Create OCR App using Salesforce Einstein OCR API | SalesforceCodex
Hi Dhanik, Thank you for sharing this wonderful post. I have a query regarding component attribute “recordId”. Where does this attribute get its value from?
Thanks, Sandeep
value for recordid is populated by aura framework for current record. see documentation for this.
Thank you,
Dhanik
Hello Dhanik,
Great post, one doubt will the same code implementation work for getting a text from PDF file if we query the pdf data in “List docs=[SELECT ContentDocumentId FROM ContentDocumentLink
WHERE ContentDocument.FileType=’JPG’ and LinkedEntityId =:record];”
and use “https://vision.googleapis.com/v1/files:asyncBatchAnnotate?key=your-api-key” as end point URL.
Please suggest.
Thanks & Regards,
Rahul S
Hello Rahul,
Concept will be same for PDF also. You have to change some request attribute. Please sample request at https://cloud.google.com/vision/docs/pdf.
Thank You,
Dhanik
I’m getting the error, “Variable does not exist” for Extracted_Text__c and Is_Extracted__c. Under what object are we supposed to create these variables?
Hello Jonathan,
It will be created in ContentVersion object.
Thank You,
Dhanik
Pingback: Ultimate Guide to Integrate Stripe with Salesforce CRM