How to integrate Automation Anywhere IQ Bot with Power Platform

This is a continuation of the OCR series, previously we showed how to extract data from PDF using Power Platform”AI Builder” and Azure “Form Recognizer“. In this post, we will see how to extract data using IQ Bot.

Automation Anywhere is one of the leading RPA tools, In Power Platform also we have “Power Automate desktop” to cover this segment but it is new to this space. Clearly, AA stands in the top 3 in the RPA market. It offers a variety of bots to automate the processes. IQ bot is used to read the image, pdf files. Unlike AI builder and Form Recognizer, Automation anywhere does not have a separate OCR engine but it has collaboration with multiple OCR engines like Azure OCR, Abby, Tessaract, Google Vision. So IQ Bot is a middle man software that facilitates the ease of training the documents with UI like the labeling tool for azure but with some more features.

Open the IQ Bot instance, it will show all the learning instances and bots for the particular environment.

Create New instance,

1. Enter the document name
2. Select Document Type, predefined types will be available. If we select one it will automatically generated the fields and we can add on top of it.

For this case, we will select as other and manually entered the fields,

Expand the advanced settings, as I said earlier multiple OCR engines are available. By default, Abby will be selected and we should select based on our needs. Our experience with OCR engines is as follows.

1. Abby – Good for printed text documents and will give accurate checkbox results. For Handwritten text it will give very poor results.
2. Azure OCR – It will give good results for both handwritten and printed text. AI builder has Azure OCR in the background, In this example also we will use Azure OCR.

If we uncheck the “My PDF documents do not have images” it will automatically use the “PDFBox” engine, So make sure to uncheck.

Click create a new instance, it will take some time to generate the System Identified Regions(SIR) in the documents.

As we can see in the document, all the areas enclosed within the blue borders are the SIR for this document. Even if we manually drag the area outside the SIR’s it will not capture.

We should select each field label, field value and validate the selection areas and if it did not map correctly we should adjust it as shown below, In this Example Sales person field is mapped to the adjacent area, we can map it to the bottom of the label.

For each field, we can write python script in the logic to manipulate the captured data. In the Eg we are appending a prefix value to the invoice no which may seem simple, In one scenario we manipulated a complex unstructured table to a structured table.

For capturing the tables, a reference column should be selected to calculate the no of rows for that table, based on the column it will render the number of rows.

Table mapping is pretty much simple, For both label and value the header area should be selected. It will do the rest of the things for us.

All the fields should be selected as optional otherwise it will wait for user interaction when we call this learning instance.

Select “See extraction results” to see the output for the current document as we can see for the table values it is showing with a red border because I purposefully left it as required for this demo.

Next step is to run the IQ bot instance automatically with the help AA task bot,

1. Call the IQ Bot learning instance using Task bot.
2. Pass the CSV to Power Automate.

why we are passing to power automate, For this instance it is fine all the data is captured correctly, But in real time scenarios the document will not be good handwritten texts, low quality document may come as input. So before pushing it to database, User has to verify the data, for that we can build a canvas app to verify the data in parallel with PDF.

We created a simple task bot for this process, let’s see what each step will do,

In this example,

1. PDF are kept in the folder
2. All the PDF’s in the folder are looped and passed in the IQ Bot upload step

1. Processed results will be saved as “CSV” format in the “Success” folder.
2. CSV files are looped and converted into “Base 64” format using VB Script.

inFile =  WScript.Arguments.Item(0)
//Gets the input file read
inByteArray = readBytes(inFile)
//Encode the data from the input file to Base64Encoded
base64Encoded = encodeBase64(inByteArray)

Function readBytes(File)
Dim inStream
Set inStream=CreateObject("ADODB.Stream")
inStream.Open
inStream.type=1
inStream.LoadFromFile file
readBytes=InStream.Read()
End Function

Function encodeBase64(bytes)
Dim Cobj, CElem
Set CObj = CreateObject("Microsoft.XMLDOM")
Set CElem = Cobj.createElement("tmp")
CElem.DataType = "bin.base64"
CElem.NodeTypedValue = bytes
encodeBase64 = CElem.Text
End Function

1. Converted Base 64 text will be passed into “Rest API” web service action. For this we need to create a Power Automate with “HTTP Trigger” with POST request.
2. Change the Header to application/json
3. Navigate to Parameter tab and select the Post section.
4. Pass the data in the below format.
5. Run the script

{
"fileName":"strFileName",
"csv":"base64Text"
}

Power automate triggered successfully and we can convert it back to CSV file, then it is in our hand we can send it as attachment or read the csv and push into a database or upload the csv in Sharepoint or create the file to local machine.

Another tip here is instead of processing the PDF directly it should be converted to an image and then passed into IQ Bot for better results. For converting into image the settings should be changed as per your document.

We will see a detailed analysis of the the three processing softwares in detail.

Please post your queries in the comment section. Happy Building 🙂