Object Detection with PowerApps AI Builder

Power Apps recently got the capability to create some computer vision AI models. One of the IMHO most popular ones is called object detection, which is used to detect (predefined) objects on images. For getting started you may use the free edtion of power apps.

Environment and CDM Entity

AI Builder is bound to a Power Apps environment and the Common Data Model. If you don’t have already created an environment, logon to https://web.powerapps.com and create a new environment.

Create new environment

To use the CDM entities you need a new database. In my case, I created a new one with USD and English as preferred language.

Create new database

It may take a while, and you may need to refersh your browser screen, but the AI Builder (Preview) option will appear on the left menu bar.

Create new Power Apps AI Builder Object Detection model

Next, go to Data > Entites and create a new entity for the type of objects you want to identify. In my case, I’m playing around with Nerf guns, therefore I created a new Nerfgun entity. It requires at least a useful name. Feel free to add more fields.

Select entity from Common Data Model

Provide entity data

In a next step you have to provide information about the different elements that shall be identified. In my cases, which Nerfs guns will be on fotos e.g. Rapidstrike, Slingfire, Cyclonshot, etc.

Entity in Common Data Model

There is an option to edit the entity via Excel. However, in my case the Excel addin is not working, and failing to authenticated 😦 If this happens to you, a workaround is to create a simple Power App and edit the entity via Power App.

Fill CDM entity with data using Power Apps

Create and train AI model

In Power Apps go to AI Builder and create a new model for object detection.

Create new AI model for object detection

Provide a name for the model and select the created entity for detection. From the records in the entity select those records that are relevant for detection.

Select object to detect using AI builder

Next comes the time consuming part, take pictures of your objects in different locations, lighting, quality, day time, etc. upload it to the power apps model. You need at least 15 pictures for each element you want to detect.

Upload images to Power Apps AI Builder

Tag each of the pictures with the corresponding object on the picture.

Tag object on images in Power App AI Builder

After uploading and tagging each picture, train the model. This may take a while and will result in a (not very usefull) quality estimation. Right now I didn’t see recall, precision, AUC, or any other more detailed information. If you are satisfied with the models estimated power, publish it so it can be used in your apps.

Train AI model for object detection

Use AI model in Power Apps

It’s easy to use the trained model in a Power Apps application e.g. on the smart phone. Create a new power app with empty layout. From the menu insert the object detection component.

Create a new PowerApps app with AI Builder

Select your object detection model for the component. Save and publish your app, load it on your phone and test it.

Test Power Apps AI Builder in real life

Dynamics 365 FO: Export Entity Store to Azure Data Lake

Since version 10 Dynamics 365 for Finance and Operations supports the entity store export to Azure data lake. The main benefits are reduced costs because Azure Cloud storage is cheap and easy access for Business Intelligence tools like PowerBI.

If you are running a local development VM, the data connection tab in system parameters ist deactived by default. However, this can be actived using the SysFlighting table.

The configuration is pretty well documented by Microsoft. I’ve performed all the necessary steps and recorded a video:

Call REST Webservice with HTTP Basic Authentication from X++

Calling a web service with HTTP Basic Authentication is easy in C#. Here I’m using a REST service via HTTP GET which is secured via Basic Authentication.

C# Code

A HTTP GET webservice call using System.Net.WebRequest and System.Net.Webrespose in C#

string url = “http://yourhost.net/service”;
string user = “YourUserName”;
string pass = “YourPassWord”;

byte[] bytes = System.Text.Encoding.UTF8.GetBytes(user+”:”+ pass);
string base64  = System.Convert.ToBase64String(bytes);

WebRequest request = WebRequest.Create(url);
request.Headers.Add(“Authorization”, “Basic “+base64);
WebResponse response = request.GetResponse();
string wsResponse = new StreamReader(response.GetResponseStream())
.ReadToEnd();

Console.WriteLine(wsResponse);
Console.ReadKey();

X++ Code

Since Dynamics 365 FO does not support packages and some syntactic sugar from C# the code is more wordy.

str url = “http://yourhost.net/service”;
str user = “YourUserName”;
str pass = “YourPassword”;

System.Text.Encoding encoding = System.Text.Encoding::UTF8;
System.Byte[] bytes = encoding.GetBytes(user + “:” + pass);
System.String base64 = System.Convert::ToBase64String(bytes);

System.String headerKey = “Authorization”;
System.String headerValue = “Basic ” + base64;
System.Net.WebRequest request = System.Net.WebRequest::Create(url);       
System.Net.WebHeaderCollection headers = request.Headers;
headers.Add(headerKey,headerValue);

System.Net.WebResponse response = request.GetResponse();
System.IO.StreamReader reader = new System.IO.StreamReader(
response.GetResponseStream());
str wsResponse = reader.ReadToEnd();

info(wsResponse);

Write product configuration model attributes to BOM and ProdBOM

pctn

https://www.youtube.com/watch?v=BaHYbDQJB_A

The product configuration model in Dynamics 365 for Finance and Operations is a great tool dynamically generate production orders including bill of materials (ProdBOM) and routes. In the product configuration model, the user is free to define attributes of different types, and use these attributes for calculations and conditions within the model. However, I had to come up with a solution to write the attribute value from the production configuration wizard to the generated ProdBOM. Here is an example code how this can be done:

Data type, table and form extensions

  1. Create a new string EDT and call it ERPPCStringAttribute
  2. Create table extensions for the BOM and ProdBOM and add a new field based on the ERPPCStringAttribute EDT
  3. Create a form extension for the ProdBOM and add the new field in the grid (Tab > Overview > Grid)

Extend the PCBOMLineDetails form

This is a tricky part. In the form for the BOM line details in the product configuration model, create a section for the attribute that looks the same like all the others. Here is a screenshot from Visual Studio:

image

You need to create a group, hat contains another group, with a checkbox and group with a string edit field and a radio button. Make sure to name the elements like shown in the screenshot.  If you are not sure, compare it with the other existing groups e.g. the SubContractor. You will need to compare the properties of each element with the corresponding properties of an existing element to make it look similar. 

image

Extend PCBOMLineDetails Form UI logic

You cannot overwrite methods on UI elements in form extensions e.g. clicked() . Therefore you have to implement the logic for the UI elements in a separate class. Create a new class ERPPCBomLineDetailsEventHandler and implement the following UI logic for clicked(), modified() and lookup()

class ERPPCBomLineDetailsEventHandler
{
   
    [FormControlEventHandler(formControlStr(PCBOMLineDetails, 
                             AllocateStringAttribute),
                             FormControlEventType::Clicked)]
    public static void AllocateAttributeString1_OnClicked(FormControl sender,
                                                     FormControlEventArgs e)
    {
        FormCheckBoxControl chx = sender;

        FormControl allocationGroup_AttributeString1 =
                sender.formRun().design().controlName(formControlStr(
                                             PCBOMLineDetails,
                                             allocationGroup_StringAttribute)
                                                       );
        FormControl attributeString1_Allocation =
                 sender.formRun().design().controlName(formControlStr(
                                             PCBOMLineDetails,
                                             StringAttribute_Allocation)
                                                       );

        PCModelingLibrary::templateSetEnabledStatus(
                                            (chx.value() == NoYes::Yes), 
                                            allocationGroup_AttributeString1,
                                            attributeString1_Allocation
                                                   );
    }

    [FormControlEventHandler(formControlStr(PCBOMLineDetails, 
                             StringAttribute_Allocation),
                             FormControlEventType::Modified)]
    public static void AttributeString1_Allocation_OnModified(
                                                      FormControl sender,
                                                      FormControlEventArgs e)
    {
        FormRadioControl radio = sender;
        FormStringControl attributeString1 =
                      sender.formRun().design().controlName(formControlStr(
                                                            PCBOMLineDetails,
                                                            StringAttribute)
                                                             );

        str label = attributeString1.labelText();

        attributeString1.text(”);

        if (radio.selection() == PCAllocation::Value)
        {
            attributeString1.extendedDataType(
                                        extendedTypeNum(ERPPCStringAttribute)
                                              );
        }
        else
        {
            attributeString1.extendedDataType(
                                        extendedTypeNum(ERPPCStringAttribute)
                                              );
            attributeString1.label(label);
        }
    }

     
    [FormControlEventHandler(formControlStr(PCBOMLineDetails, 
                             StringAttribute),
                             FormControlEventType::Lookup)]
    public static void AttributeString1_OnLookup(FormControl sender,
                                                 FormControlEventArgs e)
    {
        Object PCBOMLineDetails = sender.formRun();
        PCClass component = PCBOMLineDetails.component();
        FormRadioControl radio = 
           
sender.formRun().design().controlName(formControlStr(
                                                  PCBOMLineDetails,
                                                  StringAttribute_Allocation)
                                                 
);

        if(radio.selection() == PCAllocation::Attribute)
        {
            PCModelingLibrary::attributeLookup(sender, component);
        }       
    }

}

Extend the PCBOMLineDetails class-behind

Like many forms in Dynamics 365 for Finance and Operations the form has a class-behind that implements the business logic. You need to extend this class in order to deal with the newly created  Attribute group. Create a new class ERPPCBomLineDetails_Extension and impelement the following logic:

[ExtensionOf(formStr(PCBOMLineDetails))]
final class ERPPCBomLineDetails_Extension
{
    public PCClass component()
    {
        return component;
    }

    [ExtensionOf(FormMethodStr(PCBOMLineDetails,loadAllocations))]
    public void loadAllocations()
    {
        PCTemplateAttributeBinding      templateAttributeBinding;
        PCTemplateAttribute             fieldReference;


        PCTemplateAttributeBinding    findBindingByIDs(TableId _tableId, 
                                                       FieldId _fieldId)
        {
            PCTemplateAttributeBinding binding;

            fieldReference =
                 templateFind.findTemplateAttributeByTableIdAndFieldId(
                                                                _tableId,
                                                                _fieldId
                                                                );
           
            select  firstonly binding
            where   binding.TemplateAttribute == fieldReference.RecId
            &&      binding.TemplateComponent == templateComponent.RecId;

            return binding;
        }

        next loadAllocations();

        templateAttributeBinding = findBindingByIDs(tableNum(BOM), 
                                                    fieldNum(BOM,
                                                    ERPPCStringAttribute));


        PCModelingLibrary::templateLoadStringAllocation(component,
                                            templateAttributeBinding,
                                            StringAttribute,
                                            AllocateStringAttribute,
                                            StringAttribute_Allocation,
                                            AllocationGroup_StringAttribute);


    }

    [ExtensionOf(FormMethodStr(PCBOMLineDetails,updateRadioControls))]
    public void updateRadioControls()
    {
        next updateRadioControls();

        PCModelingLibrary::templateSetEnabledStatus(
                            (AllocateStringAttribute.value() == NoYes::Yes), 
                            AllocationGroup_StringAttribute,
                            StringAttribute_Allocation);

    }

    [ExtensionOf(FormMethodStr(PCBOMLineDetails,writeAllocations))]
    public void writeAllocations()
    {
        PCTemplateAttribute             fieldReference;

        next writeAllocations();

        if ((AllocateStringAttribute.value() == NoYes::Yes))
        {
            fieldReference = 
                   templateFind.findTemplateAttributeByTableIdAndFieldId(
                                      tableNum(BOM),
                                      fieldNum(BOM, ERPPCStringAttribute));


            PCModelingLibrary::templateSaveStringAllocation(component, 
                                               templateComponent,
                                               fieldReference,
                                               StringAttribute,
                                               StringAttribute_Allocation);
        }
    }

}

Extend the product configuration model framework

Implement the following classes to extend the product configuration model framework in Dynamics 365 Finance and Operations:

PcAdaptorBOMLine class:

[ExtensionOf(classStr(PcAdaptorBOMLine))]
final class ERPPcAdaptorBOMLine_Extension
{
    public ERPPCStringAttribute parmWANPCStringAttribute(
            ERPPCStringAttribute _stringAttribute = bom.ERPPCStringAttribute)
    {
        EcoResTextValue value;

        value.TextValue = _stringAttribute;
        this.fieldAssignment(bom.TableId,
                             fieldnum(BOM, ERPPCStringAttribute),
                             value);

        bom.ERPPCStringAttribute = _stringAttribute;                   
        return bom.ERPPCStringAttribute;
    }

}

PCGenerateBOMLine

[ExtensionOf(classStr(PCGenerateBOMLine))]
final class ERPPCGenerateBOMLine_Extension
{
    protected void setupBOM(
        BOMId       _bomId,
        InventDim   _inventDim,
        InventTable _inventTable,
        boolean     _isProduction
    )
    {
        next setupBom(_bomId,_inventDim,_inventTable,_isProduction);

        bom.ERPPCStringAttribute = templateFind.getBindingValueAsString(
                                       tableNum(BOM),
                                       fieldNum(BOM,ERPPCStringAttribute));
    }

}

Re-Initialize the framework template records

By default the framework does not recognize to create template records for the newly added attribute field. Therefore you have to delete the existing templates and trigger the framework to reinitialize. Be aware, this might harm your existing models!

PCTemplateInitialize class:

[ExtensionOf(classStr(PcTemplateInitialize))]
final class ERPPCTemplateInitialize_Extension
{
public static void main(Args _args)
{
PCTemplate tableTemplate;
PcTemplateInitialize init = PcTemplateInitialize::construct();
delete_from tableTemplate;
init.run();
}
protected void createTemplatesForBOM()
{
next createTemplatesForBom();
PCTemplate tableTemplate;
select firstonly tableTemplate where
tableTemplate.ReferencedTableId == tableNum(BOM);
this.createFieldTemplate(tableTemplate,fieldNum(BOM,
ERPPCStringAttribute ));
}
}

Start this class from Class Runner or add a menu item to call it by hand.

Test your Implementation

  1. Create a new product configuration model
  2. Add a new attribute “notes” of type string
  3. Add a BOM line to the model
  4. Open the BOM line details and assign the notes attribute to the string attribute 1
    image
  5. Save the model
  6. Create and approve the model version
  7. Create a new production order
  8. Use the configuration wizard to provide a text value for the notes attribute
    image
  9. Create the production order
  10. Verify that you can see the attribute value in the ProdBOM of your production order
    image

Send SSRS Report as Email from Report Viewer in Dynamics 365 FO

A very common task required by Dynamics 365 Finance and Operations clients is to send a report directly from the report viewer. This can be achieved with a view lines of Code. Here is video on Youtube how it works:

image
https://www.youtube.com/watch?v=0yQ8wvzzJSg

To achieve this create an extension of the SRSReportViewerForm, add a “Send Email” button. Create a OnClicked Event Handler in a Class.

[FormControlEventHandler(formControlStr(SrsReportViewerForm,
FormButtonControl1), FormControlEventType::Clicked)]
public static void FormButtonControl1_OnClicked(FormControl sender, FormControlEventArgs e)
{
SrsReportViewerControl ct = sender.formRun().design(0).controlName('ReportViewerControl');
object viewerForm = ct.formRun();
SrsReportRunController ctr= viewerForm.controller();
Microsoft.Dynamics.AX.Framework.Reporting.Shared.ReportingService.ParameterValue[] parameterValueArray;
Map reportParametersMap;

SRSReportRunService srsReportRunService = new SrsReportRunService();
srsReportRunService.getReportDataContract(ctr.parmreportcontract().parmReportName());
srsReportRunService.preRunReport(ctr.parmreportcontract());
reportParametersMap = srsReportRunService.createParamMapFromContract(ctr.parmReportContract());
parameterValueArray = SrsReportRunUtil::getParameterValueArray(reportParametersMap);str fileName = ctr.parmReportName()+ ".pdf";

SRSPrintDestinationSettings settings;
settings = ctr.parmReportContract().parmPrintSettings();
settings.printMediumType(SRSPrintMediumType::File);
settings.fileName(fileName);
settings.fileFormat(SRSReportFileFormat::PDF);
SRSProxy proxy = SRSProxy::constructWithConfiguration(SRSConfiguration::getDefaultServerConfiguration());

System.Byte[] reportBytes = proxy.renderReportToByteArray(ctr.parmreportcontract().parmreportpath(),
parameterValueArray,settings.fileFormat(),settings.deviceinfo());

if(reportBytes.Length > 0)
{
SysMailerMessageBuilder messageBuilder = new SysMailerMessageBuilder();
messageBuilder.setSubject(fileName);
messageBuilder.addAttachment(new System.IO.MemoryStream(reportBytes),fileName);
SysMailerFactory::sendInteractive(messageBuilder.getMessage());
}
}

Product Configuration Model in Dynamics 365 Finance and Operations

This video explains how you can use the product configuration model in D365 FO to model a heater. The heater can have a length between 0.5 m – 4,5 m, can have a certain color and provide an analog handle, a touch panel or a remote control. Touch panel and remote control is only available for heaters > 1m . A heater contains of heating units (4x 1m), requires connector modules between each 4 heating units and an enclosed. The product configuration model is used to define these requirements and constrains, provide a wizard for production configuration and generates the correct BOM.

image

Feature-Based reuse in the ERP domain: An industrial case study

Enterprise Resource Planning (ERP) system vendors need to customize their products according to the domain-specific requirements of their customers. Systematic reuse of features and related ERP product customizations would improve software quality and save implementation time. In our previous research, we have developed a tool-based approach supporting feature-based reuse of ERP product customizations. Our tool environment automatically infers reusable features from requirements and their associated implementation artifacts. Furthermore, it allows domain experts to search for features based on natural language requirement descriptions representing the needs of new customers. Matching features can be automatically deployed to a new ERP product. In this paper, we present an industrial evaluation of our tool-based approach conducted together with an Austrian small- to medium-sized enterprise. A domain expert used our approach to identify matching features for 20 randomly selected requirements for new customer products and identified matching features for 17 of the 20 requirements. We compared the time needed to identify and deploy the candidate features with the time required to implement the requirements from scratch. We found that, in total, over 60% implementation time can be saved by applying our reuse approach presented in this case study.

The actual paper regarding reusing of ERP customizations across multiple instances for Dynamics AX has been presented at the 22nd International System and Software Product Line Conference 2018 in Gothenburg (SE). The paper is available at ACM