SAP Leonardo Machine Learning Foundation is the Machine Learning platform of SAP and it comes with a set of pre-trained and ready-to-use models. It also has (or will have in the future) the possibility to train these models with your own data or even to bring and use your own models. When we check the SAP Cloud Platform website it tells us:
“SAP Leonardo Machine Learning Foundation enables you to enhance business processes and software applications with intelligence. By consuming easy-to-use APIs you can detect and identify objects in pictures, find similar images and text contents, or extract keywords from natural language texts”.
Very interesting stuff if I may say so and enough reason to explore this further. So in this blog I will show you how to build a simple intelligent application using the SAP Leonardo Machine Learning Foundation.
SAP API Business Hub
Before we can start building our intelligent application we need to get access to the Functional Machine Learning Services that SAP provides. The SAP Leonardo Machine Learning Foundation Functional Services API’s can be found in the SAP API Business Hub. The only thing you need to make use of these services is an API key which you can obtain by logging in to the API Business Hub with your SAP account with for example your S-number.
See the list below for a selection of the services that are available:
- Classify Image (Customizable Image Classification)
- Classify Product from Image (Product Image Classification)
- Detect Face (Face Detection)
- Detect Topic (Topic Detection)
- Extract Image Features (Customizable Image Feature Extraction)
- Recognize Optical Character (OCR)
- Score Similarity (Similarity Scoring)
- Translate (Machine Translation)
If we want to build an intelligent application we have to choose a platform to build it on. Just recently I followed an introduction course where we learned how to build an application using Mendix. For those who are not familiar with Mendix, Mendix is a low-code collaborative development platform for mobile and web-based applications. At the end of the course we were sent home with one final assignment: build your own Mendix application. So with this in mind and the very convenient fact that Mendix provides the SAP Leonardo Machine Learning Foundation Connector (which you can easily download from the Mendix app Store) the choice for Mendix was made.
Building an intelligent Application
Now it is time to put everything together. Before I started building I decided that my intelligent application should be able to perform two intelligent functions. The first one is basic image recognition and the second one is that it should be able to detect faces (recognise the number of faces and where they are in a random picture). To make things easy I decided that my App should make use of the pre-trained models that SAP provides on its sandbox environment, mainly because I didn’t have a lot of alternatives. Maybe in a future blog I will try to find out how to train a model with own data, but for now the sandbox environment will suffice. So let’s get started.
Choose a Theme
As a starting point of my application I used ‘Blank App’ theme from the Mendix SAP Apps.
I mention this because I found out (the hard way) that the SAP Leonardo Machine Learning Foundation Connector was giving errors when you start with a non-SAP theme as the base of your application.
Download the SAP Leonardo Machine Learning Foundation Connector
After my Mendix application was generated the next thing I did was download the SAP Leonardo Machine Learning Foundation Connector from the Mendix app Store into my application. It will provide you with all the necessary tools to consume the SAP Functional Machine Learning services and this makes your life a whole lot easier . To make it work you only have to provide an API key and place it in the corresponding constant as shown below.
Consume a service using a MicroFlow Action
After you have downloaded the SAP Leonardo Machine Learning Foundation Connector in your application you can make use of the Microflows provided by the connector. These Microflows can then be used by your own Microflows. In the example below the ClassifyImage Microflow from the downloaded connector is used in a Microflow that is used in my own application. In the example below I am passing the $image variable (the picture in our application) as input parameter to the ClassifyImage Microflow.
The ‘options’ input field contains two attributes ModelName and ModelVersion corresponding to model name and model version in the Leonardo API. These fields will identify a specific model and a specific version to be used in the inference, but this only makes sense if you have trained your own model. For the default inference model, and when using SAP API Business Hub, this value should be empty as it is in our case.
My Intelligent Application
Before I continue I would like to mention that this is not a Mendix blog. That means I will not get into too much detail about the application itself, but I do want to emphasize that by using the SAP Leonardo Machine Learning Foundation Connector I used almost no additional coding. Now let’s take a look at the result.
When you start the application the first thing you see is the home screen which gives you the possibility to select the functional service you want to use.
If you select ‘Image Classification’ you will have the possibility to upload any picture you want. In the example below I uploaded a picture of an unknown creature I found on the internet.
By pressing the ‘Classify Image’ button the microflow is being triggered. The microflow calls the SAP service and the results, which in this case are the probability scores of the recognized image, are communicated back to the user. So let’s press the button and find out what kind of strange creature we are dealing with here!
Wow what a surprise it’s with 89% certainty a Beagle! Now I’m far from an subject matter expert regarding dogs, but it seems pretty accurate to me! But let’s check if it was not a lucky shot and see if the application recognizes one of our nations other favorite pets.
This most certainly is not a dog! Let’s see if SAP agrees.
This looks again straight on the money, a tabby cat it is and it’s also very reassuring that it’s is not rated as a radiator or a washbasin! That’s what I call an intelligent application and proves to me that everything is working as supposed.
Now let’s checkout the other application I developed; the face detection App. When the application is started a similar screen is shown as in the previous application where we can select a picture. The behaviour however is very different. This application will now tell us how many faces it is detecting in the picture and will give the location of the faces in the picture. Let’s see how it’s doing with this random picture I grabbed from the internet with some very happy people in there.
After pressing the ‘Face Detection’ button is comes up with the following promising results.
Success again! There are obviously five people in the picture and they are all detected by the SAP Machine Learning service. The coordinates are shown as well in the raw data format provided directly by the service. In fact these are the coordinates of the boundary boxes of the faces in the picture, defined by the left top (x,y) coordinate and the right bottom (x,y) coordinate both in pixels. If we represent this data graphically in the picture, instead of showing only as coordinates, it makes much more sense as shown below!
As you can see all faces are accounted for and this again proves the intelligence of my application!
I think it’s safe to say that building an intelligent application has never been so simple. The SAP Functional Machine Learning Services are simple to use and by combining it with the SAP Leonardo Machine Learning Foundation Connector in Mendix are very easy to implement in a working application. A very powerful combination indeed. In this blog we only used the SAP sandbox services, but imagine what you can achieve by training these models with your own relevant data or even to be able to bring your own models.
I hope you enjoyed reading this blog, I certainly enjoyed making it. If you have any questions feel free to contact me at firstname.lastname@example.org