REST Web Services using Json and requiring authentication

But first…

Registration for NAV TechDays 2017 have been opened.  I will do a workshop on web services and json.  I will be using both C/AL and AL with VS Code in this workshop.

Make sure to register for the conference and if possible go to one or two of the workshops.

Now to the topic.  Yesterday I started to develop an integration solution for bokun.io.  Their API is RESTful and uses Json file formats.  It also requires authentication.

In a project like this I usually start by using the OCR Service Setup from standard NAV.  Create a Setup table and a page.

Looking at the API documentation we can see that we need to use HmacSHA1 with both Access Key and Secret Key to authenticate.  In other project I used HmacSHA256 with the Access Key for the Azure API.

First part of the authentication is the time stamp created in UTC.  I find it easy to use the DateTime DotNet variable to solve this.  There are two different formatting I needed to use.

REST service normally just use GET or POST http methods.  The authentication is usually in the request headers.  This is an example from bokun.is

The GetSignature function is

The Secret Key string and the Signature is converted to a byte array.  The Crypto class is constructed with the Secret Key Byte Array and used to compute hash for the Signature Byte Array. That hash is also a byte array that must be converted to a base64 string.  This will give you the HmacSHA1 signature to use in the request header.

My Azure project is using HmacSHA256 but the code is similar.

Azure displays the Access Keys in base64 format while bokun.is has a normal string.

A little further down the line I choose not to use XML Ports, like I did here, but still convert Json to Xml or Xml to Json.

I use the functions from Codeunit “XML DOM Management” to handle the Xml.  This code should give you the general idea.

 

 

NAV Http Web Request

In my post about Json and Rest web services I showed how to use the standard Codeunit no. 1297 for web service communication.

Today I was asked to do this in NAV 2015.  I must admit, I forgot that this Codeunit was not available in NAV 2015.

So I made one.

This one has identical functionality to the one delivered with NAV 2016.  To catch and handle the errors I use the NAV Web Request Add-in that I created and published here on my blog.

Now I can easily move that Json code down to NAV 2015.

Download here –> COD1297-NAV2015

My first Dynamics 365 Extension – step by step – sixth step

Assisted Setup and Permissions.

But first lets look at the Extension Management Codeunit.

I want to store the appId in my Extension.  This appId is used in several tables in my tenant.  This appId must always be the same for this solution even if the solution is updated.  In my last post you can see that I am also setting my appId parameter in the Git repository.  There are few easy ways to get your appId.  NAV has the function CREATEGUID.  Powershell has the function New-Guid.  But perhaps the easiest way is to use this online Guid generator.

In OnNavAppUpgradePerDatabase – executed once when the Extension is installed, I want to assign my Extension Setup permission set to the user installing.  This is required to be able to run the Assisted Setup after installation.

In OnNavAppUpgradePerCompany – executed once in every company when the Extension is installed, I want to restore data for some of the Extension tables and delete rest of the archive data.  Even if I am just using tables as a temporary tables I still need to define what to do with the non-existing-archived data.  In here I also want to remove the persistent Assisted Setup record in every company.

So, why would I like to remove the Assisted Setup record?  Not because I like to make the user execute the setup every time the Extension is updated.  The Extension has its own Setup table.  In that Setup table I store the setup information belonging to this Extension and the Setup data is restored from archive during the installation as you can see above.

In the current release the Assisted Setup record is a database record.  I know that Microsoft intends to change this and use the same discovery pattern that is used in Service Connections.  When that happens the Assisted Setup record will be temporary only.  So, by designing my Extension in this way I make sure that I will still support Assisted Setup after this change.

In NAV 2016 we had the Mini Role Center, page 9022.  Today this Role Center has been updated to fit Dynamics 365 for Financials.

From here the user can access the Extension installation we covered in the last post, and we can also access the Assisted Setup & Tasks.

I add my Assisted Setup by subscribing to the OnOpenPage Event in the above page.

Looking at the Assisted Setup record we can see that is has a media field for an Icon.  NAV setup data includes icon for the standard Assisted Setup items – so must we.

I decided to ship my icons with the code.  Each icon has a dedicated Codeunit.

To fit an icon into a Codeunit we need to convert the binary data to base64 data.  This we can do with the following code.

This allows me to create a Codeunit like this.

And then code like this to import the icon into my help resources.

Now, back to the Assisted Setup.  I start the setup for my Extension.

This sums up what needs to be done.  The basic setup, the one done with Set Defaults, is to assign permissions to all users based on their current permissions to the tables we build our extension around.

The G/L Source Name lookup table is read from a FlowField in G/L Entries.  We can therefore expect that everyone that has read access to G/L Entries must also have read access to the lookup table. The data in the lookup table is updated when ever any of the four master tables is updated.  Hence, everyone that has access to modify any of the master tables must have access to modify the lookup table.

The Set Defaults function assigns the Extension permission sets to users and user groups based on the current permissions and that should be enough in most cases.  If the user likes the advanced way that possibility is available.

When the user presses Finish the wizard data and the assigned permission sets are applied to the tenant database.

When I created the Setup Wizard page I started with a copy of the Email Setup Wizard, page 1805.

The model I have created to assign permission sets to users and user groups with a Wizard can easily be modified to fit any Extension.  Remember that all this code will soon be available on my GitHub account.

 

Using REST/Json web services from NAV

One of my most popular blog entry is the one about Json.  I have also had some questions outside this website about this topic.

This week I got a task.  We need to communicate with a payment service that uses REST web services and Json file format.

posapi

I got a document describing the service.  Some methods use GET and some use POST.  Here is how I did this.

In the heart of it all I use Codeunit 1297, “Http Web Request Mgt.”.

getaccesstoken

Every time we talk to this POS API we send an Access Token.  If we don’t have the token in memory (single instance Codeunit), we need to get a new one.  That is what the above code does.

The ParameterMgt Codeunit is what I want to focus on.  You can see that I start by inserting my “Authorization Key” into the RequestBodyBlob.  As usual, I use the TempBlob.Blob to get and set my unstructured data.

setapirequest

The interesting part here is that I use an XMLPort to create the data I need to post to the Api.

apiauthenticatexml

A simple one in this example, but nothing says it can’t be complex.  Then I convert the Xml to Json with a single function.

converttojson

The last TRUE variable means the the Document Element will be skipped and the Json will look like it is supposed to.

apikey

The REST service response is Json.

token

And to read the Json response we take a look at the GetAccessToken function.

getaccesstokenfunction

Here I start by converting from Json to Xml.

convertfromjson

And make sure my Document Element name is “posApi”.

apiaccesstokenxml

And I have the result.

As you can see from the documentation some of the Json data is more complex.  This method will work nevertheless.

For more complex date I always create tables that matches the Json structure.  These table I use temporary through the whole process so the don’t need to be licensed tables.  Here is an example where this XMLPORT

getauthorization

will read this Json

getauthorizationjson

I suggest that with our current NAV this is the easiest way to handle REST web services and Json.

 

Asynchronous web services in NAV

Asynchronous methods are very useful for web services.  To be able to start a process with one method and then check the status with another opens a lot of possibilities.

There are two known patterns in NAV that support asynchronous methods;  one is to use the STARTSESSION function to execute the business logic in another thread, the other is to create a job queue entry and let NAS handle the task.

I was talking to a client this morning and suggested using the asynchronous web services for his tasks.  I decided to write some code and test the functionality to see if my theory was working.

The first thing I need is a table for the requests.  This table contains the identification for the queue, the status and other details.

ProcessQueueTable

A single web service method is used to create a new queue entry.

AsyncWebService

I like to use the new TryFunction for my web services.  Note that when the queue has been inserted I fire an event with the newly created queue entry.  This means that I can extend this web service with events without having to modify the web service signature.

Each process needs a dedicated Codeunit.  That Codeunit is built on top of the queue record and thereby using the parameter table pattern.

TheAsyncJob2

Note that this Codeunit is also using the TryFunction and taking care of the rollback if that function fails.

The process Codeunit catches the integration event and checks the process code before starting then selected task.

ProcessStarter2

So, if I ask the web service to start a job called StartAsyncAdjustCostItemEntriesProcess, NAV will start a new session to execute the adjust cost for item entries.  The process that creates the queue and starts a new session is very quick so the response from the web service is almost instant.

Then we wait for a moment and ask the web service about the status for the newly created queue.

CheckQueueStatus

Again using the TryFunction to make sure that my web service will have a proper response to the query.

It is quite easy to extend this module by catching the OnNewQueueInserted event for each process code you build support for.

A sample C# code that I used to test this looks like this

CSharpDemoCode2

As you can imagine we can put anything into the ProcessData variable.  By using Base64 encoding anything can be converted to a text variable.

I hope this will turn on some lights and you will be able to use this in your daily work.

Attached is a zip file with the NAV objects and the C# project.

NAVAsyncWebService

 

 

 

 

JSON meets NAV

I have been using SOAP services over the last years.  Only recently the RESTful web services have become more and more popular in my integration work.  Wikipedia says:

In computing, Representational State Transfer (REST) is a software architecture style for building scalable web services. REST gives a coordinated set of constraints to the design of components in a distributed hypermedia system that can lead to a higher performing and more maintainable architecture.

RESTful systems typically, but not always, communicate over the Hypertext Transfer Protocol with the same HTTP verbs (GET, POST, PUT, DELETE, etc.) which web browsers use to retrieve web pages and to send data to remote servers. REST interfaces usually involve collections of resources with identifiers, for example /people/paul, which can be operated upon using standard verbs, such as DELETE /people/paul.

As we are used to XML as the body for our SOAP messages we can also use XML as the body for a RESTful web service.  I just finished writing a code to communicate with Azure from NAV.  This communication was using RESTful web services and XML.

So, what is JSON?  Wikipedia says:

JSON, (canonically pronounced /ˈdʒeɪsən/ JAY-sən; sometimes JavaScript Object Notation), is an open standard format that uses human-readable text to transmit data objects consisting of attribute–value pairs. It is the primary data format used for asynchronous browser/server communication (AJAJ), largely replacing XML (used by AJAX).

Although originally derived from the JavaScript scripting language, JSON is a language-independent data format. Code for parsing and generating JSON data is readily available in many programming languages.

The JSON format was originally specified by Douglas Crockford. It is currently described by two competing standards, RFC 7159 and ECMA-404. The ECMA standard is minimal, describing only the allowed grammar syntax, whereas the RFC also provides some semantic and security considerations. The official Internet media type for JSON is application/json. The JSON filename extension is .json.

With JSON it is possible to deliver similar data structure as with XML.  JSON on the other hand requires a much less metadata.  Here is an example JSON from Wikipedia:

[code lang=”javascript”]{
"firstName": "John",
"lastName": "Smith",
"isAlive": true,
"age": 25,
"address": {
"streetAddress": "21 2nd Street",
"city": "New York",
"state": "NY",
"postalCode": "10021-3100"
},
"phoneNumbers": [
{
"type": "home",
"number": "212 555-1234"
},
{
"type": "office",
"number": "646 555-4567"
}
],
"children": [],
"spouse": null
}[/code]

There is not a good support for JSON in native .NET from Microsoft.  However, with Visual Studio, Microsoft installs an external DLL in to the folder “C:\Program Files (x86)\Microsoft Visual Studio 12.0\Common7\IDE\PrivateAssemblies”

Newtonsoft

With this Json.NET in Dynamics NAV Add-ins folder we now have some way to handle JSON files.  Using this Add-in I created a NAV Codeunit to manage JSON text.

This Codeunit contains functions to build a JSON document, like

[code lang=”csharp”]
StartJSon;
AddToJSon(‘newssn’,CompanyInformation."Registration No.");
AddToJSon(‘billtossn’,BillToCustNo);
AddToJSon(‘newcompanyname’,CompanyInformation.Name);
AddToJSon(‘newemail’,CompanyInformation."E-Mail");
AddToJSon(‘register_einvoice’,EInvoiceEnabled);
AddToJSon(‘register_supdoc’,SupDocEnabled);
AddToJSon(‘register_natreg’,NRLookupEnabled);
EndJSon;
Json := Json.Copy(GetJSon);[/code]

A function to import values from a JSON document to a temporary table, like

[code lang=”csharp”]
ReadJSon(String,TempPostingExchField);

WITH TempPostingExchField DO BEGIN
SETCURRENTKEY("Line No.","Column No.");
IF FIND(‘-‘) THEN REPEAT
SETRANGE("Column No.","Column No.");
InsertFileDetails(TempPostingExchField,WebServiceURL);
FINDLAST;
SETRANGE("Column No.");
UNTIL NEXT = 0;
END;[/code]

Or just a simple way to return a single value from a simple JSON string, like

[code lang=”csharp”]FileName := GetValueFromJsonString(String,’filename’);[/code]

With these functions NAV should be able to handle JSON files without any problems.

Now you can add JSON handling to your arsenal.

Json Codeunit and required add-ins