AdvaniaGIT: About the build steps

The goal of this post is to demo from start to finish the automated build and test of an AL solution for Microsoft Dynamics 365 Business Central.

About the build steps

All build steps are execute in the same way.  In the folder ‘C:\AdvaniaGIT\Scripts’ the script ‘Start-CustomAction.ps1’ is executed with parameters.

The AdvaniaGIT custom action is executed in the same way from a build machine and from a development machine.

When we created the container in our last post from Visual Studio Code with the command (Ctrl+Shift+P) ‘Build NAV Environment’, Visual Studio Code executed

From the build task we execute ‘C:\AdvaniaGIT\Scripts\Start-CustomAction.ps1’ with these parameters

We can see that these commands are almost the same.  We have the one additional parameter in the build script to notify the scripts that we are in Build Mode.

Each AdvaniaGIT build or development machine has a ‘C:\AdvaniaGIT\Data\GITSettings.Json’ configuration file.

When the scripts are started this file is read and all the settings imported.  Then the repository setup file is imported.  The default repository setup file is ‘setup.json’ as stated in GIT settings.  If the same parameters are in both the machine settings and in the repository settings then the repository settings are used.

The same structure is used for the ‘BuildSettings’ parameter that can be passed to the custom action.  The build settings will overwrite the same parameter in both the machine settings and the repository settings.

The default settings are built around the folder structure that I like to use.  For example, we have our C/AL objects in the ‘Objects’ folder.  Microsoft has their objects in then ‘BaseApp’ folder.  Just by adding the ‘objectsPath’ parameter to the repository settings for the Microsoft repository I can use their structure without problems.

If I wan’t to execute the same exact functionality in Visual Studio Code as I expect to get from my build script I can add the ‘BuildSettings’ parameter to the command.

The folder structure

The structure is defined in settings files.  By default I have the ‘AL’ folder for the main project and the ‘ALTests’ folder for the test project.  Example can be seen in the G/L Source Names repository.

In C/AL we are using deltas and using the build server to merge our solutions to a single solution.  Therefore we have a single repository for a single NAV version and put our solutions and customization into branches.

In AL this is no longer needed.  We can have a dedicated repository for each solution if we like to, since the scripts will not be doing any merge between branches.

AdvaniaGIT: Setup and configure the build machine

The goal of this post is to demo from start to finish the automated build and test of an AL solution for Microsoft Dynamics 365 Business Central.

Setup and configure the build machine

We will create our build machine from a standard Windows 2016 template in Azure.

Docker containers and container images will take a lot of disk space.  The data are stored in %ProgramData%\docker

It if obvious that we will not be able to store the lot on the system SSD system drive.  To solve this I create an 1TB HDD disk in Azure.

After starting the Azure VM and opening the Server Manager to look at the File and Storage Service we can see the new empty disk that need configuration.

Right click the new drive to create a new volume.

And assign the drive letter

Next go to Add roles and features to add the Containers feature.  More information can be found here.  We also need to add ‘.NET Framework 3.5 Features’.

I also like to make sure that all Microsoft updates have been installed.

Now I start PowerShell ISE as Administrator.

As Windows Servers are usually configured in a way that prohibits downloads I like to continue the installation task in PowerShell.

To enable all the scripts to be executes we need to change the execution policy for PowerShell scripts.  Executing

will take care of that. 

Confirm with Yes to all.

To make sure that all the following download functions will execute successfully we need to change the TLS configuration with another PowerShell command.

Let’s download Visual Studio Code!  Use the following PowerShell command

to download the installation file to your desktop.  Start the installation.  During installation I like to select all available additional tasks.

We also need to download GIT.  Using the following PowerShell command

will download the latest version at the time of this blog post.  The only thing I change from default during GIT setup is the default editor.  I like to use Visual Studio Code.

Go ahead and start Visual Studio Code as Administrator.

Add the AdvaniaGIT extension to Visual Studio Code

Install AdvaniaGIT PowerShell Scripts!  We access the commands in Visual Studio Code by pressing Ctrl+Shift+P.  From there we type to search for the command ‘Advania: Go!’ and the when selected we press enter.

You will get a small notification dialog asking you to switch to the AdvaniaGIT terminal window.

Accept the default path for the installation but select No to the two optional installation options.

We need a development license to work with NAV and Business Central.  This license you copy into the ‘C:\AdvaniaGIT\License’ folder.  In the ‘GITSettings.json’ file that Visual Studio Code opened during AdvaniaGIT installation we need to point to this license file.

The DockerSettings.json file is also opened during installation and if you have access to the insider builds we need to update that file.

If not make sure to have all setting blank

Save both these configuration files and restart Visual Studio Code.  This restart is required to make sure Visual Studio Code recognizes the AdvaniaGIT PowerShell modules.

Let’s open our first GIT repository.  We start by opening the NAV 2018 repository.  Repositories must have the setup.json file in the root folder to support the AdvaniaGIT functionality.

I need some installation files from the NAV 2018 DVD and I will start by cloning my GitHub NAV 2018 respository.  From GitHub I copy the Url to the repository.  In Visual Studio Code I open the commands with Ctrl+Shift+P and execute the command ‘Git: Clone’.

I selected the default folder for the local copy and accepted to open the repository folder.  Again with Ctrl+Shift+P I start the NAV Installation.

The download will start.  The country version we are downloading does not matter at this point.  Every country has the same installation files that we require.

This will download NAV and start the installation.  I will just cancel the installation and manually install just what I need.

  • Microsoft SQL Server\sqlncli64
  • Microsoft SQL Server Management Objects\SQLSysClrTypes
  • Microsoft Visual C++ 2013\vcredist_x64
  • Microsoft Visual C++ 2013\vcredist_x86
  • Microsoft Visual C++ 2017\vcredist_x64
  • Microsoft Visual Studio 2010 Tools For Office Redist\vstor_redist

To enable the windows authentication for the build containers we need to save the windows credentials.  I am running as user “navlightadmin”.  I securely save the password for this user by starting a command (Ctrl+Shift+P) and select to save container credentials.

For all the docker container support I like to use the NAV Container Helper from Microsoft.  With another command (Ctrl+Shift+P) I install the container helper to the server.

To complete the docker installation I execute.

in Visual Studio Code Terminal.

We need to point docker to our data storage drive.  Kamil Sacek already pointed this out to us.

I use Visual Studio Code to update the docker configuration.  As pointed out here the default docker configuration file can be found at ‘C:\ProgramData\Docker\config\daemon.json’. If this file does not already exist, it can be created.  I update the ‘data-root’ configuration.

Now let’s restart the server by typing

or manually.

After restart, open Visual Studio Code as Administrator.

Now to verify the installation let’s clone my Business Central repository.  Start command (Ctrl+Shift+P) ‘Git: Clone’ and paste in the Url to the repository.

This repository has a setup.json that points to the Business Central Sandbox.

Make sure to have the Integrated Terminal visible and let’s verify the installation by executing a command (Ctrl+Shift+P) ‘Advania: Build NAV Environment’ to build the development environment.

The image download should start…

You should now be able to use the command (Ctrl+Shift+P) ‘Advania: Start Client’,  ‘Advania: Start Web Client’, ‘Advania: Start FinSql’ and ‘Advania: Start Debugger’ to verify all the required NAV/BC functionality.

If you are happy with the results you should be able to install the build agent as shown by Soren Klemmensen here.

 

Using the Translation Service for G/L Source Names

Until now I have had my G/L Source Names extension in English only.

Now the upcoming release of Microsoft Dynamics 365 Business Central I need to supply more languages.  What does a man do when he does not speak the language?

I gave a shout out yesterday on Twitter asking for help with translation.  Tobias Fenster reminded me that we have a service to help us with that.  I had already tried to work with this service and now it was time to test the service on my G/L Source Names extension.

In my previous posts I had created the Xliff translation files from my old ML properties.  I manually translated to my native language; is-IS.

I already got a Danish translation file sent from a colleague.

Before we start; I needed to do a minor update to the AdvaniaGIT tools.  Make sure you run “Advania: Go!” to update the PowerShell Script Package.  Then restart Visual Studio Code.

Off to the Microsoft Lifecycle Services to utilize the translation service.

Now, let’s prepare the Xliff files in Visual Studio Code.  From the last build I have the default GL Source Names.g.xlf file.  I executed the action to create Xliff files.

This action will prompt for a selection of language.  The selection is from the languages included in the NAV DVD.

After selection the system will prompt for a translation file that is exported from FinSql.  This I already showed in a YouTube Video.  If you don’t have a file from FinSql you can just cancel this part.  If you already have an Xliff file for that language then it will be imported into memory as translation data and then removed.

This method is therefore useful if you want to reuse the Xliff file data after an extension update.  All new files will be based on the g.xlf file.

I basically did this action for all 25 languages.  I already had the is-IS and da-DK files, so they where updated.  Since the source language is en-US all my en-XX files where automatically translated.  All the other languages have translation state set to “needs-translation”.

All these files I need to upload to the Translation Service.  From the Lifecycle Services menu select the Translation Service.  This will open the Translation Service Dashboard.

Press + to add a translation request.

I now need to zip and upload the nl-NL file from my Translations folder.

After upload I Submit the translation request

The request will appear on the dashboard with the status; Processing.  Now I need to wait for the status to change to Completed.  Or, create requests for all the other languages and upload files to summit.

When translation has completed I can download the result.

And I have a translation in state “needs-review-translation”.

Now I just need to complete all languages and push changes to GitHub.

Please, if you can, download your language file and look at the results.

Why do we need Interface Codeunits

And what is an interface Codeunit?

A Codeunit that you can execute with CODEUNIT.RUN to perform a given task is, from my point of view, an interface Codeunit.

An interface Codeunit has a parameter that we put in the

This parameter is always a table object.

We have multiple examples of this already in the application.  Codeunits 12 and 80 are some.  There the parameter is a mixed set of data and settings.  Some of the table fields are business data being pushed into the business logic.  Other fields are settings used to control the business logic.

Table 36, Sales Header, is used as the parameter for Codeunit 80.  Fields like No., Bill-to Customer No., Posting Date and so on are business data.  Fields like Ship, Invoice, Print Posted Documents are settings used to control the business logic but have no meaning as business data.

Every table is then a potential parameter for an interface Codeunit.  Our extension can easily create a table that we use as a parameter table.  Record does not need to be inserted into the table to be passed to the Codeunit.

Let’s look at another scenario.  We know that there is an Interface Codeunit  with the name “My Interface Codeunit” but it is belongs to an Extensions that may and may not be installed in the database.

Here we use the virtual table “CodeUnit Metadata” to look for the Interface Codeunit before execution.

This is all simple and strait forward.  Things that we have been doing for a number of years.

Using TempBlob table as a parameter also gives us flexibility to define more complex interface for the Codeunit.  Tempblob table can store complex data in Json or Xml format and pass that to the Codeunit.

Let’s take an example.  We have an extension that extends the discount calculation for Customers and Items.  We would like to ask this extensions for the discount a given customer will have for a given Item.  Questions like that we can represent in a Json file.

And the question can be coded like this.

The Interface Codeunit could be something like

With a Page that contains a single Text variable (Json) we can turn this into a web service.

That we can use from C# with a code like

This is just scratching the surface of what we can do.  To copy a record to and from Json is easy to do with these functions.

And even if I am showing all this in C/AL there should be no problem in using the new AL in Visual Studio Code to get the same results.

Upgrading my G/L Source Names Extension to AL – step 4 addendum

In the last blog post we completed the translation to our native language.  Since then I have learned that I also need to include translation files for EN-US, EN-GP and EN-CA.

With the latest update of AdvaniaGIT tools that was an easy task.  Just asked to create Xlf file for these languages and skipped the part where we import C/AL translation.

I have also been pointed to a new tool that can work with Xlf files.  Multilingual Editor: https://marketplace.visualstudio.com/items?itemName=MultilingualAppToolkit.MultilingualAppToolkitv40

Now I call out to all who are ready to help me with the translation.  Please fork my NAV2018 repository and send me Xlf translation files for your native language.  Or just download one of the translation files and send me your language.

Our next step is to code sign the App file and send it to Microsoft.

Upgrading my G/L Source Names Extension to AL – step 4

We are on a path to upgrade G/L Source Names from version 1 to version 2.  This requires conversion from C/AL to AL, data upgrade and number of changes to the AL code.

A complete check list of what you need to have in your AL extension is published by Microsoft here.

Our task for today is to translate the AL project into our native language.

To make this all as easy as I could I added new functionality to the AdvaniaGIT module and VS Code extension.  Make sure to update to the latest release.

To translate an AL project we need to follow the steps described by Microsoft here.

To demonstrate this process I created a video.

 

Upgrading my G/L Source Names Extension to AL – step 3

When upgrading an extension from C/AL to AL (version 1 to version 2) we need to think about the data upgrade process.

In C/AL we needed to add two function to an extension Codeunit to handle the installation and upgrade.  This I did with Codeunit 70009200.  One function to be execute once for each install.

And another function to be executed once for each company in the install database.

For each database I add my permission sets to the installation users and for each company I restore the setup data for my extension and populate the lookup table for G/L Source Name.

The methods for install and upgrade have changed in AL for extensions version 2.  Look at the AL documentation from Microsoft for details.

In version 2 I remove these two obsolete function from my application management Codeunit and need to add two new Codeunits, one for install and another for upgrade.

In the code you can see that this Codeunit is of Subtype=Install.  This code will  be executed when installing this extension in a database.

To confirm this I can see that I have the G/L Source Names Permission Sets in the Access Control table .

And my G/L Source Name table also has all required entries.

Uninstalling the extension will not remove this data.  Therefore you need to make sure that the install code is structured in a way that it will work even when reinstalling.  Look at the examples from Microsoft to get a better understanding.

Back to my C/AL extension.  When uninstalling that one the data is moved to archive tables.

Archive tables are handled with the NAVAPP.* commands.  The OnNavAppUpgradePerCompany command here on top handled these archive tables when reinstalling or upgrading.

Basically, since I am keeping the same table structure I can use the same set of commands for my upgrade Codeunit.

So, time to test how and if this works.

I have my AL folder open in Visual Studio Code and I use the AdvaniaGIT command Build NAV Environment to get the new Docker container up and running.

Then I use Update launch.json with current branch information to update my launch.json server settings.

I like to use the NAV Container Helper from Microsoft  to manually work with the container.  I use a command from the AdvaniaGIT module to import the NAV Container Module.

The module uses the container name for most of the functions.  The container name can be found by listing the running Docker containers or by asking for the name that match the server used in launch.json.

I need my C/AL extension inside the container so I executed

Then I open PowerShell inside the container

Import the NAV Administration Module

and I am ready to play.  Install the C/AL extension

Now I am faced with the fact that I have opened PowerShell inside the container in my AdvaniaGIT terminal.  That means that my AdvaniaGIT commands will execute inside the container, but not on the host.

The simplest way to solve this is to open another instance of Visual Studio Code.  From there I can start the Web Client and complete the install and configuration of my C/AL extension.

I complete the Assisted Setup and do a round trip to G/L Entries to make sure that I have enough data in my tables to verify that the data upgrade is working.

I can verify this by looking into the SQL tables for my extension.  I use PowerShell to uninstall and unpublish my C/AL extension.

I can verify that in my SQL database I now have four AppData archive tables.

Pressing F5 in Visual Studio Code will now publish and install the AL extension, even if I have the terminal open inside the container.

The extension is published but can’t be installed because I had previously installed an older version of my extension.  Back in my container PowerShell I will follow the steps as described by Microsoft.

My AL extension is published and I have verified in my SQL server that all the data from the C/AL extension has been moved to the AL extension tables and all the archive tables have been removed.

Back in Visual Studio Code I can now use F5 to publish and install the extension again if I need to update, debug and test my extension.

Couple of more steps left that I will do shortly.  Happy coding…

 

Upgrading my G/L Source Names Extension to AL – step 2

So, where is step 1?  Step 1 was converting C/AL code to AL code.  This we did with AdvaniaGIT and was demonstrated here.

First thing first!  I received the following email from Microsoft.

Hello,

The decision has been made by our SLT, that the use of a Prefix or Suffix is now a mandatory requirement. If you are already using this in your app(s), great. If not, you will want to do so.

We are coming across too many collisions between apps in our internal tests during builds and have seen some in live tenants as well. It makes the most sense to make this a requirement now. If you think about it in a live situation, if a customer installs an app before yours and then tries yours but gets collision issues, they may just decide to not even continue on with yours.

Also, I have been made aware that adding a prefix or suffix after you already have a v2 app published can make the process complicated for you. Therefore, since you all have to convert to v2 anyway, now is a good time to add in the prefix/suffix.

The following link provides the guidelines around using it here

If you haven’t reserved your prefix yet, please email me back to reserve one (or more if needed).

Thank you,

Ryan

Since my brand is Objects4NAV.com I asked for 04N as my prefix and got it registered.  Since we got this information from Microsoft, every object that we develop in NAV 2018 now has our companies prefix in the name.

Starting my AL development by opening Visual Studio Code in my repository folder.  I updated my setup.json to match the latest preview build as Docker container and then selected to Build NAV Environment using AdvaniaGIT.

After download and deployment of the container I noticed that the container had a brand new version of the AL Extension for Visual Studio Code.  I looked at the version installed and that was an older version.

I uninstalled the AL Language extension and restarted Visual Studio Code.

As you can see on the screenshot above we now don’t have any AL Language extension installed.  I executed the Build NAV Environment command from AdvanaiGIT to install the extension on the Docker container.  In this case I already had a container assigned to my branch so only three things happened.

  • uidOffset in the container database was updated.  This is recommended for C/AL development.
  • License file is updated in the container database and container service.  The license used is the one configured in branch setup.json or the machine settings GITSettings.json
  • AL Language Extension is copied from the container to the host and installed in Visual Studio Code.

Again, restarting Visual Studio Code to find that the latest version of AL Language Extension has been installed.

I then executed two AdvaniaGIT actions.

  • Update Launch.json with current branch environment.  This will update the host name and the service name in my AL Launch.json file to make sure that my AL project will be interacting with the branch container.
  • Open Visual Studio Code in AL folder.  This will open another instance of Visual Studio Code in the AL folder.

Immediately after Visual Studio Code was opened it asked for symbols and I agreed that we should download them from the container.

Everything is now ready for AL development using the latest build that Microsoft has to offer.

I started  Edit – Replace in Files in Visual Studio Code.  All my objects have a name that start with G/L Source Name.  I used this knowledge to apply the prefix.

By starting with the double quote I make sure to only update the object names and not captions.  All captions start with a single quote.  I got a list of all changes and needed to confirm all changes.

The field name I add to G/L Entry table does not match this rule so I needed to rename that my self.  Selecting the field name and pressing F2 allows me to rename a field and have Visual Studio Code update all references automatically.

Pressing F5 started my build, publish and debug.

My extension is installed and ready for testing.

There are a few more steps that I need to look into before publishing the new version of G/L Source Names to Dynamics 365.  These steps will appear here in the coming days.  Hope this will be useful to you all.

Don’t worry about DotNet version in C/AL

When using DotNet data type in NAV C/AL we normally lookup a sub type to use.  When we do the result can be something like

Then, what will happen when moving this code from NAV 2016 to NAV 2017 and NAV 2018.  The Newtonsoft.Json version is not the same and we will get a compile error!

Just remove the version information from the sub type information.

And NAV will find the matching Newtonsoft.Json library you have installed and use it.

This should work for all our DotNet variables.

AzureSQL database gives me change tracking error

I just uploaded a SQL bacpac to AzureSQL.  This I have done a number of times.  Connected my service to the SQL database and tried to start the service.

This time I got an error.  Looking in Event Viewer I can see.

I looked into the SQL database, and sure enough there was a line in sys.change_tracking_databases table.  The problem was that in that table the [database_id] was equal to 48 while

resulted in 49.  Hence the error and my service tier failing to start.

To remove the change tracking from the database I executed (found here)

The service tier will take care of turning the change tracking on again when it starts.  You might need to repeat these steps if restarting the service tier.

According to Microsoft a fix is in the pipeline and likely ships in CU2.