Stephen Smith's Blog

Musings on Machine Learning…

Archive for the ‘Software Development Lifecycle’ Category

Out-of-Order Instructions

leave a comment »

Introduction

We think of computer processors executing a set of instructions one at a time in sequential order. As programmers this is exactly what we expect the computer to do and if the computer decided to execute our carefully written code in a different order then this terrifies us. We would expect our program to fail, producing wrong results or crashing. However we see manufacturers claiming their processors execute instructions out-of-order and that this is a feature that improves performance. In this article, we’ll look at what is really going on here and how it can benefit us, without causing too much fear.

Disclaimer

ARM defines the Instruction Set Architecture (ISA), which defines the Assembly Language instruction set. ARM provides some reference implementations, but individual manufacturers can take these, customize these or develop their own independent implementation of the ARM instruction set. As a result the internal workings of ARM processors differs from manufacturer to manufacturer. A main point of difference is in performance optimizations. Apple is very aggressive in this regard, which is why the ARM processors in iPads and iPhones beat the competition. This means the level of out-of-order execution differs from manufacturer to manufacturer, further this is much more prevalent in newer ARM chips. As a result, the examples in this article will apply to a selection of ARM chips but not all.

A Couple of Simple Cases

Consider the following small bit of code to multiply two numbers then load another number from memory and add it to the result of the multiplication:

MUL R3, R4, R5 @ R3 = R4 * R5
LDR R6, [R7]   @ Load R6 with the memory pointed to by R7
ADD R3, R6     @ R3 = R3 + R6

The ARM Processor is a RISC processor and its goal is to execute each instruction in 1 clock cycle. However multiplication is an exception and takes several clock cycles longer due to the loop of shifting and adding it has to perform internally. The load instruction doesn’t rely on the result of the multiplication and doesn’t involve the arithmetic unit. Thus it’s fairly simple for the ARM Processor to see this and execute the load while the multiply is still churning away. If the memory location is in cache, chances are the LDR will complete before the MUL and hence we say the instructions executed out-of-order. The ADD instruction then needs the results from both the MUL and LDR instruction, so it needs to wait for both of these to complete before executing it’s addition.

Consider another example of three LDR instructions:

LDR R1, [R4] @ memory in swap file
LDR R2, [R5] @ memory not in cache
LDR R3, [R6] @ memory in cache

Here the memory being loaded by the first instruction, has been swapped out of memory to secondary storage, so loading it is going to be slow. The second memory location is in regular memory. DDR4 memory, like that used in the new Raspberry Pi 4, is pretty fast, but not as fast as the CPU and it is also loading instructions to process, hence this second LDR might take a couple of cycles to execute. It makes a request to the memory controller and its request is queued with everything else going on. The third instruction, assumes the memory is in the CPU cache and hence processed immediately, so this instruction really does take only 1 clock cycle.

The upshot is that these three LDR instructions could well complete in reverse order.

Newer ARM processors can look ahead through the instructions looking for independent instructions to execute, the size of this pool will determine how out-of-order things can get. The important point is that instructions that have dependencies can’t start and that to the programmer, it looks like his code is executing in order and that all this magic is transparent to the correct execution of the program.

Since the CPU is executing all these instructions at once, you might wonder what the value of the program counter register (PC) is? This register has a very precisely defined value, since it is used for PC relative addressing. So the PC can’t be affected by out-of-order execution. 

Coprocessors

All newer ARM processors include floating-point coprocessors and NEON vector coprocessors. The instructions that execute on these usually take a few instructions cycles to execute. If the instructions that follow a coprocessor instruction are regular ARM instructions and don’t rely on the results of coprocessor operations, then they can continue to execute in parallel to the coprocessor. This is a handy way to get more code parallelism going, keeping all aspects of the CPU busy. Intermixing coprocessor and regular instructions is another great way to leverage out-of-order instructions to get better performance.

Compilers and Code Generation

This indicates that if a compiler code generator or an Assembly Language program rearranges some of their instructions, they can get more things happening at once in parallel giving the program better performance. ARM Holdings contributes to the GNU Compiler Collection (GCC) to fully utilize the optimization present in their reference implementations. In the ARM specific options for GCC, you can select the ARM processor version that matches your target and get more advanced optimizations. Since Apple creates their own development tools under XCode, they can add optimizations specific to their custom ARM implementations.

As Assembly Language programmers, if we want to get the absolute best performance we might consider re-arranging some of our instructions so that instructions that are independent of each other are in a row and hopefully can be executed in parallel. This can require quite a bit of testing to reverse engineer the exact out-of-order instruction capability of your particular target ARM processor model. As always with performance optimizations, you must test the performance to prove you are improving things, and not just making your code more cryptic.

Interrupts

This all sounds great, but what happens when an interrupt happens? This could be a timer interrupt to say your time-slice is up and another process gets to use the ARM Core, or it could be that more data needs to be read from the Wifi or a USB device.

Here the ARM CPU designer has a choice, they can forget about the work-in-progress and handle the interrupt quickly, or they can wait a couple of cycles to let work-in-progress complete and then handle the interrupt. Either way they have to allow the interrupt handler to save the current context and then restore the context to continue execution. Typically interrupt handlers do this by saving all the CPU and coprocessor registers to the system stack, doing their work and then restoring state.

When you see an ARM processor advertised as designed for real-time or industrial use, this typically means that it handles interrupts quickly with minimal delay. In this case, the work-in-progress is discarded and will be redone after the interrupt is finished. For ARM processors designed for general purpose computing, this usually means that user performance is more important than being super responsive to interrupts and hence they can let some of the work-in-progress complete before servicing the interrupt. For general purpose computing this is ok, since the attached devices like USB, ethernet and such have buffers that can hold enough contents to wait for the CPU to get around to them.

A Step Too Far and Spectre

Hardware designers went even further with branch prediction, where if a conditional branch instruction needs to wait for a condition code to be set, they don’t wait but keep going assuming one branch direction (perhaps based on the result from the last time this code executed) and keep going. The problem here is that at this point, the CPU has to save the current state, incase it needs to go back when it guesses wrong. This CPU state was saved in a CPU cache that was only used for this, but had no security protection, resulting in the Spectre attack that figured out a way to get at this data. This caused data leakage across processes or even across virtual machines. The whole spectre debacle showed that great care has to be taken with these sorts of optimizations.

Heat, the Ultimate Gotcha

Suppose your your ARM processor has four CPU cores and you write a brilliant Assembly language program that deploys to use all four cores and fully exploits out-of-order execution. Your program is now using every bit of the ARM CPU, each core is intermixing regular ARM, floating point and NEON instructions You have intermixed your ARM instructions to get the arithmetic unit operating in parallel to the memory unit. This will be the fastest implementation yet. Then you run your program, it gets off to a great start, but then suddenly slows down to a crawl. What happened?

The enemy of parallel processing on a single chip is heat. Everything the CPU does generates a little heat. The more things you get going at once the more heat will be generated by the CPU. Most ARM based computers like the Raspberry Pi assume you won’t be running the CPU so hard, and only provide heat dissipation for a more standard load. This is why Raspberry Pis usually do so badly playing high-res videos. They can do it, as long as they don’t overheat, which typically doesn’t take long.

This leaves you a real engineering problem. You need to either add more cooling to your target device, or you have to deliberately reduce the CPU usage of your program, where perhaps paradoxically you get more work done using two cores rather than four, because you won’t be throttled due to overheating.

Summary

This was a quick overview of out-of-order instructions. Hopefully you don’t find these scary and keep in mind the potential benefits as you write your code. As newer ARM processors come to market, we’ll be seeing larger and larger pools of instructions executed in parallel, where the ability for instructions to execute out-of-order will have even greater benefits.

If you are interested in machine code or Assembly Language programming, be sure to check out my book: “Raspberry Pi Assembly Language Programming” from Apress. It is available on all major booksellers or directly from Apress here.

Written by smist08

November 15, 2019 at 11:11 am

Raspberry Pi Assembly Language Programming

leave a comment »

 

Introduction

My new book “Raspberry Pi Assembly Language Programming” has just been published by Apress. This is my first book to be published by a real publisher and I’m thrilled to see it appearing on websites of booksellers all over the Internet. In this blog post I’ll talk about how this book came to exist, the process of writing and publishing it and a bit about the book itself.

For anyone interested in this book, here are a few places where it is available:

Most of these sites let you see a preview and the table of contents.

How this Book Came About

I purchased my Raspberry Pi 3+ in late 2017 and had a great deal of fun playing with it. I wrote quite a few blog posts on the Pi, a directory of these is available here. The Raspberry Pi package I purchased included a breadboard and a selection of electronic components. I put together a set of LEDs connected to the Pi’s GPIO ports. I then wrote a series of articles on making these LEDs flash using various programming languages including C, Python, Scratch, Fortran, and Erlang. In early 2018 I was interested in learning more about how the Pi’s ARM processor works and delved into Assembly language programming. This resulted in two blog posts, an introduction and then my flashing LED program ported to ARM Assembly Language.

Earlier this year I was contacted by an Apress Talent Acquisition agent who had seen my blog articles on ARM Assembly Language and wanted to know if I wanted to develop them into a book. I thought about it over the weekend and was intrigued. The material I found when writing the blog articles wasn’t great, and I felt I could do better. I replied to the agent and we had a call to discuss the book. He had me write up a proposal and possible table of contents. I did this, Apress accepted it and sent me a contract to sign.

The Process

Apress provided a Word style sheet and a written style guide. My writing process has been to write in Google Docs and then have my spouse, a professional editor, edit it. The collaboration of Google Docs is just too good to do away with. So I wrote the chapters in Google Docs, got them edited and then transferred them to MS Word and applied the Apress style sheet.

I worked with a coordinating editor at Apress who was very energetic in getting all the pieces done. She found a technical editor who would provide a technical review of each chapter as I wrote it. He was located in the UK, so often I would submit a chapter and see it edited overnight.

Once I had submitted all the chapters then a senior development editor gave the whole book a review. At that point I thought I was done, but then the book was given to Springer’s (Apress’s parent company) production department who did another editing pass. I was surprised that the production department still found quite a few things that needed fixing or improving.

After all that the book appeared fairly quickly. I like the cover, they used my photo of my breadboard with the flashing LEDs. As of today, the book is available at most booksellers, some with stock and some on preorder. I signed the contract in June and did the bulk of the writing in July and August. Overall, I’m pretty happy with the process and how things turned out.

The Book

My philosophy was to introduce complete working programs from Chapter 1 with the traditional “Hello World” program. I only covered topics where you could write the code with the tools included with the Raspberry Pi and run them. I lay the foundations for how to write larger Assembly programs, with how to code the various structured programming constructs, but also include a chapter on how to interoperate with C and Python code.

Raspbian is a 32-bit operating system as older Raspberry Pi’s and the Raspberry Pi Zero can only run 32-bit code. I didn’t want to leave out 64-bit code, as there are 64-bit versions of Linux from other distributions like Ubuntu that are available for the Pi. So I included a chapter on ARM 64-bit Assembly along with guidelines on how to port your 32-bit code to 64-bit. I then included 64-bit versions of several of the programs we had developed along the way.

There is a lot of interest in ARM Assembly Language, especially from hackers, as all phones, tablets and even a few laptops are running ARM processors now. I included a number of hacking related topics like how to reverse engineer code, as security professionals are very interested in this as they work to protect the mobile devices utilized by their organizations.

The ARM Processor is a good example of a RISC processor, so if you are interested in RISC, this book will give a good introduction to the concepts, like how to do everything with instructions that are only 32-bits in length. Once you understand ARM Assembly, picking up the Assembly language of another RISC processor like the Risc-V becomes much easier.

The book also covers how to program the floating point processor included with most ARMs along with the NEON vector processor that is available on newer Raspberry Pis.

Summary

If you are interested in learning Assembly Language, please check out my book. The Raspberry Pi provides a great platform to do this. Even if you only program in higher level languages, knowing Assembly Language will help you understand what is going on at a deeper level. How modern processors design their Assembly Language to maximize program performance and minimize memory usage is quite fascinating and I hope you find the topic as interesting as I do.

 

Written by smist08

November 1, 2019 at 11:22 am

Getting Productive with Julia

with 3 comments

Introduction

Julia is a programming language that is used quite extensively by the scientific community. It is open source, it just reached its version 1.0 milestone after quite a few years of development and it is nearly as fast as C but with many features associated with interpretive languages like R or Python.

There don’t seem to be many articles on getting all up and running with Julia, so I thought I’d write about some things that I found useful. This is all based on playing with Julia on my laptop running Ubuntu Linux.

Run in the Cloud

One option is to avoid any installation hassles is to just run in the cloud. You can do this with JuliaBox. JuliaBox gives you a Jupyter Notebook interface where you can either play with the various tutorials or do your own programming. Just beware the resources you get for free are quite limited and JuliaBox makes its money by charging you for additional time and computing power.

Sadly at this point, there aren’t very many options for running Julia in the cloud since the big AI clouds seem to only offer Python and R. I’m hoping that Google’s Kaggle will add it as an option, since the better performance will open up some intriguing possibilities in their competitions.

JuliaBox gives you easy direct access to all the tutorials offered from Julia’s learning site. Running through the YouTube videos and playing with these notebooks is a great way to get up to speed with Julia.

Installing Julia

Julia’s website documents how to install Julia onto various operating systems. Generally the Julia installation is just copying files to the right places and adding the Julia executable to the PATH. On Ubuntu you can search for Julia in the Ubuntu Software App and install it from there. Either way this is usually pretty easy straight forward. This gives you the ability to run Julia programs by typing “julia sourefile.jl” at a command prompt. If you just type “julia” you get the REPL environment for entering commands.

You can do quite a lot in REPL, but I don’t find it very useful myself except for doing things like package management.

If you like programming by coding in your favorite text editor and then just running the program, then this is all you need. For many purposes this works out great.

The Juno IDE

If you enjoy working in full IDE’s then there is Juno which is based on the open source Atom IDE. There are commercial variants of this IDE with full support, but I find the free version works just fine.

To install Juno you follow these instructions. Basically this involves installing the Atom IDE by downloading and running a .deb installation package. Then from within Atom, adding Julia support to the IDE.

Atom has integration with Julia’s debugger Gallium as well as provides a plot plane and access to watch variables. The editor is fairly good with syntax highlighting. Generally not a bad way to develop in Julia.

Jupyter

JuliaBox mentioned above uses Jupyter and runs it in the cloud. However, you can install it locally as well. Jupyter is a very popular and powerful notebook metaphor for developing programs where you see  the results of each incremental bit of code as you write it. It is really good at displaying all sorts of fancy formats like graphs. It is web based and will run a local web server that will use the local Julia installation to run. If you develop in Python or R, then you’ve probably already played with Jupyter.

To install it you first have to install Jupyter. The best way to do this is to use “sudo apt install jupyter”. This will install the full Jupyter environment with full Python support. To add Julia Jupyter support, you need to run Julia another way (like just entering julia to get the REPL) and type “Pkg.add(“IJulia”)”. Now next time you start Jupyter (usually by typing “jupyter notebook”), you can create a new notebook based on Julia rather than Python.

Julia Packages

Once you have the core Julia programming environment up and running, you will probably want to install a number of add-on packages. The package manager is call Pkg and you need to type “using Pkg” before using it. These are all installed by the Pkg.add(“”) command. You only need to add a package once. You will probably want to run “Pkg.update()” now and again to see if the packages you are using have been updated.

There are currently about 1900 registered Julia packages. Not all of them have been updated to Julia version 1.0 yet, so check the compatibility first. There are a lot of useful packages for things like machine learning, scientific calculations, data frames, plotting, etc. Certainly have a look at the package library before embarking on writing something from scratch.

Summary

These are currently the main ways to play with Julia. I’m sure since Julia is a very open community driven system, that these will proliferate. I don’t miss using the giant IDEs Visual Studio or Eclipse, these have become far too heavy and slow in my opinion. I find I evenly distribute my time between using Jupyter, Juno and just edit/run. Compared to Python it may appear their aren’t nearly as many tools for Julia, but with the current set, I don’t feel deprived.

 

Written by smist08

October 10, 2018 at 3:55 am

Avoiding Airline Collisions with Julia

leave a comment »

Introduction

I was just watching an old episode of “Mayday: Air Crash Investigations“, on the crash of a Russian passenger jet with a DHL cargo plane over Switzerland. In this episode, both planes had onboard collision avoidance systems, but one plane listened to air traffic control rather than the collision avoidance system and went down rather than up, resulting in the collision. In reading about the programming language Julia recently, I had noticed several presentations on the development of the next generation of collision avoidance systems, in Julia. This piqued my interest, along with the fact that my wife is currently getting her pilot’s license, to have a slightly deeper look into this.

Modern airliners have employed an onboard Traffic Collision Avoidance Systems (TCAS) since the 1980s. TCAS is required on any passenger airplane that takes more than 19 passengers. These systems work by monitoring the transponders of nearby aircraft and determining when a collision is imminent. At this point it provides a warning to the plane’s pilot along with a course of action. The TCAS systems on the two aircraft communicate so one plane is ordered to go up and the other to descend.

Generally there are three layers to collision avoidance that operate on different timescales. At the coarsest level planes travelling in one direction are required to be at a different altitude than planes in the reversion direction. Usually one direction gets even altitudes like 30,000 feet and the reverse gets odd altitude like 31,000 feet. At a finer level, air traffic control is responsible for keeping the planes apart at medium distances. Then close up (minutes apart) it is TCAS’s job to avoid the collisions. This is partly due to the aftermath of the Russian/DHL crash and partly due to a realization that the latency in communications with air traffic control is too great when things get too close for comfort.

Interestingly it was the collision of two passenger plane’s over the Grand Canyon in 1956 that caused congress to create the FAA and started the development of the current TCAS system. It took thirty years to develop and deploy since it required computers to get much smaller and faster first.

Why Julia

The FAA has funded the development of the next generation of traffic avoidance which has been dubbed ACAS X. This started in 2008 and after quite a bit of study, it was decided to use Julia extensively in its development. Reading the reasons for why Julia was selected is rather scary when you consider what it highlights about the current TCAS system.

Problem 1 – Specifications

A big problem with TCAS was that the people that defined the system wrote the specification first as English like pseudo-code and then re-wrote that as a more programmy pseudo-code with variables and such. Then others would take this code and implement it in Mathlab to test the algorithms. Then the people who actually made the hardware would take this and re-implement it in C++ or Assembler. When people had a recent look at all this code, they found it to be a big mess, where the different specs and code bases had been maintained separately and didn’t match. There was no automation and very little validation. The first idea of fixing this code base was rejected as completely unreliable and impossible to add new features to.

They wanted to the new system to take advantage of modern technologies like satellite navigation systems, GPS, and on-board radar systems. This means the new system will work with other planes that don’t have collision avoidance or perhaps don’t even have a transponder. In fact they wanted the new system to be easily extensible as new sensor inputs are added. Below is a small example of the reams of pseudo code that makes up TCAS.

The hope with Julia is to unify these different code bases into one. The variable pseudo-code would actually be true Julia code and the English code would be incorporated into JavaDoc like comments in the code (actually using Latex). This would then eliminate the need to use Mathlab to test the pseudo-code. The consensus is that Julia code is easily as readable as the above pseudo-code but with the advantage of being runnable and testable.

The FAA doesn’t have the authority to mandate Avionics hardware companies run Julia on their ACAS X systems, but the hope is that the performance of Julia is good enough that they won’t bother reimplementing the system in C++ and that everything will be the same Julia code. Current estimates have the Julia code running 1.5 times the speed of C code and the thought is that with newer computer chips, this should be sufficient. The hope then is that the new system will not have the translation errors that dog TCAS.

Now that the specification is true computer code many other tools can be written or used to help check correctness, such as the tool below which generates a flowchart from the Julia code/specification.

Problem 2 – Testing/Validation

Certainly with TCAS implementing the system in Mathlab was hard. But then Mathlab is quite slow and that greatly restricts the number of test cases that can be effectively be automated. The TCAS system is based on a huge number of giant decision trees and billions of test cases. A number of test/validation frameworks have been developed to test the new ACAS X system including using theorem proving, probabilistic model checking, adaptive stress testing, simulations and weakest precondition code analysis.

Now if the Avionics hardware manufacturers run the actual Julia code, there will have only been one code base from specification to deployment and it will have all been very thoroughly developed, tested and validated.

Summary

The new ACAS X system is currently being flight tested and is projected to start being deployed in regular commercial aircraft starting in 2020. Looking at the work that has gone into this system, it looks like it will make flying much safer. Hopefully it also sets the stage for how future large safety-critical systems will be developed. Further it looks like the Julia programming language will play a central part in this.

Written by smist08

October 7, 2018 at 10:28 pm

Performance Testing in Swift

with one comment

Introduction

A couple of blog posts ago I covered writing my first Swift program for iOS so that I could draw a Koch Snowflake on an iPad or an iPhone. Then last time I covered adding unit tests to that project. This time I’m going to add performance tests.

In the process of adding performance tests, I had to refactor the test project and we’ll also look at why that was and how it makes it better going forwards as more tests are added. I’ll also mention a few things that should be done if this project gets a bit bigger.

I put an updated version of the Koch Snowflake project on Google Drive here.

Performance Tests in XCode

Of course you could instrument your program yourself and perhaps write the performance results out to a file or something, for that matter you can drill down into the Swift test case class and have a look at their implementation. But XCode gives you a bit of support so you generally don’t need to. If you add self.measureBlock {} around code in a unit test then the time taken of the code inside the measureBlock will be recorded and reported inside XCode as shown in the following screenshot.

Screen Shot 2016-06-01 at 3.32.53 PM

Actually it does a bit more than that. When you add measureBlock to a unit test, then when you run that unit test, it won’t just be run once, but will be run ten times, so that the average and standard deviation will be recorded. Due to this it is crucial that any performance tests are idempotent. You can also set a baseline, so the percentage deviation from the baseline gets recorded. This is shown in the following screenshot that is a drill down from the previous screen shot.

Screen Shot 2016-06-01 at 3.33.04 PM

Hence XCode gives a fairly painless way to add some performance metrics to your unit tests.

Test Case Organization

Generally, you want your unit tests to run against every build or your product, so you want then to run in a second or two. Once the performance tests get longer, you will probably want to separate them off into a separate test group and then run this test group perhaps once over night. I haven’t done that, but if the project gets any bigger then I will.

In fact, the test framework inside XCode is quite good for performing integration tests (which would run against real databases and real servers), but since these may require some setup or be quite time consuming, you could also set these to run once per night.

There is also a separate framework for UI testing, which again is too slow to run against every build, but makes sense to run every night.

Refactoring the Unit Tests

For the performance test, I wanted to record the time it takes to draw the Koch Snowflake at various fractal levels. To do this I wanted to do something like the previous testInitialViewController routine, but it contained a lot of setup code. So first the unit test framework includes setUp function that is called before the unit tests are run and a tearDown routine that is called after then finished. So I moved the creating of the graphic context to this routine, along with the code to get the view controller started. Then it was fairly easy to add tests for fractal levels 3 through 7.

Last time I just had 2 unit tests, each was quite large and performed multiple things. Now we’ve split things up into more unit tests that do less, which is generally a better practice. This was actually forced on me since you can only have one measureBlock in any unit test, so I couldn’t performance test the different fractal levels in the same unit test (at least with separate timings). Really I should break up the turtle graphics tests into multiple unit tests, perhaps next time.

The reason I went all the way to fractal level 7, was that the performance reports in XCode are often 2 decimal places (or sometimes 3 decimal places) on the number of seconds the test takes. For my fractal, the drawing is quite quick so I needed go this high to get some longer test times recorded (kind of a good problem to have). I could have gone higher or put them in an additional loop, but thought this was sufficient.

//
//  KochSnowFlakeTests.swift
//  KochSnowFlakeTests
//
//  Created by Stephen Smith on 2016-05-13.
//  Copyright © 2016 Stephen Smith. All rights reserved.
//

import XCTest

@testable import KochSnowFlake

class KochSnowFlakeTests: XCTestCase {
    var storyboard:UIStoryboard!
    var viewController:ViewController!

    override func setUp() {
        super.setUp()
        // Put setup code here. This method is called before the invocation of each test method in the class.

        UIGraphicsBeginImageContextWithOptions(CGSize(width: 50, height: 50), false, 20);

        self.storyboard = UIStoryboard(name: "Main", bundle: nil)

        self.viewController = storyboard.instantiateInitialViewController() as! ViewController
        _ = viewController.view
        viewController.viewDidLoad()
    }

    override func tearDown() {
        // Put teardown code here. This method is called after the invocation of each test method in the class.
        UIGraphicsEndImageContext();
        super.tearDown()
    }

    func testTurtleGraphics() {
        // Test the turtle graphics library.
        // Note we need a valid graphics context to do this.

        let context = UIGraphicsGetCurrentContext();
        let tg = TurtleGraphics(inContext: context!);
        XCTAssert(tg.x == 50, "Initial X value should be 50");
        XCTAssertEqual(tg.y, 150, "Initial Y value should be 150");
        XCTAssertEqual(tg.angle, 0, "Initial angle should be 0");
        tg.move(10);
        XCTAssertEqual(tg.x, 60, "X should be incremented to 60");
        XCTAssertEqual(tg.y, 150, "Initial Y value should be 150");
        XCTAssertEqual(tg.angle, 0, "Initial angle should be 0");
        tg.turn(90);
        tg.move(10);
        XCTAssertEqualWithAccuracy(tg.x, 60, accuracy: 0.0001, "X should be o 60");
        XCTAssertEqualWithAccuracy(tg.y, 160, accuracy: 0.0001, "Initial Y value should be 160");
        XCTAssertEqual(tg.angle, 90, "Initial angle should be 90");
        tg.turn(-45);
        tg.move(10);
        XCTAssertEqualWithAccuracy(tg.x, 60 + 10 * sqrt(2) / 2, accuracy: 0.0001, "X should be o 60+10*sqrt(2)/2");
        XCTAssertEqualWithAccuracy(tg.y, 160 + 10 * sqrt(2) / 2, accuracy: 0.0001, "Initial Y value should be 160+10*sqrt(2)/2");
        XCTAssertEqual(tg.angle, 45, "Initial angle should be 45");
    }

    func testPerformanceLevel3()
    {
        // Test that the storyboard is connected to the view controller and
        // that we can create and use the view and controls.

        viewController.fractalLevelTextField.text = "3"
        self.measureBlock {
            self.viewController.textChangeNot("dummy")
            self.viewController.fracView.drawRect(CGRect(x:0, y:0, width: 50, height: 50))
        }

        XCTAssertTrue(viewController.fracView.level == 3)
        // This next line is just to get 100% code coverage.
        viewController.didReceiveMemoryWarning()
    }

    func testPerformanceLevel4() {
        // This is an example of a performance test case.
        viewController.fractalLevelTextField.text = "4"
        self.measureBlock {
            self.viewController.textChangeNot("dummy")
            self.viewController.fracView.drawRect(CGRect(x:0, y:0, width: 50, height: 50))
        }
    }

    func testPerformanceLevel5() {
        // This is an example of a performance test case.
        viewController.fractalLevelTextField.text = "5"
        self.measureBlock {
            self.viewController.textChangeNot("dummy")
            self.viewController.fracView.drawRect(CGRect(x:0, y:0, width: 50, height: 50))
        }
    }

    func testPerformanceLevel6() {
        // This is an example of a performance test case.
        viewController.fractalLevelTextField.text = "6"
        self.measureBlock {
            self.viewController.textChangeNot("dummy")
            self.viewController.fracView.drawRect(CGRect(x:0, y:0, width: 50, height: 50))
        }
    }

    func testPerformanceLevel7() {
        // This is an example of a performance test case.
        viewController.fractalLevelTextField.text = "7"
        self.measureBlock {
            self.viewController.textChangeNot("dummy")
            self.viewController.fracView.drawRect(CGRect(x:0, y:0, width: 50, height: 50))
        }
    }
}

 

Summary

I found adding performance tests to my fractal iOS application quite easy. XCode gives quite nice support to perform these tests painlessly, hopefully motivating more programmers to include them.

At this point I’m not going to optimize the code as it is running fast enough. But if I ever take on drawing more sophisticated or complicated fractals, then drawing speed becomes really important. Some things to consider would be how efficient in the recursive algorithm used, and whether I’m efficiently using floating point and integer arithmetic (or are there unnecessary conversions or perhaps too much precision being used).

Written by smist08

June 7, 2016 at 2:15 am

Adding Unit Tests to My Swift Application

with 3 comments

Introduction

Last time I blogged on creating my first Swift application that would display a Koch Snowflake fractal on an iPad or and iPhone. This time I’m going to look at adding some unit tests to that program to see how that works in XCode and Swift. It appears that Apple has built some nice unit testing support into XCode so doing this was actually quite easy.

I put the Koch Snowflake Swift project on Google Drive here. Feel free to download it and play with it. Refer to this project or the previous blog post for the code we are actually testing in this article.

Test Driven Development

If I was following proper Test Driven Development (TDD) then I should have written the unit tests before I wrote the actual code. This would force me to think about the design of the APIs and the testability of the program first, and then would force quick cycles of writing tests, writing code so the tests pass and then refactoring the code to make it all better. However, in this case I ported an old Objective-C program to Swift to play with Swift and now that I’m getting more familiar with Swift, I’m finally looking at the unit testing framework. So what I’m doing is perhaps not a good practice, but adding the unit tests later is better than not adding them at all.

The danger I could have run into is that the program could have turned out to be quite hard to test due to perhaps some bad design decisions and then required quite a bit of refactoring to make it properly unit testable. Fortunately, in this case the program is fairly simple and adding a good set of unit tests was fairly straight forward and didn’t require any changes to the program.

One criticism of TDD is that there is a perception that it interferes with good program design and architecture. This isn’t true. The misconception is that the first tests need to be truly useful, whereas the first tests usually just call the API and set how the API to the various classes is used. The classes are just skeletons and the unit tests pass as long as everything compiles. Getting everything to compile is the first step and thinking about all the APIs before writing the real code that does stuff is the first step to getting a good modular design.

Apple has adopted TDD for a lot of their own projects, for instance their Core Data module was developed this way. Since Apple developers use XCode and now many are using TDD this means that a lot of good support for TDD is being baked into XCode. It also means that newer Apple technologies are much easier to use in a TDD environment than some of their older ones. But even for the older ones there are lots of clever examples on the web of how to work around the various challenges they introduce.

Screen Shot 2016-05-31 at 10.41.21 AM

Testing Turtle Graphics

When you create a project with XCode you have the option to include unit tests (which I did) and this gives you a test project that you can run with some dummy tests in it. So I figured first let’s add some unit tests to my TurtleGraphics class. This is pretty simple you move the turtle and turn the turtle and it leaves a trail behind it. For drawing fractals turtle graphics makes the algorithms very easy since it fits in well with the recursive algorithms typically used.

The turtle graphics class uses Apple’s Quartz2D library to draw on the screen. Perhaps the first possible design flaw is that when I instantiate the turtle graphics library I need to pass its constructor a valid graphics context. The underlying Quartz2D library routines certainly don’t like this being nil and throw an exception if you try that (ie my first try). So I thought I would just create a graphic context object but after battling with Swift a bit, I found I couldn’t do this because the underlying class doesn’t have any constructors. This also prevented me from mocking the class (at least using the real class and replacing just a few methods). This is because Apple wants to make sure you get a valid graphics context from one of their factory methods. If you are actually in the drawing routine this is passed to you by the system, but as a unit test, we aren’t running in a screen drawing context. Fortunately there is a way to get a graphics context to draw on an in-memory image file, so I used that.

To solve this, I could have refactored the drawing routines into another class and had this class accept nil for the graphics context and just return when this happens. This is basically just making a new class that pass through to the real class when the program runs, or acts as a stub class when running unit tests. I didn’t like this so much since then if the real program did pass nil, it wouldn’t get caught and the program just wouldn’t work properly. Even though I’m not testing the actual placement of pixels on the screen, it makes me feel good to know the actual real drawing routines are being exercised during the unit tests, so if they do get an invalid argument or something the problem will be caught.

The unit test framework lets unit tests get at variables in a class to help you set up the asserts to prove correctness (you import the module via @testable import KochSnowFlake rather than just import), so to test it, I just did some simple move and turns and then tested that the internal state of the turtle graphics module was correct. Notice when I turned 45 degrees I used trigonometry to calculate the new position, for testing its usually good to test against a calculation, rather than running the program to see what the value is and then using that. Notice the use of XCTAssertEqualWithAccuracy, since we are using floating point arithmetic, rounding errors creep in, so the equals will never be exact. You can configure XCode to either stop on each error or run to completion just recording all the errors, its up to you. For this I usually just stopped on an error and then fixed the problem.

func testTurtleGraphics() {
 // Test the turtle graphics library.
 // Note we need a valid graphics context to do this.
 
 UIGraphicsBeginImageContextWithOptions(CGSize(width: 50, height: 50), false, 20);
 let context = UIGraphicsGetCurrentContext();
 let tg = TurtleGraphics(inContext: context!);
 XCTAssert(tg.x == 50, "Initial X value should be 50");
 XCTAssertEqual(tg.y, 150, "Initial Y value should be 150");
 XCTAssertEqual(tg.angle, 0, "Initial angle should be 0");
 tg.move(10);
 XCTAssertEqual(tg.x, 60, "X should be incremented to 60");
 XCTAssertEqual(tg.y, 150, "Initial Y value should be 150");
 XCTAssertEqual(tg.angle, 0, "Initial angle should be 0");
 tg.turn(90);
 tg.move(10);
 XCTAssertEqualWithAccuracy(tg.x, 60, accuracy: 0.0001, "X should be o 60");
 XCTAssertEqualWithAccuracy(tg.y, 160, accuracy: 0.0001, "Initial Y value should be 160");
 XCTAssertEqual(tg.angle, 90, "Initial angle should be 90");
 tg.turn(-45);
 tg.move(10);
 XCTAssertEqualWithAccuracy(tg.x, 60 + 10 * sqrt(2) / 2, accuracy: 0.0001, "X should be o 60+10*sqrt(2)/2");
 XCTAssertEqualWithAccuracy(tg.y, 160 + 10 * sqrt(2) / 2, accuracy: 0.0001, "Initial Y value should be 160+10*sqrt(2)/2");
 XCTAssertEqual(tg.angle, 45, "Initial angle should be 45");
 
 UIGraphicsEndImageContext();
 
 }

Testing the View Controller

Now let’s test things at a higher level. Generally, we don’t do UI testing in unit tests, but we can do some basic tests to ensure everything is created properly and is more or less hooked up correctly. XCode does have built in support for UI testing and perhaps we’ll have a look at that in a later blog post. In this case we are going to create the storyboard (where the UI is defined) and from that instantiate our View Controller. Then we do a dummy access for the view to get the delay loaded elements loaded and start calling the View Controllers methods starting with viewDidLoad(). For our test we set the fractal level to 3 in the text box and then call the notification methods and see if it then updated properly in our View.

func testInitialViewController()
 {
 // Test that the storyboard is connected to the view controller and
 // that we can create and use the view and controls.
 
 let storyboard = UIStoryboard(name: "Main", bundle: nil)
 
 let viewController = storyboard.instantiateInitialViewController() as! ViewController
 _ = viewController.view
 viewController.viewDidLoad()
 
 viewController.fractalLevelTextField.text = "3"
 viewController.textChangeNot("dummy")
 XCTAssertTrue(viewController.fracView.level == 3)
 // This next line is just to get 100% code coverage.
 viewController.didReceiveMemoryWarning()
 }

Test Coverage

The big questions that is usually asked about unit tests, is how much code to they cover (or exercise). XCode can give this report which you can see in the screenshot:

Screen Shot 2016-05-31 at 10.42.53 AM

Amazingly we nearly get 100% coverage. The only module with poor coverage is AppDelegate.swift which is code generated by XCode and then I never use it. The reason we get such good coverage is that this is a fairly simple program and it draws the snowflake when it initializes. But the key point here is that I am testing a lot of things fairly easily and fairly quickly. It tests the functionality of the turtle graphics library and it tests that the UI is all hooked up properly and working.

A true TDD adherent would delete all the code in AppDelegate.swift, since code shouldn’t be written unless it is to make a unit test pass. But I tend to give an exception to code generated by the tools like XCode, to me this code is Apple’s responsibility and not mine. Plus, if I ever do add code to this class then I would need to start adding unit tests and take it a bit more seriously.

Harder Cases

This program is pretty simple, so I didn’t need to do anything too fancy to unit test it. This is mainly because there is no database access, no hardware access (beyond drawing and one text field) and no communication with a server. If you add these then you would need to add mock, stub and/or fake classes to assist in testing. You want your unit tests to be fast and run on every compile. You don’t want to require databases be setup or that servers are running somewhere in a certain state. Fortunately, Swift is a very powerful object oriented language and creating all of these is fairly easy (extensions are especially helpful). The main problems you run into are in using the Objective-C libraries like UIKit, but again you aren’t the first to run into these and there are tons of sample code, blogs and lessons on how to deal with these.

Another tricky area is with asynchronous calls. When you make a call to the server, the response will be returned asynchronously. What this means that as soon as your unit test sends the server request, it will complete. Since the asynchronous request comes back on a separate thread later, you need to ensure your main thread doesn’t just end and you wait for the results, or the tests in the callback will never be executed. There are lots of examples on how to do this, but it definitely adds some complexity to your tests.

Continuous Build Server

If you have a team, then you will probably want a continuous build server that monitors your source code repository (probably Git) and then does a build and test each time something new is checked in. Apple has an XCode Server that you can purchase if you are a registered developer. Or you could use the Jenkins Build Server which is open source and free. Jenkins also has an XCode plugin to make life a bit easier. Though XCode is quite good at letting you control it from the command line. This way you can be notified as soon as something breaks and ensure problems get fixed right when they are caused, which is the easiest time to fix them.

Summary

That turned out to be quite a long posting on unit tests. XCode and Swift have a very powerful and easy to work with unit testing framework built in. If you are developing a new application, certainly consider using TDD for your development methodology. If you are porting or doing a new version of an app, then certainly try adding some unit tests. Unit tests will help you greatly over the lifetime of your product, making it more reliable, bug free and easier to maintain.

Written by smist08

May 31, 2016 at 8:17 pm

Accessing Sage 300c’s Business Logic from the Web UIs

with 5 comments

Introduction

In the Sage 300 VB UIs, a user would do something in the UI (press a button or tab out of a field) and then the VB UI would be notified of this and would possibly execute a number of Sage 300 Business Logic (View) calls and based on their results update various other fields and possibly provide user feedback via a message box.

In the Web UIs we want to do similar processing since we want to re-use the tried and true Sage 300 Business Logic, but we have to be careful since now the Web UI is half running as JavaScript in the Browser and half running as .Net assemblies on the server. We have to be careful of the communication between the Browser and the server since there will be quite a bit of latency in each call over the Internet. Generally, we never want one user action to result in more than one call to the server (and ideally most user actions shouldn’t result in any calls to the server).

This blog post talks about where you put your code to access the Sage 300 Business Logic and how a UI interaction in the Browser flows through the system to execute this business logic.

Architecture

In the new Web UI architecture, we access the Sage 300 Business logic from our Business Repository classes. The base classes for these provide a wrapper of the Sage 300 .Net API to actually access the Views, but hiding the details of things like session and database link management. Then above this layer are the usual ASP.Net MVC Models and Controllers.

cna2arch

The Sage 300c Web UI Architecture

Generally, we want to put all this logic in the Business Repository so it can be used by multiple higher level clients including the Web UIs, our new RESTful WebAPI and services which are available for other applications to utilize.

Some of the layering is in place ready for additional functionality like customization. We need provide the common interfaces that can act as the basis for programmatic customization by inserting custom modules into the processing flow via Unity Interception.

Moving VB Code

In VB we often make lots of Business Logic (View) calls all interspersed with lots of interactions with various UI controls. This code has to be separated where the Business Logic (View) calls will go in the Business Repository which runs on the server and then the part that interacts with the controls has to move to the JavaScript code running in the Browser. The Business Repository has to provide the necessary data in a single payload which the mode/controller will transport to the Browser for processing.

The easiest way for the repository to transfer data is to have the model provide extra fields for this communication. This way no extra layers need to be involved, the business repository just populates these fields and the JavaScript layers pull them out of the returned JSON object and use them.

But you only want to add so much to the model, since you don’t want it to be too cumbersome to move around and you might want more focused calls. For these we usually define special calls in the controller and these go through a services layer to execute the code in the repository. The service call only passes the exact data needed (like parameters to a function) and knows what data to expect back.

Example

Adding extra fields to the model is fairly straight forward, so let’s trace through the logic of making a services call. In this example we’ll look at the simple case of checking a customer’s credit limit in A/R Invoice Entry (which is using a stateful business repository). We’ll start up in the JavaScript code and work our way down through the layers to get an idea of who does what.

So let’s start near the top. In the A/R Invoice Entry UI there are various times when the credit limit needs to be looked up. So the JavaScript code in the InvoiceEntryBehaviour.js file has a routine to initiate this process. Note that server calls are asynchronous so the response is handled in a callback function.

    showCreditLimit: function (result) {
        // Open Credit Check pop up window
        if (result) {
            var jsonResult = JSON.parse(result);
            if (jsonResult.ShowCreditCheck) {
                arInvoiceEntryRepository.getCreditCheck(jsonResult.id,
                    sg.utls.kndoUI.getFormattedDate(jsonResult.docDate),
                    sg.utls.kndoUI.getFormattedDate(jsonResult.dueDate),
                    "n" + invoiceEntryUI.CurrencyDecimals, jsonResult.totalPaymentAmountScheduled,
                    jsonResult.prepaymentAmount);
            } else {
                onSuccess.onCreditClose();
            }
        }
        invoiceEntryUI.ModelData.isModelDirty.reset();
    },

This calls a function in the InvoiceEntryRepository.js file to actually make the call to the server:

    getCreditCheck: function (customerNumber, documentDate, dueDate, decimals, invoiceAmount,
        prepaymentAmount) {
        var data = {
            id: customerNumber,
            docDate: documentDate,
            dueDate: dueDate,
            decimals: decimals,
            totalPaymentAmountScheduled: invoiceAmount,
            prepaymentAmount: prepaymentAmount
        };
        sg.utls.ajaxPostHtml(sg.utls.url.buildUrl("AR", "InvoiceEntry", "GetCreditLimit"), data,
              onSuccess.loadCreditLimit);
    },

This will initiate the call to the server. The URL will be built something like servername/Sage300/AR/InvoiceEntry/GetCreditLimit. The ASP.Net MVC infrastructure will use configuration by convention to look for a matching entry point in a loaded controller and hence call the  GetCreditLimit method in the InvoiceEntryController.cs file:

        [HttpPost]
        public virtual ActionResult GetCreditLimit(string id, string docDate, string dueDate,
               string decimals,decimal totalPaymentAmountScheduled, decimal prepaymentAmount)
        {
            try
            {
                return PartialView(AccountReceivable.ARInvoiceCreditCheck,
                      ControllerInternal.GetCreditLimit(id, docDate, dueDate, decimals,
                      totalPaymentAmountScheduled, prepaymentAmount));
            }
            catch (BusinessException businessException)
            {
                return JsonNet(BuildErrorModelBase(CommonResx.NotFoundMessage, businessException,
                    InvoiceEntryResx.Entity));
            }
        }

Which will call the InvoiceControllerInternal.cs GetCreditLimit method:

        internal ViewModelBase<CustomerBalance> GetCreditLimit(string customerNumber,
            string documentDate,
            string dueDate, string decimals
            , decimal totalPaymentAmountScheduled, decimal prepaymentAmount)
        {
            var creditBalance = Service.GetCreditLimit(customerNumber, totalPaymentAmountScheduled,
                 prepaymentAmount);

            if (creditBalance.CalcCustomerOverdue == CalcCustomerOverdue.Yes &&
                creditBalance.CustomerBalanceOverdue > creditBalance.CustomerAmountOverdue)
            {
                creditBalance.CustomerCreditMessage = string.Format(
                        InvoiceEntryResx.CustCreditDaysOverdue,
                        creditBalance.CustomerDaysOverdue,
                        creditBalance.CustomerBalanceOverdue.ToString(decimals),
                        creditBalance.CustomerAmountOverdue.ToString(decimals));
            }

            if (creditBalance.CalcNatAcctOverdue == CalcNatAcctOverdue.Yes &&
                creditBalance.NatAcctBalanceOverdue > creditBalance.NatAcctAmountOverdue)
            {
                creditBalance.NationalCreditMessage = string.Format(
                        InvoiceEntryResx.NatCreditDaysOverdue,
                        creditBalance.NatAcctDaysOverdue,
                        creditBalance.NatAcctBalanceOverdue.ToString(decimals),
                        creditBalance.NatAcctAmountOverdue.ToString(decimals));
            }

            creditBalance.CustomerCreditLimit =
                 Convert.ToDecimal(creditBalance.CustomerCreditLimit.ToString(decimals));
            creditBalance.CustomerBalanceVal =
                 Convert.ToDecimal(creditBalance.CustomerBalanceVal.ToString(decimals));
            creditBalance.PendingARAmount =
                 Convert.ToDecimal(creditBalance.PendingARAmount.ToString(decimals));
            creditBalance.PendingOEAmount =
                 Convert.ToDecimal(creditBalance.PendingOEAmount.ToString(decimals));
            creditBalance.PendingOtherAmount =
                 Convert.ToDecimal(creditBalance.PendingOtherAmount.ToString(decimals));
            creditBalance.CurrentARInvoiceAmount =
                 Convert.ToDecimal(creditBalance.CurrentARInvoiceAmount.ToString(decimals));
            creditBalance.CurrentARPrepaymentAmount =
                 Convert.ToDecimal(creditBalance.CurrentARPrepaymentAmount.ToString(decimals));
            creditBalance.CustomerOutstanding =
                 Convert.ToDecimal(creditBalance.CustomerOutstanding.ToString(decimals));
            creditBalance.CustomerLimitExceeded =
                 Convert.ToDecimal(creditBalance.CustomerLimitExceeded.ToString(decimals));
            creditBalance.NatAcctCreditLimit =
                 Convert.ToDecimal(creditBalance.NatAcctCreditLimit.ToString(decimals));
            creditBalance.NationalAccountBalance =
                 Convert.ToDecimal(creditBalance.NationalAccountBalance.ToString(decimals));
            creditBalance.NatAcctOutstanding =
                 Convert.ToDecimal(creditBalance.NatAcctOutstanding.ToString(decimals));
            creditBalance.NatAcctLimitLeft =
                 Convert.ToDecimal(creditBalance.NatAcctLimitLeft.ToString(decimals));
            creditBalance.NatAcctLimitExceeded =
                 Convert.ToDecimal(creditBalance.NatAcctLimitExceeded.ToString(decimals));

            return new ViewModelBase<CustomerBalance> { Data = creditBalance };
        }

This routine first calls the GetCreditLimit service in InvoiceEntryEntityService.cs:

        public virtual CustomerBalance GetCreditLimit(string customerNumber,
            decimal totalPaymentAmountScheduled, decimal prepaymentAmount)
        {
            var repository = Resolve<IInvoiceEntryEntity<TBatch, THeader,
                 TDetail, TPayment, TDetailOptional>>();
            return repository.GetCreditLimit(customerNumber,
                 totalPaymentAmountScheduled, prepaymentAmount);
        }

Who then calls the repository GetCreditLimit routine in InvoiceEntryRepository.cs. This routine then appears to do regular View processing using the base repository wrapper routines that insulate us from the session/dblink handling logic as well as do some basic error processing:

        public virtual CustomerBalance GetCreditLimit(string customerNum,
            decimal totalPaymentAmountScheduled, decimal prepaymentAmount)
        {
            _header.Read(false);
            _creditCheck.SetValue(CustomerBalance.Fields.CustomerNumber, customerNum);
            _creditCheck.SetValue(CustomerBalance.Fields.CurrentARInvoiceAmount,
                totalPaymentAmountScheduled);
            _creditCheck.SetValue(CustomerBalance.Fields.CurrentARPrepaymentAmount,
                prepaymentAmount);
            _creditCheck.Process();
            return _creditCheckMapper.Map(_creditCheck);
        }

Finally, down in the business repository, the code should look fairly familiar to anyone you has done any C# coding using our Sage 300 .Net API. Further this code should also appear somewhere in the matching VB code, and besides being translated to using the .Net API, its become quite separated from the UI control code (in this case the JavaScript).

At the end of this all the calls return propagating the returned data back to the Browser in answer to the AJAX call that it made.

It might look like a lot of code here, but remember the business repository and JavaScript bits have corresponding VB code. Then the other layers are there to make all the code more re-usable so that it can be used in contexts like WebAPIs and allow interfaces to provide the hooks needed for customization.

Summary

This article is intended to give you an idea of where to put your code that accesses the Sage 300 Business Logic and then how to call that from the Web UIs. There are a lot of layers but individually most of the layers are fairly simple and most of the code will appear in the Business Repository and the JavaScript behavior code.

Written by smist08

February 12, 2016 at 3:27 am