Stephen Smith's Blog

All things Sage 300…

Automated Testing in Sage ERP Accpac Development

with 7 comments

All modern products rely heavily on automated testing to maintain quality during development. Accpac follows an Agile development methodology where development happens in short three week sprints where at the end of every sprint you want the product at a shippable level of quality. This doesn’t mean you would ship, that would depend on Product Management which determines the level of new features required, but it means that quality and/or bugs wouldn’t be a factor. When performing these short development sprints, the development team wants to know about problems as quickly as possible so any problems can be resolved quickly and a backlog of bugs doesn’t accumulate.

The goal is that as soon as a developer checks in changes, these are immediately built into the full product on a build server and a sequence of automated tests are run on that build to catch any introduced problems. There are other longer running automated tests that are run less frequently to also catch problems.

To perform the continuous builds and to run much of our automated tests we use “Hudson” ( Hudson is an extremely powerful and configurable system to continuously build the complete product. It knows all the project dependencies and knows what to build when things change.  Hudson builds on many other tools like Ant (similar to make or nmake) and Ivy among others to get things done ( The key thing is that it builds the complete Accpac system, not just a single module. Hudson also has the ability to track metrics so we get a graph of how the tests are running, performance metrics, lines of code metrics, etc. And this is updated on every build. Invaluable information for us to review.

The first level of automated testing are the “unit tests” ( These are tests that are included with the source code. They are short tests where each test should run in under 1 second. They are also self contained and don’t rely on the rest of the system being present. If other components are required, they are “mocked” with tools like easy-mock ( Mocking is a process of simulating the presence of other system components. One nice thing about “mocking” is that it makes it easy to introduce error conditions, since the mocking component can easily just return error codes. The unit tests are run as part of building each and every module. They provide a good level of confidence that a programmer hasn’t completely broken a module with the changes they are checking in to source control.

The next level of automated tests are longer running. When the Quality Assurance (QA) department first tests a new module, they write a series of test plans, the automation team takes these and transfers as many of these as possible to automated test scripts. We use Selenium (, which is a powerful scripting engine that drivers an Internet Browser simulating actual users. These tests are run over night to ensure everything is fine. We have a subset of these call the BVT (Build Validation Test) that runs against every build as a further smoke test to ensure things are ok.

All the tests so far are functional. They test whether the program is functioning properly. But further testing is required to ensure performance, scalability, reliability and multi-user are fine. We record the time taken for all the previous tests, so sometime they can detect performance problems, but they aren’t the main line of defense. We use the tool JMeter ( to test multi-user scalability. JMeter is a powerful tool that simulates any number of client’s accessing a server. This tool tests the server by generating SData HTTP requests from a number of workstations and bombarding the server with them. A very powerful tool. For straight performance we use VBA macros that access the Accpac Business Logic Layer to write large numbers of transaction or very large transactions, which are all timed to make sure performance is fine.

We still rely heavily on manual QA testers to devise new tests and to find unique ways to break the software, but once they develop the tests we look to automate them so they can be performed over and over without boring a QA tester to death. Automated testing is a vibrant area of computer science where new techniques are always being devised. For instance we are looking at incorporating “fuzz testing” ( into our automated test suites. This technique if often associated with security testing, but it is proving quite useful for generalized testing as well. Basically fuzz testing takes the more standard automated test suites and adds variation, either by knowing how to vary input to cause problems, or just running for a long time trying all possible combinations.

As we incorporate all these testing strategies into our SDLC (Software Development Life Cycle) we hope to make Accpac more reliable and to make each release more trouble free than the previous.

Written by smist08

April 5, 2010 at 5:24 pm

7 Responses

Subscribe to comments with RSS.

  1. Stephen,

    Interesting, as again, to read this blog post. Whatever you have mentioned, as tools that are getting used at various types of testing, are they applicable only for ver 6.0 or they can also be used for ver 5.6 / 5.5 also?

    Is it a generally accepted kind of testing that we can adopt for any SDK development exercise?


    Sundaresan Ramanathan

    April 9, 2010 at 10:18 am

    • These tool are all for 6.0A testing. They are all open source tools for doing web development and testing. This means anyone developing with the 6.0A SDK will be able to use them also. We should make sure there are articles posted on the Wiki on how to make best use of these tools.


      April 10, 2010 at 4:14 am

  2. […] 6.0A Sage ERP Accpac 6 Performance Testing Sage ERP Accpac 6 Security Sage ERP Accpac 6.x Mobility Automated Testing in Sage ERP Accpac Development How to Layout Forms in Sage ERP Accpac 6 Creating a Web Form for Accpac 6 Client Logic for an […]

  3. […] also use automated testing. We extensively use Fitnesse and Selenium as I blogged about here and here. We can string all these tests together, or run them repeatedly to create long running […]

  4. […] complete test coverage using various levels of automated testing. I covered some aspects of this here. You still want to do some manual testing to ensure that things still feel right, but you don’t […]

  5. […] these branches. As part of the build we run a number of unit tests, but we also have a large set of automated tests that run on a separate set of servers, and now we want to run these automated tests on the […]

  6. […] As part of developing for the cloud, we want to release fairly regularly and can’t afford long manual regression tests. As a result we have a lot of emphasis on Unit Tests and Automated tests, such as those blogged here. […]

Leave a Reply

Fill in your details below or click an icon to log in: Logo

You are commenting using your account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: