Stephen Smith's Blog

Musings on Machine Learning…

Posts Tagged ‘easymock

Unit Testing Web UIs in Sage 300 ERP

leave a comment »

Introduction

Unit Testing is a technique to test individual units of source code. Usually in the Java world this means a set of tests to test an individual class. The idea is to test the class (or unit) in complete isolation from the rest of the system, so calls to other parts of the system would be stubbed or mocked out. Unit test are typically written by the software developer to perform white box testing for their class. Unit testing is only one sort of testing there would be other types of testing for various integration testing, user testing, load testing, manual testing, etc.

The goal of unit testing is to show that the individual parts of a program are correct and then to deliver the following benefits:

  • Establish a well-defined written contract of what the code must do.
  • Find problems early in the development cycle.
  • Facilitate change and refactoring since you can rely on the unit tests to prove things still work.
  • Simplify integration since you know the building blocks work correctly.
  • Provide documentation, the unit tests show examples of actually using the class being tested.

In Extreme Programming and Test Driven Development, the unit tests are written first and then passing the unit tests acts as an acceptance criterion that the code is complete.

JUnit

JUnit is a unit testing framework for Java programs. It is integrated into most IDEs like Eclipse and it is easy to run the tests from build tools like Ant. Usually in Eclipse you have a test tree with all the test classes, usually a test class for each real class and the test class containing all the tests for the real class. JUnit support a number of annotations that you put on the methods in the test class to specify which ones are tests to run as well as optionally specifying any order or dependency requirements (the best unit tests run completely independently of each other). Then it adds a number of methods to call to test for pass and failure of the tests (via various forms of assert) and to report the results back to the test framework.

GWT

The Google Web Toolkit (GWT) supports unit testing with something called GWTTestCase which bridges GWT to JUnit. If you want to interact with GWT controls then you have to use this. However it’s relatively slow. You want unit tests to run blazingly fast. A unit test should only take a few milliseconds to execute. If it takes longer then it won’t be run often. You want it so your unit tests can be run every time you compile and so they don’t slow you down as you work.

We use GWTTestCase to test our extension of the GWT CellTable widget. These tests take 10 minutes to run. This is way too slow to expect them to be run on a regular basis. Hence they don’t provide the immediate feedback to a developer working on this widget that we would like.

Fortunately since in GWT you are writing all your classes in Java, if you structure your program correctly you can have most of your classes not interact with GWT or easily mock the GWT part of it. Then you can use JUnit directly to unit test your program.

MVC

One of the key goals of us enforcing MVC design patterns on our developers creating UIs is to facilitate good unit testing. This way if most of the code that needs testing is the Model and Controller, these can be tested with JUnit and we don’t need to use GWTTestCase. This greatly simplifies unit testing and greatly improves the productivity and speed of the tests. Usually our View part is very small and mostly implemented in the SWT framework, so the unit testing is then in framework unit tests (many of which are GWTTestCase), and not in the actual UIs.

EasyMock

EasyMock is a library that can dynamically “mock” classes that aren’t part of the test. With unit tests we only want to test the one class at a time and we want the tests to run quickly. We don’t want to require a particular database be in place and then have to wait for the tests to run as it makes database queries. We want the tests to run in milliseconds and we want the tests to run in spite of what might be happening in the rest of the system. To this end the test framework needs to replace the real classes that the class being tested calls and interacts with, with something appropriate for testing. These test replacement classes then have the ability to return unexpected or rare results such as network disconnection errors, or other rare hard to setup type of scenarios. One way to do this is to write “stub” classes that replace the real classes and then only have code to do what is required for the testing. Writing and maintaining stub classes is difficult since you need to keep them in sync with the classes they are stubbing; keeping these correct can become a major amount of work.

EasyMock offers an easier way. It generates the classes on the fly using Java’s proxy mechanism. This way the class interface is always up to date since it is generated from the real class. You first run in a “recording” mode where you record what you want the various methods to return to the test cases. Then you switch to “play” mode and run the tests.

For this to work effectively, you must design classes so classes they use can be mocked. This usually means creating them externally and then passing them to the constructor rather than creating secret internal instances. Generally this is good object oriented design anyway and usually difficulty in constructing unit tests is a “code smell” that something is wrong with the design.  For simple utility classes or simple classes in the same package, it’s often easier to leave them in place rather than mock them, then they get a bit of testing as well and as long as they run quickly, shouldn’t be a problem.

Example

Below is one of the tests from the BOM Detail popup form in Order Entry. This shows the structure of a JUnit test and shows how to mock a big class (namely the Model) that this class interacts with. Notice that the Model is created externally to this class and passed into the constructor, so it is easy to mock. First we record what the mock class should do and then we switch to replay mode to do the actual test.

public class NestedViewBOMDetailsCallbackTest
{
private static final int TEST_PARENT_COMPONENT_NUMBER = 42;

/**
* Tests that the callback’s onPopupClosed handler will filter on the child
* BOMs of the parent component number that was passed in at callback
    * creation time.
*/
@Test
public void testOnPopupClosedFiltersOnChildrenOfCallerParentBOMNumber()
{
// Set up the mock model and the expected method calls that should be
// made on it by the class under test.
OEDocumentBOMDetailsModel mockModel = EasyMock.createMock(OEDocumentBOMDetailsModel.class);
mockModel.filterOnChildBOMsOf(EasyMock.eq(TEST_PARENT_COMPONENT_NUMBER), EasyMock.isA(EntryFilterCallback.class));
EasyMock.expectLastCall();

// Tell EasyMock we’ve finished our setup so that any time we interact
// with the mock, EasyMock records what calls were made on the mock.
EasyMock.replay(mockModel);

// Call the class under test to exercise the method under test.
NestedViewBOMDetailsCallback callback = new NestedViewBOMDetailsCallback(mockModel, EasyMock
            .createMock(BOMDetailsView.class), TEST_PARENT_COMPONENT_NUMBER);
callback.onPopupClosed(null);

// Check that the expected method calls on the mock happened when we
// exercised the method under test.
EasyMock.verify(mockModel);
}
}

Summary

Unit testing is an effective form of testing that does find quite a few bugs and helps keep code running smoothly as a system is developed and refactored going forwards. Sometimes the discipline of unit testing forces programmers to ensure they fully understand a problem before coding resulting in a better solution. Certainly this can’t be the only form of testing, but it is an important building block to a quality software system.

Advertisements

Automated Testing in Sage ERP Accpac Development

with 7 comments

All modern products rely heavily on automated testing to maintain quality during development. Accpac follows an Agile development methodology where development happens in short three week sprints where at the end of every sprint you want the product at a shippable level of quality. This doesn’t mean you would ship, that would depend on Product Management which determines the level of new features required, but it means that quality and/or bugs wouldn’t be a factor. When performing these short development sprints, the development team wants to know about problems as quickly as possible so any problems can be resolved quickly and a backlog of bugs doesn’t accumulate.

The goal is that as soon as a developer checks in changes, these are immediately built into the full product on a build server and a sequence of automated tests are run on that build to catch any introduced problems. There are other longer running automated tests that are run less frequently to also catch problems.

To perform the continuous builds and to run much of our automated tests we use “Hudson” (http://wiki.hudson-ci.org/display/HUDSON/Meet+Hudson). Hudson is an extremely powerful and configurable system to continuously build the complete product. It knows all the project dependencies and knows what to build when things change.  Hudson builds on many other tools like Ant (similar to make or nmake) and Ivy among others to get things done (http://java-source.net/open-source/build-systems). The key thing is that it builds the complete Accpac system, not just a single module. Hudson also has the ability to track metrics so we get a graph of how the tests are running, performance metrics, lines of code metrics, etc. And this is updated on every build. Invaluable information for us to review.

The first level of automated testing are the “unit tests” (http://en.wikipedia.org/wiki/Unit_testing). These are tests that are included with the source code. They are short tests where each test should run in under 1 second. They are also self contained and don’t rely on the rest of the system being present. If other components are required, they are “mocked” with tools like easy-mock (http://easymock.org/). Mocking is a process of simulating the presence of other system components. One nice thing about “mocking” is that it makes it easy to introduce error conditions, since the mocking component can easily just return error codes. The unit tests are run as part of building each and every module. They provide a good level of confidence that a programmer hasn’t completely broken a module with the changes they are checking in to source control.

The next level of automated tests are longer running. When the Quality Assurance (QA) department first tests a new module, they write a series of test plans, the automation team takes these and transfers as many of these as possible to automated test scripts. We use Selenium (http://en.wikipedia.org/wiki/Selenium_(software)), which is a powerful scripting engine that drivers an Internet Browser simulating actual users. These tests are run over night to ensure everything is fine. We have a subset of these call the BVT (Build Validation Test) that runs against every build as a further smoke test to ensure things are ok.

All the tests so far are functional. They test whether the program is functioning properly. But further testing is required to ensure performance, scalability, reliability and multi-user are fine. We record the time taken for all the previous tests, so sometime they can detect performance problems, but they aren’t the main line of defense. We use the tool JMeter (http://jakarta.apache.org/jmeter/) to test multi-user scalability. JMeter is a powerful tool that simulates any number of client’s accessing a server. This tool tests the server by generating SData HTTP requests from a number of workstations and bombarding the server with them. A very powerful tool. For straight performance we use VBA macros that access the Accpac Business Logic Layer to write large numbers of transaction or very large transactions, which are all timed to make sure performance is fine.

We still rely heavily on manual QA testers to devise new tests and to find unique ways to break the software, but once they develop the tests we look to automate them so they can be performed over and over without boring a QA tester to death. Automated testing is a vibrant area of computer science where new techniques are always being devised. For instance we are looking at incorporating “fuzz testing” (http://en.wikipedia.org/wiki/Fuzz_testing) into our automated test suites. This technique if often associated with security testing, but it is proving quite useful for generalized testing as well. Basically fuzz testing takes the more standard automated test suites and adds variation, either by knowing how to vary input to cause problems, or just running for a long time trying all possible combinations.

As we incorporate all these testing strategies into our SDLC (Software Development Life Cycle) we hope to make Accpac more reliable and to make each release more trouble free than the previous.

Written by smist08

April 5, 2010 at 5:24 pm