Stephen Smith's Blog

Musings on Machine Learning…

Archive for September 2010

Preparing for the Sage ERP Accpac 6.0A Launch

with 4 comments

Here in R&D we are working hard to release Sage ERP Accpac 6.0A. This is a really major release for us. It introduces our next generation of Web Based UIs. Whenever the major number in Accpac’s version changes it signals the introduction of a major new technology (1 – Windows, 2 – SQL Server, 3 – Crystal Reports, 4 – 32 Bit, 5 – VB ActiveX UIs). Version 6 is no different, introducing our Sage Web Toolkit (SWT) based zero-client web based (HTML/JavaScript) UIs and the introduction of SData REST based Web Services. I’ve already written quite a few articles on the new features and technologies in Version 6:

This posting is to look at what we are doing now that we are code complete. Many of you can see some of the activities as you participate in the Alpha, Beta or Controlled Release programs, but behind the scenes there are many things going on to prepare for release.

Code Complete

We reached code complete back on August 20, 2010. This was after completing 20 3-week agile scrum sprints ( Prior to the start of the scrum sprints, quite a bit of work had been done working out the framework and doing the underlying research. But these 20 sprints were the bulk of the work. This was our first project where we used agile scrum as our development methodology. As a result we had a few problems along the way, but we learned a lot and are all in agreement that the agile method is far superior to the old waterfall methodology we were using previously. The goal of agile is that each sprint is self-contained, in that any feature (or story) is completed entirely within one sprint including development, QA and documentation. Ideally the product is in a releasable state at the end of each sprint.


Once we reached code complete, we started a regression test phase. This is intended to be a thorough regression test of the entire product to ensure nothing is broken. Ideally if you are perfect at using the agile methodology, you shouldn’t need to do this. You just ship when you are code complete. But we aren’t perfect and we are still learning how to do agile better. In fact agile is a continuous improvement (or kaizen type of process that never sits still. You are always striving to improve. However Accpac is a large and complicated product with many interactions between all its parts. So we really do need to run a full regression to find and remove any funny bugs that were introduced unawares during the sprint process. We continue to do this testing until we release.

Security Audit

Version 6 will still be an on-premise application. For this version, Sage won’t be hosting it as a SaaS product. However we suspect that some customers will expose the Accpac Web Portal and other screens to the Internet at large. They won’t keep it safely hidden behind multiple layers of corporate firewall protection. Once Accpac is exposed to the Internet, security becomes a much larger concern. Application security is more than just switching to using HTTPS instead of HTTP. To this end we embarked on educating our programmers and QA on Internet security threats and added security related practices to our coding standards. Doing this is often called adding a SDL (Security Development Lifecycle As part of this we had an external security firm do a complete security audit on all our new web based technologies. The external firm did find a number of important security vulnerabilities that were all fixed, and we learnt a lot to avoid these problems in the future. This way we received external confirmation that we were following good security practices and would provide a challenge to hackers trying to break in. This testing was started just before code complete and finished during regression.

Performance Testing

With any Web application performance is always an important priority. Over the past couple of releases we’ve been putting more and more performance testing into place to ensure the Accpac application performs well in all areas. This include the performance of a single user doing things, how long does it take for screens to appear, how long to perform processing tasks, etc. Then we have multi-user testing where we want to ensure that the single-user performance doesn’t degrade badly once many users are in the system doing the same thing. Over the web there are a couple of extra complexities such as how good is your network connection. We want Accpac to work well over any Internet broadband network, not just on a fast corporate LAN. Some of the testing we are doing is:

  • Using Selenium ( to run a wide variety of user tests and record the performance on a weekly basis and compare to benchmarks set by Product Management. These are single user tests to make sure performance is good for a single user. We also use Selenium for automated functional testing.
  • Using JMeter ( to run a wide variety of tests under multi-user load. JMeter doesn’t test what runs in the browser, it simulates what the browser does at the network level to test the server under heavy load. Again these tests are run weekly and tracked to watch the trends.
  • Perform weekly manual multi-user testing. Every week we have all of QA test against the same server and run “scripts” of typical tasks that real customers would do. This gives us confidence that we work well in a real customer environment and it validates that our automated tests are testing the right things.
  • Using Fiddler ( to record all HTTP requests made by Accpac and to ensure that these are kept to a bare minimum and that each one is responded to quickly. This is to ensure that we will work well over slower Internet connections with higher latency.
  • Continue to use VBA macros to ensure any changes haven’t had a detrimental effect on the business logic performance.

Quality Debt Reduction

With every release we want to improve the quality of Accpac. With this release we have cataloged all known defects at the start of the release and a release certification requirement is that the number of total defects must be lower at release. So from release to release the number of open defects is reducing. There is also a lot of work going on so we prioritize these defects so that the ones that affect customers the most (based on severity and frequency) will be fixed first. We want to ensure this isn’t a numbers game where we just fix lots of issues about under-lining or field widths in reports. The real aim to ensure we have real measurable metrics that are published in a dashboard that is visible to everyone in Sage up to the CEO, to ensure we are improving quality in a real and measurable way.


To help our ISVs ( come up to speed quickly, we’ve released “alpha” versions of the software for them to work with. We release these versions well ahead of code complete so ISVs have plenty of time to learn the new Web UI programming model and incorporate it into their products. We’ve been holding training classes on this; this year at South African Insights (Feb), North American Insights (May) and TPAC (August). We will be holding this training again in Australia and Asia. Having ISVs on board and developing using the new Web based technology is crucial to its success.


We have now had two Beta releases. These are to start to get business partners familiar with the new features; to test it and to provide feedback. Beta started just before code complete and allows some room to incorporate feedback (which we have) as well as provide enough time to fix bugs. This gives BPs a chance to run test conversions of client data to ensure it can be upgraded to the new version, plus to determine any work that will need to be done to customizations. Since ERP applications tend to be mission critical, we don’t want the beta installed in live environments, just test environments.

If any BPs can send us their client’s data, this is a great help for our real world testing. For the client and BP this ensures that we have used their data as a test case and they will know it will upgrade without problems. Of course the customer has to be OK with sharing their possibly very sensitive data. But if they do, we greatly appreciate it. Sage has a data retention and protection policy that spells out the steps we take to safeguard the data and how long we will keep it.

Controlled Release

Next is controlled release. This is when we take a release candidate of the software and install it in live environments. Due to the mission criticality of these environments, we ensure we have Support and Development on call to quickly address any issues. This is the final test to ensure everything works fine in a real customer environment. We will then help to move the customer to the final version (if necessary). The controlled release program is very important to us and we put a lot of weight in how it goes when deciding whether to release.

Release Certification

Now that all the testing is completed, all the alpha, beta and controlled release programs are finished, we need to decide if we should release. We now prepare a “release certification” report of the results and statistics from all the testing and all the feedback, bring all the stakeholders together and have a meeting to discuss the release. Beta and controlled release feedback is very prominent here. At this meeting we approve, reschedule or reject the release. This is more than just development, this meeting is to also ensure that all training material is in place, documentation is complete, language translations are done, marketing material is ready, the Sage Web site is ready, etc. Once we have the go ahead from this meeting, the release will proceed and you will be able to download the product.


 This was a quick summary of some of the tasks that are being performed prior to the release of Sage ERP Accpac 6.0A. This is a huge project that involves many people. All of whom are working hard to make this a successful release and to launch the next generation of Accpac technology.

Written by smist08

September 25, 2010 at 4:08 pm

A Short History of the Accpac View Interface

with 6 comments

In the beginning, there was the View. The View is a Sage ERP Accpac Business Logic object, for an introduction to Views see: The View is a Windows DLL with a set interface. Views expose a common set of functions, which are displayed below (this set of functions has grown over time):

There are functions to control the View, manipulate fields, get meta-data, manipulate records and manipulate sets of records. Each of these routines has a corresponding function inside the View DLL. In the early days all these functions were exported from the View DLL and theoretically could be called directly. However we never endorsed this, did it ourselves or provided instructions on how to do that. In a way calling these functions directly in the View DLL was the first View interface, even though it was never used. Not listed here is the real function that is called in the View DLL, namely rotoEntry. This function takes a structure as its argument that specifies the function to call and has some generic arguments that are used as the arguments to the various functions following a set convention. Again the users of Views have never loaded the View DLL and called the rotoEntry function directly.


You can call the Views by a couple of System Manager DLLs called Roto and RotoView. Roto is the low level object manager in Accpac, it handles the loading of View DLLs and figuring out things like what to load when Views are sub-classed. Roto also routes messages within Accpac to make sure the right View is called when a View call is made. RotoView is the first View interface that people actually used and is still in use today. Rotoview manages Roto to load the right View and provides convenient functions that correspond to the real functions inside all View DLLs. Below is a diagram showing RotoView and some of  its functions using Roto to access  a View through its rotoEntry routine.

RotoView provides an easy to use API for accessing Views, you just call viewLoad, viewOpen and then start doing real work. The Views still use this interface to directly talk to each other. All other interfaces talked about below also ultimately call the RotoView interface to do their real work.


One thing that was considered difficult when using the View interface is something called “View Composition” which is how you combine a number of low level Views into higher level objects. For instance to enter an order into O/E you need to use the order header view and order detail view together to enter the order. To use these they need to be composed together to make an order set of Views. This process can be a bit tricky so we wanted to hide it from users using macros. In the original version of CA-Accpac/2000, it shipped with a macro language called CA-BLE (Computer Associates Basic Language Engine). In the macro language we wanted to ability to call Views, but to hide some of the complexity, such as composing Views. The Icmd and cmd interfaces were intended to do this. The View compositions were specified in the Objects section of the application’s xx.ini file, and were done automatically for you. These interfaces were also used for the original import/export program used by our CA-Realizer based User Interface programs. This then led to the architecture of CA-Accpac/2000 1.0A as shown in the diagram below (Roto and RotoView are combined into Roto):

Icmd and cmd are nearly the same, Icmd is just a little lower level and is the actual interface used by the old import/export program. Some other programs used the cmd interface such as the data integrity checker program. Cmd/Icmd are still included with System Manager for any third party programs that might use them, but aren’t used by anything that Sage ships.


A weakness of all the APIs discussed so far is that they are only callable by a true Accpac SDK application launched from the Accpac Desktop. As we developed Accpac for Windows 3.0A we were developing the iConnect Server which needed to run independently on a server and not from the desktop. We were also getting complaints that it was hard to integrate Accpac to an existing standalone application, since you needed to develop an Accpac SDK part that would talk to the Views, the standalone non-SDK application couldn’t just call the Views directly itself.

And so was born XAPI. The main innovation in XAPI was that you first opened a session to Accpac providing basic sign-on information then went ahead and used the Views. This meant that anyone could use the XAPI interface whether they were an SDK application or not. So XAPI consists of some session management routines, then a set of routines very similar to what is in RotoView, except that they take the session as a parameter rather than the roto view handle used by RotoView. XAPI also contains routines to print reports and access other System Manager info easily. This way iConnect and non-SDK applications could fully integrate into Accpac. Later the iConnect program would be evolved into the Process Server, but both of these programs are long gone. The XAPI interface remains since it is used by many third party programs to interface to Accpac to this day.

This interface also stopped hiding the View compositions. We found that hiding compositions made it very hard to do some things with the CMD interfaces.


A further weakness of all the APIs mentioned so far is that they are all native DLL interfaces. These are easy to call from C programs, but problematic for other programming systems like Delphi or FoxPro. For Accpac version 4.0A we went 32-Bits and changed macro languages from CABLE to VBA. To integrate to VBA you need to provide a COM interface. So we could kill two birds with one stone, provide a COM interface to allow an easily usable interface for programming systems that have trouble calling DLLs directly and add VBA as Accpac’s macro language.

So was born the a4wcom interface. This is a COM interface that is layered on top of the XAPI interface and hence provides the same functionality of XAPI. It can be used by SDK or non-SDK applications, it can call the Views and print reports. Further it’s easy to use by any programming system that can call COM objects which includes Delphi, Foxpro, VBA, VB, VBScript, JavaScript, Perl, etc.


Advance to Accpac Advantage Series 5.0A. With this release we re-wrote our User Interface programs from CA-Realizer to Visual Basic (VB). To do this we needed a strong COM interface. Unfortunately you can’t just change an existing COM interface since it will break everything that uses it. So we made the decision to leave a4wcom alone and develop a version 2 COM interface for Accpac namely a4wcomex. This interface doesn’t go through XAPI, it goes directly to RotoView. It’s also designed so that it can be “remoted” meaning that it can be called over the Internet. This was then the basis for Accpac 5.x web deployed mode. See the diagrams below for how remoting is fit into our model (the top purple boxes are the typical components that make up a VB UI program). The AccpacCOMAPI below is really a4wcomex.


When talking to Views from a VB UI program, the VB code doesn’t talk directly to a4wcomex, instead the VB UI code talks to an ActiveX control called the DataSource control. The DataSource control then talks to the AccpacCOMAPI to perform the actual View operations. The reason we do this is that the DataSource is actually a visual control which means it can be configured at design time in the form designer. This means you specify which views to use and such in the form designer saving you writing quite a bit of code, most View compositions can also be specified this way. Another function of the DataSource is that it provides caching to reduce the number of View calls over the Internet when running in Web deployed mode.


With the Accpac Advantage Series 5.0A release we introduced Web Deployed mode where the UIs run as ActiveX controls inside the IE browser. With the original release we accomplished this by using DCOM in place of COM to do the View calls over the Internet. However this proved problematic, because DCOM is slow and DCOM uses random ports to communicate, which means it is very firewall unfriendly. You don’t know what ports it might use, so you have to leave them all open, which is very bad from a security point of View.

So with version 5.1A we added the option to use .Net Remoting as the communications protocol to talk over the Internet. To use .Net Remoting we needed to develop a .Net layer that sat over top of a4wcomex which we could then remote. Since we had to develop this layer anyway, we made it publicly available as another View interface for the added convenience of third party applications that were developing in .Net languages like C# or VB.Net.


If you have a .Net interface to something, then Microsoft through Visual Studio provides a tool that lets you expose that .Net interface externally as a SOAP web services interface. So when we developed the .Net interface, we got the SOAP interface for free.

However, you tend to get what you pay for, and like relying on DCOM to give you internet access for free over COM, it turned out to not work all that well.

With Version 6 we are removing the SOAP web service to be replaced by the much superior REST based SData web services interface.


Fast forward to Accpac 6.0 where we are using the Google Web Toolkit to move all our user interface programs from VB to the web using HTML and JavaScript. For server processing we are handling Web service requests via a Java program which then needs to talk to the View. Java doesn’t like using COM or .Net. And for that matter it’s hard to use those on Linux Servers (if we ever want to do that).

So for Version 6 we’ve developed a native Java interface that is similar to the a4wcomex interface and talks to RotoView directly. The Java interface also has the distinction of being our first truly multi-threading interface. Multi-threaded programs could call our other interfaces, but would probably be blocked. The Java interface allows many threads to be running at once all doing work without overly blocking each other from operating.


SData is Accpac 6.x’s new REST base Web Services Interface. For more information on SData see,, and

The way the SData interface works is we have a Java Server application (SDataServlet) that receives the SData webserver requests; it parses and processes these converting the SData web service requests into View calls which are made via the new Sage Accpac Java interface (SA-Java).

Written by smist08

September 18, 2010 at 5:34 pm

Accpac’s Business Logic

with 17 comments

I thought I might spend a few blog postings talking about Sage ERP Accpac’s Business Logic. With all the talk about version 6’s new web based UIs, the Business Logic hasn’t been talked about much lately. However the Business Logic is still the heart and soul of Accpac. It contains all the difficult application logic that provides the true business value and ROI to the product. This blog posting will talk a bit about the architecture and an overview of Business Logic, then in future postings we’ll go into some of the details. Below is an architectural diagram of Accpac showing the Business Logic as the middle tier of a three tier architecture. The Business Logic talks to the Database layer through a database independent API layer (which we talked about here:

Then the various users of the Business Logic talk to it through a number of standard APIs that are part of the Accpac System Manager. Individual Business Logic objects are called Views (not to be confused with database views).

Encapsulates One Logical Entity and Its Operations

Each Accpac View encapsulates one logical entity and its operations. These entities are things like A/R Customer, G/L Accounts, O/E Order headers, etc. They don’t have to be physical objects, they can be abstract concepts like G/L Posting (the View that posts G/L batches). Even though these entities are very different, each and every View has exactly the same API. Within Accpac, rather than have a unique API for each object, where whenever you encounter a new object you need to learn a new API; every Business Logic Entity in Accpac has the same API. This means that if you learn how to do things with A/R Customers then you have also learned how to do the same things to A/P Vendors or G/L Accounts. This is also why there are many different types of APIs to access the Views. There is the .Net interface, two COM interfaces, several DLL interfaces and the forthcoming Java interface. Since the API is the same for every View, we just need to implement this interface in each technology which is reasonably easy to do.

Implemented as DLLs

Each View is implemented as a Windows DLL (or in our Linux port as a Linux Shared Object (SO)). These DLLs can be implemented in any language that can produce DLLs with the set of API functions that we specify. In the original CA-Accpac/2000 1.0B, G/L, A/R and A/P had their Views implemented in COBOL. Since then all our Views are written in C (sometimes with parts in C++). This is a fairly low level way to do things. It might be nice if we used a higher level language like Java, but then we couldn’t have our Views as DLLs. The benefit of DLLs (or SOs) is that they are the lowest level objects in the operating system and you can build anything on top of them. This is how we can have so many different APIs to access the Views, since anything has a mechanism to call DLLs, since you need to have this in order to talk to the operating system (which is just a set of DLLs (or SOs)). The Accpac SDK contains templates and tools to allow you to generate Views from tables that describe what the View needs to do. Then you fill in routines to code the real functionality. You can open multiple copies of the same View each with its own context, this way a View can be used in many ways at one time.

Standard Protocols

Views often don’t act in isolation. Often it takes the co-operation of several Views to accomplish a task. For instance there are two Views, one for the Order header and another for each detail line. For a given order there is one order header and then multiple details, each one for the purchase of an I/C Item or miscellaneous charge. To enter an order the order header and detail Views need to work together. They do this to ensure transactional integrity, multi-user correctness and that various totals are maintained correctly. To do this we have a set of View protocols. If you are entering documents through Header/Detail type Views then there are protocols that you need to follow. Following these well documented protocols (see the System Manager User Guide or SDK Programming Guide) will ensure things go smoothly. If you don’t follow the protocols than you will get strange errors and you transactions may be rejected. These protocols are followed to insert, update and delete documents.

Combine Logical and Physical Data

The lower level Views or Data Views are thin wrappers over various database tables. They expose the fields in the underlying table almost identically to what is documented in the data dictionary. A good many Views are of this type, exposing the physical data in the database. However they can also add some logical fields such as calculated fields, fields looked up from other Views (like perhaps a description for a G/L account) or anything else that will be useful to the user of this View. Sometimes you can set these logical fields to control how the View does calculations, such as setting whether you want taxes calculated automatically or you will manually enter them.

Avoids Duplication of Code

When a View needs data from a table that it doesn’t manager, it doesn’t just go to the database to get the data, it calls the View that manages that other table. This avoids duplication of code since the code for manipulating that object is in one place, namely its View. Then every View that needs that same data calls that View. In fact Views are the primary user of other Views and the majority of View calls are made by other Views rather than the external users of Views like macros or UIs. This also means the other Views can make use of all the logical fields added by the other Views as well as get at the physical data.

Maintain Database Integrity

Views are responsible for maintaining database integrity. We do not rely on the database server software to do this. Views will maintain field level integrity as data is put to the View records. If it’s something simple like a field needs to be upper case and the caller passed in lower case, it will simply convert the data to upper case as it comes in. This is to make the Views more resilient and friendlier to the View’s users. If it can simply fix data, it will. At the next level it will reject bad data, for instance if you enter an invalid currency code into a currency field, it will immediately reject it (actually there is a flag as to whether it immediately rejects it or waits for you to try to insert or update the record). At insert/update time the View will ensure there is record level integrity, any fields that need to be validated as a set together will be. It is also possible in a header/detail situation where data has to be cross validated across several records. This is all handled for you by the View. This is why we insist that anyone writing data into an Accpac company database, must do this through the Accpac Views. Writing data directly to the database through ODBC or ADO is very frowned on and can cause you to become unsupported.

Centralize Data Security

It might appear that Accpac controls data security through its UIs. This is because if you don’t have access to something then either an entire UI is hidden from you or if you don’t have access to a feature in a UI, then that feature is hidden. However the UI is asking the View if the person has access and then hiding the feature as an UI nicety. If the UI was badly behaved and didn’t hide the feature and tried to use that feature in the View then the View would reject it. This way security it maintained whether the data comes from an UI, a macro, import or from a third party application using one of our APIs.

Allow Distributed Application Architecture

Since the View API is well defined and contains a fixed set of functions for all Views, it allows us to easily serialize the calls. This is the process of taking the arguments for functions and putting them out as a serial set of bytes that can be transmitted over a communications link. This allows the caller to use an API that serializes the calls, sends the serialized stream over a communications link like the Internet, have the receiver interpret the serialized stream and make the View call. This enabled various remote APIs such as the old iConnect and Process Servers along with the newer DCOM and .Net protocols in use today. The new SData protocol works a bit differently since it is based on the standard REST protocol, which is already serialized as an URL and XML payload. This is translated into a number of View calls when received by the server.


Hopefully this gives an indication of some of the things Views do and they work. In future blog posts I’ll look at various aspects of the Views in more detail.

Written by smist08

September 11, 2010 at 8:14 pm

TPAC 2010

with one comment

TPAC (Third Party Advantage Conference, is a conference for Sage ERP Accpac third party developers, that is held yearly in Vancouver, Canada. This year it was held at the River Rock Casino Resort ( The idea is that this gives a venue for all the Accpac ISVs (Independent Software Vendors) to showcase their wares to Accpac Business Partners. There is a trade show where everyone gets a booth and each vendor gives a presentation on their products. Then there is plenty of time for networking and education (including a dinner boat cruise). This year there were 32 exhibitors and around 150 attendees. This show is run independently of Sage, though Sage is a sponsor.

Sage’s big North American conference is Insights ( This conference highlights all of Sage’s many products, and although the ISVs have booths and presentations, the main focus of the show is Sage, its direction and announcements. TPAC is nice because it is a more intimate affair with only Accpac partners and ISVs, many who have known each other since the Accpac Plus days. The focus in on the ISVs and they have the spotlight. This makes it a very focused and valuable conference for everyone that attends.

The main development center for Accpac is in Richmond, B.C., only a 10 minute drive away from the conference. This meant that many Sage Product Managers and Developers could attend either the whole show, or at least attend for part. Business Partners also have to option of visiting the Sage development center as part of their visit to Vancouver if they want to.

Pre-conference we ran SDK training for 24 or so ISVs on how to develop Accpac 6 Web Based User Interface programs for their applications. This training ran over 2 days and was held at the Sage Accpac development center in Richmond. The training was very successful and a couple of Web Based UIs based on this technology were being displayed at the TPAC tradeshow.

Although Sage isn’t the focus of the show, Sage people give a few presentations including a Sage ERP Accpac 6 roadmap presentation to highlight the road ahead. Sage ERP Accpac 6 is now in beta, so promoting the beta program, highlighting the marketing plan being put together for the release and showing what comes next. There was also a town hall session given by Laurie Schultz, Accpac’s General Manager. There was generally a very positive reception for version 6.0A and a lot of positive feedback on the current version 5.6A.

ISVs are of crucial importance to any ERP software vendor. No vendor can do everything on their own. The key goal of each vendor is to provide a compelling software development platform along with a core set of applications developed on that platform.  This is a huge undertaking. Then the ERP vendor relies on ISVs to take the platform and develop all sorts of vertical applications for specific industries, useful add-on programs or integrations to other stand-alone applications. For Accpac, the System Manager is the development platform that provides the frameworks, structure and supporting APIs for developing ERP modules. The System Manager is the runtime component and the Accpac SDK contains all the samples, documentation and tools to develop ERP modules to run on SM. Then Accpac provides the core accounting applications including General Ledger, Accounts Payable, Accounts Receivable, Inventory Control, Order Entry, Purchase Order, Payroll, and Project and Job Costing. Then we rely on ISVs to fill in other industry specific modules like POS (Point of Sale), Manufacturing, Service Management, etc.

To provide some concrete examples of Accpac ISVs, here are the ones that exhibited at TPAC. This isn’t an endorsement of any particular vendor; it just means they exhibited at TPAC. This isn’t meant to exclude anyone, if an ISV isn’t on the list it’s only because they didn’t exhibit at TPAC or I made a mistake. Many of the ISVs below have more products than I list, these are just a sampling.

Written by smist08

September 3, 2010 at 5:35 pm

Posted in Business

Tagged with , ,