Preparing for the Sage ERP Accpac 6.0A Launch
- The Portal: https://smist08.wordpress.com/2009/12/03/the-sage-erp-accpac-6-0a-portal/
- Inquiry: https://smist08.wordpress.com/2009/12/11/sage-erp-accpac-6-0-adhoc-query/
- Quotes to Orders: https://smist08.wordpress.com/2009/12/17/sage-erp-accpac-6-0-quote-to-orders/
- Data Snapshots: https://smist08.wordpress.com/2009/12/24/sage-erp-accpac-6-0-data-portlets/
- User Assistance: https://smist08.wordpress.com/2010/06/19/sage-erp-accpac-6-user-assistance/
This posting is to look at what we are doing now that we are code complete. Many of you can see some of the activities as you participate in the Alpha, Beta or Controlled Release programs, but behind the scenes there are many things going on to prepare for release.
We reached code complete back on August 20, 2010. This was after completing 20 3-week agile scrum sprints (http://en.wikipedia.org/wiki/Scrum_(development)). Prior to the start of the scrum sprints, quite a bit of work had been done working out the framework and doing the underlying research. But these 20 sprints were the bulk of the work. This was our first project where we used agile scrum as our development methodology. As a result we had a few problems along the way, but we learned a lot and are all in agreement that the agile method is far superior to the old waterfall methodology we were using previously. The goal of agile is that each sprint is self-contained, in that any feature (or story) is completed entirely within one sprint including development, QA and documentation. Ideally the product is in a releasable state at the end of each sprint.
Once we reached code complete, we started a regression test phase. This is intended to be a thorough regression test of the entire product to ensure nothing is broken. Ideally if you are perfect at using the agile methodology, you shouldn’t need to do this. You just ship when you are code complete. But we aren’t perfect and we are still learning how to do agile better. In fact agile is a continuous improvement (or kaizen http://en.wikipedia.org/wiki/Kaizen) type of process that never sits still. You are always striving to improve. However Accpac is a large and complicated product with many interactions between all its parts. So we really do need to run a full regression to find and remove any funny bugs that were introduced unawares during the sprint process. We continue to do this testing until we release.
Version 6 will still be an on-premise application. For this version, Sage won’t be hosting it as a SaaS product. However we suspect that some customers will expose the Accpac Web Portal and other screens to the Internet at large. They won’t keep it safely hidden behind multiple layers of corporate firewall protection. Once Accpac is exposed to the Internet, security becomes a much larger concern. Application security is more than just switching to using HTTPS instead of HTTP. To this end we embarked on educating our programmers and QA on Internet security threats and added security related practices to our coding standards. Doing this is often called adding a SDL (Security Development Lifecycle http://www.microsoft.com/security/sdl/default.aspx). As part of this we had an external security firm do a complete security audit on all our new web based technologies. The external firm did find a number of important security vulnerabilities that were all fixed, and we learnt a lot to avoid these problems in the future. This way we received external confirmation that we were following good security practices and would provide a challenge to hackers trying to break in. This testing was started just before code complete and finished during regression.
With any Web application performance is always an important priority. Over the past couple of releases we’ve been putting more and more performance testing into place to ensure the Accpac application performs well in all areas. This include the performance of a single user doing things, how long does it take for screens to appear, how long to perform processing tasks, etc. Then we have multi-user testing where we want to ensure that the single-user performance doesn’t degrade badly once many users are in the system doing the same thing. Over the web there are a couple of extra complexities such as how good is your network connection. We want Accpac to work well over any Internet broadband network, not just on a fast corporate LAN. Some of the testing we are doing is:
- Using Selenium (http://seleniumhq.org/) to run a wide variety of user tests and record the performance on a weekly basis and compare to benchmarks set by Product Management. These are single user tests to make sure performance is good for a single user. We also use Selenium for automated functional testing.
- Using JMeter (http://jakarta.apache.org/jmeter/) to run a wide variety of tests under multi-user load. JMeter doesn’t test what runs in the browser, it simulates what the browser does at the network level to test the server under heavy load. Again these tests are run weekly and tracked to watch the trends.
- Perform weekly manual multi-user testing. Every week we have all of QA test against the same server and run “scripts” of typical tasks that real customers would do. This gives us confidence that we work well in a real customer environment and it validates that our automated tests are testing the right things.
- Using Fiddler (http://www.fiddler2.com/fiddler2/) to record all HTTP requests made by Accpac and to ensure that these are kept to a bare minimum and that each one is responded to quickly. This is to ensure that we will work well over slower Internet connections with higher latency.
- Continue to use VBA macros to ensure any changes haven’t had a detrimental effect on the business logic performance.
Quality Debt Reduction
With every release we want to improve the quality of Accpac. With this release we have cataloged all known defects at the start of the release and a release certification requirement is that the number of total defects must be lower at release. So from release to release the number of open defects is reducing. There is also a lot of work going on so we prioritize these defects so that the ones that affect customers the most (based on severity and frequency) will be fixed first. We want to ensure this isn’t a numbers game where we just fix lots of issues about under-lining or field widths in reports. The real aim to ensure we have real measurable metrics that are published in a dashboard that is visible to everyone in Sage up to the CEO, to ensure we are improving quality in a real and measurable way.
To help our ISVs (http://en.wikipedia.org/wiki/Independent_software_vendor) come up to speed quickly, we’ve released “alpha” versions of the software for them to work with. We release these versions well ahead of code complete so ISVs have plenty of time to learn the new Web UI programming model and incorporate it into their products. We’ve been holding training classes on this; this year at South African Insights (Feb), North American Insights (May) and TPAC (August). We will be holding this training again in Australia and Asia. Having ISVs on board and developing using the new Web based technology is crucial to its success.
We have now had two Beta releases. These are to start to get business partners familiar with the new features; to test it and to provide feedback. Beta started just before code complete and allows some room to incorporate feedback (which we have) as well as provide enough time to fix bugs. This gives BPs a chance to run test conversions of client data to ensure it can be upgraded to the new version, plus to determine any work that will need to be done to customizations. Since ERP applications tend to be mission critical, we don’t want the beta installed in live environments, just test environments.
If any BPs can send us their client’s data, this is a great help for our real world testing. For the client and BP this ensures that we have used their data as a test case and they will know it will upgrade without problems. Of course the customer has to be OK with sharing their possibly very sensitive data. But if they do, we greatly appreciate it. Sage has a data retention and protection policy that spells out the steps we take to safeguard the data and how long we will keep it.
Next is controlled release. This is when we take a release candidate of the software and install it in live environments. Due to the mission criticality of these environments, we ensure we have Support and Development on call to quickly address any issues. This is the final test to ensure everything works fine in a real customer environment. We will then help to move the customer to the final version (if necessary). The controlled release program is very important to us and we put a lot of weight in how it goes when deciding whether to release.
Now that all the testing is completed, all the alpha, beta and controlled release programs are finished, we need to decide if we should release. We now prepare a “release certification” report of the results and statistics from all the testing and all the feedback, bring all the stakeholders together and have a meeting to discuss the release. Beta and controlled release feedback is very prominent here. At this meeting we approve, reschedule or reject the release. This is more than just development, this meeting is to also ensure that all training material is in place, documentation is complete, language translations are done, marketing material is ready, the Sage Web site is ready, etc. Once we have the go ahead from this meeting, the release will proceed and you will be able to download the product.
This was a quick summary of some of the tasks that are being performed prior to the release of Sage ERP Accpac 6.0A. This is a huge project that involves many people. All of whom are working hard to make this a successful release and to launch the next generation of Accpac technology.