Posts Tagged ‘Sage’
A lot of my blog posts are to answer questions that I frequently receive. Then I have a ready answer of a blog posting link, or perhaps people read my blog and it saves me receiving an e-mail. This blog posting is along the same lines as I get asked quite frequently about our SDLC (Software Development Lifecycle). Usually this is in regards to someone trying to fill out a giant RFP full of questions that are mostly irrelevant to purchasing ERP software.
I covered various aspects of our development process in other blog posting which I’ll refer to here. Plus our process is always evolving as we learn from our experiences and try to improve. What I’m writing about here is specifically what the development team for Sage 300 ERP does, but a lot of it is also used by other Sage teams on other projects. There are always slight variations as the different teams do have their own preferences and as long as they follow the general standards that’s ok.
Within R&D we use the Agile Development Methodology, but R&D exists within a larger context within Sage, much of which doesn’t use Agile frameworks. As a result our Agile development has to fit somewhat within a larger non-Agile system that tracks and coordinates the various projects going on around Sage. This is to ensure all departments know what is going on and can plan accordingly.
We have a general PMO department that tracks all the various projects. It coordinates getting approval for projects, determining release criteria and coordinating all the various departments such as Marketing, Product Management, IS, etc. So they can do their pieces at the appropriate time.
Initial product ideas come from our Innovation Process, but before converting an innovation idea into a larger development effort there is usually some POC work and some initial rough estimates that then lead to a project kickoff process where an initial business plan, along with an initial project plan are presented and either approved or rejected.
The project is then tracked by PMO as it goes through development and then at the end when the Agile part of the development is done, there is a release certification meeting where all the stakeholders get together that the solution is ready for customers. This includes that the software is ready and of a high quality, but also that support is ready, training material is available, back end systems are setup to take orders, marketing is ready and all the other pieces that go into a product launch.
Also at this time we run a final regression to ensure that everything is working correctly. Generally this is only a couple of weeks as a sanity check that the Agile process below worked correctly.
Before making the product available to all customers, we first spend a few months in a controlled release with a handful of live customers to ensure everything works well for them. This not only tests the software, but their ability to be on-boarded, supported and trained. After this has proceeded successfully then the product is made available for everyone.
Some of these reasons for this non-Agile framework is that a number of parts of our organization having adopted Agile yet and eventually they will need to if we are to have truly effective cloud based products. The other reason that I blogged about here, is the need to coordinate many disparate teams working on different parts of a larger solution.
We’ve been using Agile programming for a number of years now. We use 2 week sprints, have sprint planning, maintain a backlog, have daily standups, sprint demos, sprint retrospectives and all the other aspects of Agile. We use VersionOne to manage our projects. In Agile you execute the backlog story by story and have very tight rules on what it means for a story to be “done”. This way as each story is completed, it is fully tested, documented and ready for use. The important thing here is to not build up a large list of defects that need to be fixed near the end of the project. Basically when the last story is finished (done) then the product should be ready to put in customer’s hands.
The doneness criteria vary a bit by Agile team, but here are the doneness criteria for a team on one of our current projects:
- All tasks items in a backlog story have a closed status
- The code builds successfully without compiler errors and warnings and without ReSharper issues
- The code is deployed to test environment successfully
- The code is checked into the correct repository and branch
- The code conforms to Sage Branding Guidelines.
- The code performs to performance standards
- The code is reviewed for applicable use of all documented standards.
- The code is refactored and reviewed for good OOP implementation
- The code conforms to UX wireframes, design and CSS guidelines.
- Code coverage minimum percentage is 70% with a target of +90%
- The UI displays correctly in all supported browsers (IE, Safari, Firefox, Chrome)
- Unit tests are included and run successfully
- Automation tests are included and run successfully in test environment
- The environment has not been corrupted (database, etc.)
- The QA and QA Review tasks are included and complete
- Reviewed and accepted by the Product Owner
- Implement and document any build and/or deployment changes
- Knowledge Transfer Activities (wiki updated, code walkthrough, group code review, etc.)
- Remaining hours for tasks are set to zero and the backlog item is closed
Ensuring stories are done is the key to a well-run Agile process, probably more important than a well-structured prioritized backlog.
As part of developing for the cloud, we want to release fairly regularly and can’t afford long manual regression tests. As a result we have a lot of emphasis on Unit Tests and Automated tests, such as those blogged here.
Similarly branching strategy, source code management, build management and continuous integration are also important parts of this process.
This was a quick overview of our Sage 300 SDLC. With any big project there are always a lot of moving parts and these have to be tracked accurately and reliably. Agile doesn’t mean that we don’t have deadlines or firm requirements. It does mean that we develop the most important things first and build quality in from the very start.
I’m currently travelling to Bangalore, India to visit a large number of off-shore team members we work with on our projects. So for something different, I thought I’d try travel-blogging. I’ll be writing this blog as I go along and then periodically post my progress. This is my second trip to India, I visited Chennai back in 2008 for ten days.
Sage is partnered with several Indian companies to provide extra capacity for our projects. For this trip I’m visiting Sonata which has teams participating in several important Sage projects. I blogged previously on accelerating projects, which was really talking about our adding capacity through additional teams at Sonata. It’s great to have extra capacity and the ability to get more done, but it’s also a big challenge keeping all the teams moving in the same direction and continuously removing roadblocks and bottlenecks. Our goal is to treat the Sonata teams as if they were regular Sage agile teams with full access to all Sage resources like source control and other internal systems. To make this process work many Sonata folk visit our Richmond office and we have several staff visiting Bangalore. This is my turn.
Indian Visa Process
To back up a bit, travelling to India is a bit more difficult than other places due to the Visa process. To do this I needed letters from Sage and from Sonata giving my reasons for travel and that I’m still being paid by a Canadian company. Then you need to fill out and extremely long on-line visa application that includes detailed questions on yourself, your spouse and your parents. Gathering all the data for this form took several days, and when you save this form, it is quite buggy retrieving what you had before, so you need to check it closely. You also need a US sized passport photo and a Canadian passport with 1 year left (since I wanted a 1 year multi-entry visa). With all this you make an appointment with the company (BLS) that processes the visas. The earliest I could book was 1 week later. Then you show up for your appointment and they double check all your papers and take the rather large fee ($200). They take all this as well as your passport and promise to process it in 7 business days. Basically at this point they give it all to the Indian consulate for processing. Fortunately this all went fine and 5 days later their website said my passport was ready for pickup. Generally of all the countries I’ve visited, this is by far the hardest visa process.
If you don’t travel much, you should go to a travel clinic to get the right shots when visiting India. I travel quite a bit and all my shots are up to date. The first time I went to India, I needed to get 5 injections. I do recommend taking something like Dukoral since I’ve never had any tummy troubles when I’ve taken this ahead of a trip. Malaria pills may or may not be required depending on where you are going.
It’s a Long Way
I travelled to Bangalore via Hong Kong. It’s a 14 hour flight to Hong Kong and then a 6 hour flight on to Bangalore. Generally the best way to endure a long flight is to sleep through it. The Hong Kong flight left at 2:30pm, so I wasn’t sleepy until near end. By the time I was on the Bangalore flight I was so tired I slept through most of the flight. For some reason all international flights in and out of Bangalore arrive and leave between midnight and 4am. When you arrive you don’t really care what time it is just want to get to the hotel and sleep, so make sure you book for a day earlier, so you don’t need to then wait till 2pm to check in.
Managing in Bangalore
Some Indian cities can be quite hard to navigate. But Bangalore is fairly easy. You do need know how to cross the streets (i.e. make eye contact and walk slowly across the traffic, which will flow around you). If you are worried about having to eat lots of extremely hot Indian food, then don’t worry there are many good restaurants from other cultures like Italian or Mexican. Plus generally I don’t find the Indian food that hot here (perhaps it’s toned down for the tourists). I find the level of English was quite good and haven’t had any problems communicating.
Getting directions and finding your way around isn’t that hard and the area around my hotel (in the downtown old part of the city) seems quite safe. There are quite a few parks around and you can see quite a bit just walking around.
The office is 10km away and parts of the journey can be quite congested, but I find the traffic here to be better than in Chennai, Bangkok or Ho Chi Minh City. They are building a new elevated metro system which is causing quite a bit of road disruption along the way, but they seem to keep traffic flowing.
The office building is located in the Global Village Tech Park outside the city. There are quite a few tech companies located here including Sonata, MindTree, Accenture, HP and Texas Instruments. The office environment I’m at is quite nice. The building is modern and the work environment is quite pleasant. It uses an open office concept and provides a nice productive team environment.
Since this is India the company parking is quite different than what you would expect in North America. To maneuver quickly through traffic, two wheels is the way to go. If everyone using motorbikes was to switch to cars it would be total grid lock here.
This ends part 1 of my travel to India. I’m now here and settled in. Getting here is half the battle, now it’s time to get some productive work done.
From the Sage 300 ERP to Sage CRM integration there is the ability to run a number of Sage 300 ERP screens. These are the older VB screens being run as ActiveX controls from the IE browser. Not to be confused with the newer Quote to Order web based screens. A common request is how to customize these screens to you run the customized screen from Sage CRM rather than the base screen.
This blog posting covers how to run customized screens from Sage CRM. As a bonus, as part of this it also shows how to wrap a Sage 300 screen, so that it handles version updates seamlessly and doesn’t require you to re-compile your solution when we release a new version of the base screen. As a result this mechanism requires you use VB to wrap the base control for deployment. The ideas presented here probably can be ported to other programming systems, but it may not be easy.
A sample project that wraps Order Entry is located on Google Drive here. This project will be used for most of the examples in the document, so feel free to load it up and follow along. In order to view the wrapper, simply unzip the file, and open up the CRMOEOrderUI.vbp.
Create the Wrapper
The following instructions will show the basic steps on how to create a Sage 300 UI Browser Wrapper. The wrapper can then be referenced by an ASP page. There should be a constant interaction between the UI, the wrapper, and the ASP page (ie. UI calls UI_OnUIAppOpened in the wrapper, the wrapper raises the UIWasUnLoaded event to the ASP page, and the ASP page in turn catches the event, and closes the window containing the wrapper (and attached Accpac UI).
1. Open up Visual Basic and select a new Active X Control. Click Open.
2. Go to Project/ References, and select ACCPAC COM API Object, ACCPAC Data Source Control, ACCPAC Signon Manager, VB IObjectSafety Interface, ACCPAC Application Installer, and ACCPAC Session Manager.
3. The project name determines the name of the wrapper (OCX). In this case, the wrapper name will be “eCRMOEOrderUI”.
4. The name that you give the UserControl should be descriptive of what is contained on it. In this case, give the UserControl the same name as the Accpac UI that is wrapped (in this case, OEOrderUI).
5. When you are coding refer to the Accpac UI as “UserControl” (ie. UserControl.Width, UserControl.height).
6. We use the VBControlExtender to wrap the Order Entry OCX control dynamically when UserControl_Show is called (see code for UserControl_Show accompanied with this document). When referencing elements and methods within the Order Entry OCX control you would use ctlDynamic.object. The control is installed and opened using the AccpacOcxRegHelper.CLS which makes entries in to the registry that allows the VBControlExtender to reference the control by name as opposed to CLSID which is returned from Roto.
7. Now you are ready to begin writing the code that will catch the events thrown by the Accpac UI, and raise your own events to the ASP that will contain your wrapper.
8. Go into your code view and begin instantiating your events, objects, and variables.
9. Begin by declaring your objects that are going to handle events thrown by the AccpacDataSource controls in the related Accpac OCX controls. In this case, event handlers of the AccpacOE1100 class are being declared so that they can detect the events thrown by the class.
10. Next, declare the events that you will want to raise to the ASP page.
11. Declare your public variables
12. Declare your remaining variables. In this case, mSignonMgr is going to be used to sign on the Accpac UI with the signon manager so that the signon screen does not keep popping up every time that the UI is loaded. mlSignonID is going to be the signon ID.
13. Outline your functions that will be called by the ASP Page. In this case, the ASP page will give the values that are to be used to populate the UI, or to insert the customer ID into the UI’s customer field for a new customer quote.
14. Next, list out the events that can be called by the UI AccpacDataSources. In the screenshot below, you can see that the wrapper is checking the eReason variable being passed, and depending on what eReason is being passed, a different event will be raised to the ASP page (AddNew, Delete etc) in the RaiseEventEX sub.
15. Other functions are also called by the Accpac UI. The wrapper will be notified of these events through ctlDynamic_ObjectEvent (see below). Once the UI has opened ctlDynamic_ObjectEvent is called with an event name of “OnUIAppOpened” and a private sub UI_OnUIAppOpened is called and objects in the wrapper are initialized, and the UIWasLoaded event is raised to the ASP page notifying it that the UI has been opened.
16. Finally, define the Get properties that are available to the ASP page so that it can resize its windows when the UI has been loaded onto the ASP page. In this case, the ASP page will resize its windows to be the same width, height, and unit of measurement as the UI.
17. Now, you have successfully entered all the code that the wrapper will use to receive the function calls from the UI, as well as raise the events to the ASP page.
Customize the Sage CRM ASP Page
You now have a wrapped OCX now you can follow the ASP page in Sage CRM (for example, OE_OrderUI.asp as follows) to call your customized OCX.
Then it will open the OE Order Entry screen for order ORD000000000076.
In OE_OrderUI.asp file, it has following code:
eCRMOEOrderUI raises the following events:
UIWasLoaded(), UIWasUnLoaded(), AddNew(), Delete(), Update(), FieldChange(), Init(), Read(), Fetch()
eCRMOEOrderUI exposes the following Properties:
UIWidth(Read Only), UIHeight(Read Only), TwipsPerPixelX(Read Only), TwipsPerPixelX(Read Only)
eCRMOEOrderUI exposes the following Functions:
PopulateUI(OrderID As String, CustomerID As String);
CreateNewQuote(CustomerID As String);
<SCRIPT for=”eCRMOEOrderUI” Event=”UIWasLoaded()”>
var width = eCRMOEOrderUI.UIWidth / eCRMOEOrderUI.TwipsPerPixelX;
var height = eCRMOEOrderUI.UIHeight / eCRMOEOrderUI.TwipsPerPixelY;
if ((BrowserDetect.browser==”Explorer”) && (BrowserDetect.version >= 7))
width += 35;
height += 130;
width += 35;
height += 100;
var left = (screen.width – width) / 2;
var top = (screen.height – height) / 2;
width = eCRMOEOrderUI.UIWidth / eCRMOEOrderUI.TwipsPerPixelX;
height = eCRMOEOrderUI.UIHeight / eCRMOEOrderUI.TwipsPerPixelY;
BorderWidth = ClientWidth() – width;
BorderHeight = ClientHeight() – height;
bLoaded = true;
Hopefully you find this helpful in customizing Sage 300 ERP screens. Even if you don’t run them from Sage CRM, not having to re-build them for each Product Update can save you some time.
Back in January I wrote a blog on Branching by Feature. In this article I want to talk about some problems with branching by feature as well as talk about some other approaches. We’ve been doing branching by feature since a bit before the previous article was written and although some things are working out, there are definitely some disadvantages.
Becoming Overly Cautious
The idea of branching by feature is that features aren’t committed into the trunk until they are complete and fully tested. This sounds great, but it leads to some bad behaviors. People get overly cautious about merging their feature back into the trunk. People want to have a perfect merge and just keep delaying it and delaying it. Further the people using the trunk tend to resist merges and keep delaying them asking for more testing or more reviews.
This then leads to all the features being off on separate branches. But what if you are doing something that requires two of these features? You have no way to get them together. Then you have build servers continuously building all these branches, automated testing servers testing them and various other infrastructure being tied up.
Lack of Continuous Integration
To me the main drawback of branch by feature is a lack of continuous integration. Any bad interactions by the outstanding features are not found until much later. A lot of times when these problems are found, people claim it’s due to a lack of testing on the branch and that it was merged too early. But generally these problems couldn’t be discovered until the merge happens.
I tend to find that merge by feature just prolongs integration testing so long that a lot of serious problems are found so much later. Generally the longer between when a bug is introduced and when it’s discovered makes fixing it that much harder and more disruptive. If things are left too long then people have moved on to other projects and don’t like the distraction of going back.
Reducing Merge Hell
Another problem is that the longer things remain on branches, the more work it is to merge them back into the trunk. You can minimize this by continuously merging the trunk back into your branch. But then you have the overhead of continuously managing any conflicts. Plus there is a lot of room for error in this process. Every time you are resolving conflicts in a merge, you have the possibility of making a mistake and erasing someone else’s changes or introducing a bug.
This can lead to an extreme case of having to resolve hundreds of merge conflicts which always leads to errors and worse some pretty extreme conflicts between people or teams that have messed up each other’s code.
For an ERP package like Sage 300, they are composed of many modules like G/L, A/P, A/R, I/C, O/E, P/O, etc. We can source control the whole thing in one repository. This has the advantage that you get everything you need by extracting everything in the one repository. This can be quite convenient. However when you create branches when you merge them back in you do run the risk of conflicting with things that you didn’t expect. Often people just push those conflicts they don’t understand with their own changes. This usually then overwrites someone else’s work.
Currently we have everything together in one big repository, but we are going to break it up into separate repositories, one for each Accounting Module, one for System Manager, one for language translated strings, one for documentation, etc. This way we reduce the number of branches we need and we also reduce the danger of affecting things on too global a scale.
Reducing the number of branches greatly reduces the amount of complexity in the whole process. It also simplifies the process of merging features into the trunk.
This does mean there may be features that can’t be committed atomically as one transaction, it will require two commits in two separate repositories. However we feel that keeping down the scope of the commits is more important than strictly maintaining this atomicity.
You could do your branches at a lower level in the source code tree, but then if you find you need something elsewhere, it’s a fair bit of work to re-branch. This generally leads to people just including everything in their branch which then leads to all the merge problems.
Merge to Trunk Quicker
We are also working to get the different groups and teams to merge features back into trunk much quicker. We make it clear that we do expect to find problems, but that we want to find these earlier and have no expectation that all problems be found and fixed on feature branches. This way we can run the main full set of automated tests off of trunk and don’t need farms of automated test machines testing every branch.
Also for the consumers of these features, they will be all together quicker for people to use off the main builds. This way you won’t need to install multiple branch build to test multiple features.
As we move more and more into a cloud mode of software delivery, major releases become a thing of the past. In fact for cloud services we don’t really talk about releases anymore. We really just have a live service that is being continuously updated. To do this we need a good continuous delivery infrastructure and a reliable mechanism to develop individual features and merge them to trunk for immediate integration, full automated testing and then deployment to the live cloud service.
As we’ve been on this journey we keep tweaking our branching and build procedures to achieve this goal. Our first branching strategy was progress, but still led to a lot of problems that we are addressing with this new strategy.
If you were able to attend the Sage 300 ERP roadmap sessions at Sage Summit you would have heard that the next major release of Sage 300 ERP (named 2016 but released in 2015) will be dropping support for Pervasive.SQL and Oracle as database servers. This means the only supported database will be Microsoft SQL Server. Although we will support several versions of SQL Server long with the Azure SQL flavor.
The intent of this article is to help make sure everyone has plenty of advanced warning about this change. To help explain the rationale behind this decision, and to help people formulate migration plan if you aren’t already running SQL Server.
The first Windows version of Sage 300 ERP (then called CA-Accpac/2000) was released supporting one database which was good old Btrieve 6.15. We all have fond memories of those days when the world was much simpler, we just needed a simple robust database manager without any other real concerns. At that time we had a good bundling deal with Btrieve so we could include a database engine with every System Manager. At that time Btrieve was owned by Novell. At that point in time Btrieve was a good low cost database manager that supported transactioning, it was used by many ERPs, and was relatively easy to install and administer. Novell sold off Btrieve back to its original developers and that evolved into Pervasive.SQL and last year that was acquired by Actian.
Pervasive.SQL still has the same qualities that Btrieve 6.15 had, but it hasn’t really kept up with its competitors. SQL Server now has a free edition and Microsoft is much more favorable to doing bundling deals. Plus there are now many better low cost database alternatives such as SQLLite and MySQL.
Over that past years the higher end databases have become much easier to install and manage. Long gone are all the configurable parameters that would plague SQL Server installations (i.e. the defaults now work for most cases). So now Pervasive.SQL isn’t as easy to use.
Anyway Btrieve was the first database that Sage 300 ERP supported, and I think a lot of people have fond memories of Btrieve, but today it doesn’t seem to have a place anymore.
A lot of Sage 300 ERP installations require integrations to many other products, and nearly none of these support Pervasive.SQL. Hence if you want integration with BI tools, or other ERP related software, you are almost always forced to use SQL Server anyway. In the early days of Sage 300, SQL Server was very expensive and most products supported Btrieve as a low cost alternative, but today the need for that has disappeared and we are one of the last vendors to still be supporting Pervasive.SQL.
We’ve had Oracle support for a while now. However the sales numbers have never really justified the resources required to support this platform. Oracle tends to be the database of choice for larger companies that tend to be bigger than Sage 300’s target market. We’ve made a few government and large company sales because we support Oracle, but generally these customers would have been as well served by SQL Server.
Our perspective is that the demand for Oracle has waned and that they are really pursuing larger and larger companies and moving further and further away from our market segment.
Multiple Product Integrations
Most Sage 300 ERP sites these days involve multiple products working together such as Sage CRM and Sage HRMS. Generally people only want to work with one database system and the common one across the various products is SQL Server. Most products support a choice of databases, like Sage CRM supports Oracle and SQL Server and Sage HRMS supports FoxPro and SQL Server. To get a more uniform experience across all these products really only works well if you choose SQL Server. It’s generally nicer to have just one set up database operations for things like backup.
Further when you start to use more advanced cross product reporting tools, these can only do their job if all the products are based on the same database engine (so that SQL joins can work properly, etc.).
The Sage 300 ERP architecture is still the same and supports multiple databases, whether we support another database than SQL Server in the future will depend on the future of the database market. It might be a lighter weight SQL engine like SQLLite is best. Or one of the new NoSQL databases that are becoming popular like HBase or MongoDB. Certainly the NoSQL databases support capabilities that SQL Server can only dream of. Similarly products like SQLLite also run on all the mobile an alternate operating systems opening up all sorts of other possibilities. Chances are these will be introduced in a hybrid manner combined with SQL Server rather than as solutions that handle 100% of our system’s needs.
For the short term we will be concentrating on SQL Server which means can use some features that are more specific to SQL Server. However one of our keys to success has been sticking to the core SQL engine functionality. That we work fine with SQL Express and Azure SQL (unlike a number of competitors). So we will be careful to ensure anything we do doesn’t break our database independence or our flexibility in supporting all flavors of SQL Server.
Moving to SQL
If you are running an unsupported database and want to move to Sage 300 ERP 2016 then you will need to convert the database. To convert from an unsupported database like Pervasive.SQL, DB2 or Oracle, you need to run Database Dump on your databases, create SQL databases for these in SQL Management Studio, create the entries in Database Setup and then run Database Load. Make sure that you update and test your backup/restore and disaster recovery plans to ensure that you are still protected.
The conversion must be done before upgrading, since the 2016 version doesn’t include the unsupported database drivers and can’t access these databases and hence can’t do a Database Dump on them.
If you leave Pervasive, DB2 or Oracle databases in Database Setup then these won’t show up in any sign on dialogs. We’ve changed the message when you run the desktop, so that if you don’t have any databases because they are unsupported, why this is the case and to let you run Database Setup.
If you don’t want to switch to SQL Server, it just means the last version you can upgrade to is Sage 300 ERP 2014. This will be supported for its normal lifecycle. When it goes out of support, of course your software will still operate. But you won’t be able to get any new Service Packs or Hotfixes. This should present a quite large window on when to switch. These days, nearly all new sales are SQL Server and the number of SQL installs is the largest share and of course every one already running SQL Server won’t be affected.
The database world is changing and Sage 300 ERP needs to change with it. That’s why we are making these changes. We hope that converting your Pervasive or Oracle databases to SQL Server won’t be too painful and that you will get quite a few long term benefits from this move.
A new flavor of Sage Intelligence was demonstrated by Himanshu Palsule during his keynote at Sage Summit called Sage Intelligence Go! (SIG). I thought I’d spend a bit of time this week providing a few more details about what it is and where it fits into the scheme of things.
Sage Intelligence is a business intelligence/reporting tool used by many Sage ERP products. For those who have been around the Sage 300 ERP community, they will recognize it as Alchemex which Sage purchased a few years ago. This is a product that runs as a Microsoft Excel add-in which extracts data from the ERP to be manipulated by the power of the Excel spreadsheet engine. This is a very popular way of doing Financial Reporting since you often want to present the data graphically or you want to perform complex calculations or you want to slice and dice the data using pivot tables. Excel provides a great platform for performing these tasks. The original Financial Reporter bundled with Sage 300 ERP works in a similar manner as an Excel Add-in. Sage Intelligence is used for quite a few things beyond Financial Accounting including Sales Analysis and other predictive type BI functions.
Sage Intelligence has been around for a while and is a good mature product. Since it is written with the full Excel add-in SDK, it must run in Excel and specifically the locally installed version of Excel. This means it isn’t particularly well suited for cloud applications such as Office Online.
Microsoft has now released a newer SDK for developing Office apps. This new SDK is designed to allow you to develop applications that can still run in the locally installed full Microsoft Excel, but they can also run inside the cloud/web based Excel Online as well as the new specialty versions of Office like the one for the iPad.
As Sage ERP applications move to the cloud, you would want to have your Financial Reporter be cloud based as well. You would want to be able to edit Financial Reports in the browser as well as display and interact with reports on any device including your tablet or smart phone. This is where Sage Intelligence Go! comes in. It is written in the new Office Apps SDK and will run on the online and device version s of Excel as well as the full locally installed Excel. This you don’t need a Windows PC at all to use your cloud based ERP and cloud based Financial Reporter. However you still have all the power of Excel helping you visualize and manipulate your reports.
Sage One Accounting
Sage One Accounting by Sage Pastel is such a cloud based ERP system. This product has Sage Intelligence Go! as an option for performing various reporting needs. When SIG started out here, it was much simpler. The original Office Apps SDK was very simple and didn’t nearly have the power of the various older SDKs supported by local Excel. However as time as gone by, the Office App SDK has become much stronger and the functionality is now much more in line with what we expect of such a solution.
For those of you who attended Sage Summit and saw James Whittaker’s keynote, they would have seen the importance Microsoft is placing on the new Office Apps. Basically to switch from using the browser and search, or using Apps from a mobile device App store, you will get your data directly in your business applications via this new sort of App. If you go to the Office App Store, and search on Sage you will find a Sage One app for Outlook and Sage Intelligence Go! If you search a bit more broadly, you won’t find that many non-trivial applications. Sage Intelligence Go! is probably the most sophisticated Office App in the store.
Notice that you don’t run SIG reports from the ERP application, they run from Excel. Once you have the XLSX file, you can just run it anytime from any version of Excel and have full access to your data. This is really a new paradigm for doing reporting. For SIG this works really well. Whether other applications fit this model is yet to be seen.
SIG needs to communicate both with the Office API (whether cloud or local) as well as with the ERP to get the data to process. Both of these functions are accomplished via RESTful web services. The communication with the ERP must be via Web Services since these could originate from the SIG Office App running in the Microsoft Office Online Cloud or from the Office App running in the local Excel. It isn’t possible to use the traditional methods of database integration like ODBC when the two may be running in completely different locations.
Basically the Sage Cloud based ERP exposes a standard set of Web Services that are protected by Sage ID. When the user starts SIG, they get a Sage ID login prompt from with-in Excel and then this is transmitted through to the Sage Cloud ERP which uses the Sage ID to look up which tenants this user runs as. This information is related back to SIG in case it needs a second prompt to act which tenant (company) to access.
Sage Intelligence Go consists of a server component that runs in the cloud as well as the Office App that runs in Excel. The Server portion of SIG uses the provided Web Services to load the data to be reported on from the ERP into its in-memory database for processing. This in-memory database is maintained in the cloud where the SIG service runs. Then the Office App part of SIG interacts with the server side to present the correct required data and perform various processing functions.
This solution requires the ERP provide Web Services that are exposed to the Internet in general, so that it can be access via Excel running anywhere whether installed locally, running on an iPad or running as a Web Application in the cloud. This means these Web Services need to be secure and available 24×7 with very good availability. For this reason you will see SIG first integrating to Sage Cloud based ERPs (like the current Sage One Accounting by Sage Pastel) and later via the Sage Data Cloud which will offer these services on behalf of on-premised installed ERP systems.
Sage Intelligence Go! is our solution for cloud reporting. Its primary purpose is to provide Financial Reporting capabilities, but it is also capable of handling other BI type reporting needs. This is our solution to moving a lot of reporting functions into the cloud.
Sage ERP Online Services (SEOS) was a system developed alongside the online/cloud versions of Sage 200 ERP (UK) and Sage Morano ERP (Spain). The purpose of this was to provide a number of common sets of functionalities for all Sage online applications such as creating new tenants and users, managing audit logs, managing cloud credentials and managing cloud resources. The system also handled common functions such as integration to the various Sage billing systems. Although SEOS was originally developed as part of the Sage 200 and Sage Morano online projects, it was always intended to be a general tool used by all Sage cloud products. SEOS has been in successful production with both Sage 200 and Sage Morano for some time now, it is now being adopted by various other Sage cloud products both in Europe and in North America.
A lot of what SEOS does is behind the scenes and customers and partners generally don’t see it as a separate service. But I thought it might be of interest to people to know what is or will be going on behind the scenes in our various cloud products. This article is just talking about SEOS in general and doesn’t make any claims as to which will be the next product to adopt SEOS or any timelines for such adoption. But for people using Sage 200 or Sage Morano, they probably already have some experience using the SEOS portal.
One of the key goals of SEOS is to automate everything so there are no manual processes in creating new tenants or users, but additionally it orchestrates disaster recovery, backup/restore and the auto-scaling of resources.
SEOS has a number of different ways to interact with it. There is a web portal for DevOps/IS type people who run the system, there is a web portal for customers and partners and then there are a number of APIs for the various ERPs to communicate with. Some of the main components are shown below.
The main job an ERP has to do to integrate to SEOS is provide a provisioning engine. This engine runs all the time (actually at least two of them for high availability). This engine checks the SEOS job queue to see if it has anything to do. Tasks it might be asked to perform include creating a new tenant, creating a new user or suspending a tenant.
Bootstrapping the System
To create a new cloud instance of an ERP that is integrated with SEOS just requires the provisioning engine running. This is either setup manually or via some PowerShell scripts. Once the provisioning engine is going, everything else is done in the provisioning engine as it gets messages from SEOS. When it gets its first create tenant message, it will see that no application servers are running and create a couple (for high availability), it will create the database and do anything else that is needed for that tenant to be able to operate. For the first tenant that could mean creating quite a few storage, database, VM or other resources.
Then as tenants are added, the provisioning engine will be monitoring (with the help of SEOS) the usage of the system and will start additional resources (like application servers), if it determines that the system needs to scale to handle the additional load.
Thus nearly all the work of creating a large cloud system, possibly consisting of hundreds of VMs will all be brought about automatically via the SEOS/provisioning engine system.
This then helps with disaster recover, since when SEOS switches to the alternate data center, it handles switching the databases and basically bootstraps the whole process the same way. Note that databases are a bit different since they are already being replicated to the alternate site and you just want to switch the backup to primary and then use that.
When the provisioning engine requires a new resource like an Azure SQL database or a new Web Role (VM), it doesn’t just go to the cloud API to get it (say using the Azure SDK). Instead it asks SEOS for the resource and SEOS creates it for the ERP. This way the ERP isn’t using the native cloud API, instead it just uses the SEOS API. This then opens up the possibility for hosting these cloud ERPs in different clouds.
Currently all the SMB ERPs are hosted in the Microsoft Azure cloud because we get very good pricing on this and it meets our needs extremely well. However we don’t want to put all our eggs in one basket and if conditions change dramatically, we can much more easily switch other providers. There are other reasons we may need to do this, for instance Azure doesn’t have a data center in Africa and we have a lot of customers in Africa, so we may need a provider closer than Singapore.
DevOps is the group that runs all the various Sage Cloud offerings (its official name varies from region to region, but the idea is the same). Having DevOps manage dozens of cloud products all working different ways with different maintenance procedures would be a huge challenge. SEOS brings all these aspects together into one set of procedures.
Take logging for example. It’s easy for any application to generate huge log files for diagnostic and auditing purposes. There are lots of good APIs for doing this. But managing these logs is a challenge. The logs need to be archived for future reference. Generally there are several types of logs, like the Windows Event Log, and application diagnostic log and an application security log. All these need to be kept in a central spot including backing them up. There has to be easy ways to search them, say by tenant, event type or date time. Many people use Elastic Search to search their logs. SEOS provides a uniform way for managing these and automates most of the process. This way DevOps only needs to know one way of managing logs and not a separate way for each ERP. Plus SEOS automates the whole process, avoiding manual procedures and mistakes.
Sage is a large company and operates around the world. Most of our products are charged for and the backend systems that do billing vary around the world. Various geographic regions have extra regulatory requirements and different business rules. SEOS handles providing usage data to the billing engine via a standard adapter. Each region has to write an adapter to interface to SEOS. Removing this burden of interfacing to the Sage billing systems is a huge benefit to the ERP teams who really don’t want to have to deal with these details.
If you are a partner or customer using Sage SMB Cloud offerings in Europe, then you have probably already seen SEOS and know something about it. If you are using Sage SMB Cloud offerings in other parts of the world, then you will probably start to see SEOS appearing over the next while. These will probably appear more as features or services, but these are being brought to you by the SEOS global initiative.