Stephen Smith's Blog

Introduction

I’ve previously blogged on the enhancements for the framework for creating custom SData feeds for applications here and here. In this posting I’m looking at enhancements to our core SData protocol support. We’ve been working hard to add more features and correct some inconsistencies and bugs.

The main purpose of this exercise is to make SData work better for integrators and to make sure the Sage 300 ERP SData feeds work well with other Sage tools like the Sage CRM dashboard and the Sage Argos Mobile Framework.

Global Schema

Generally SData features are mostly of interest to programmers. However some, like this one, enhance existing integrations between different products. Global schema is a mechanism to return all the SData meta-data for a dataset (company) in a single call. In version 6.0A, you could only get the metadata for one SData resource per call. Rather esoteric. But having this enhances our integration to the Sage CRM SData dashboard. Previously when you created an SData widget pointing to an Sage 300 ERP SData feed you needed to specify the $schema for a specific feed, something like:

http://sage300erpdemo.na.sage.com/SDataServlet/sdata/sageERP/accpac/SAMINC/arCustomersFinder/$schema

Now you can give the $schema to the company using an URL like:

http://sage300erpdemo.na.sage.com/SDataServlet/sdata/sageERP/accpac/SAMINC/$schema

Which means you don’t need to know the resource component of the URL. In Sage CRM it looks like this, first you give the URL to the global schema:

Then you get a list of SData resources to pick from in a more human readable form:

Previously you only got the feed you specified. Then you select a feed and hit next to choose the fields you want from that feed.

SData Validation Tool

Sage has written a tool that will validate the correctness of SData feeds. This tool is available here (you need to scroll down to near the bottom of the page). The intent of this tool is for anyone, whether internal or external to Sage to be able to validate any Rest web services for SData compliance against what is described in the spec at the SData Website. This tool was around in the 6.0A days, but it needed work. Back then 6.0A passed the feed validator. With the new tool 6.0A has a lot of problems reported against it. With 2012, quite a bit of work went into making our feed compliant. Which means you can expect them to work as the specification states and integrations with other SData aware products and tools becomes much easier. This tool is continuously being updated and will probably auto-update itself as soon as you install it. Below is a screenshot. Hopefully by release a few of the remaining errors will have been corrected.

Argos

Argos is a framework for creating mobile SData clients using HTML5, JavaScript and CSS. This was originally developed by the Sage SalesLogix group to create the mobile interface for the Sage SalesLogix Mobile product. However since SalesLogix uses SData as its Web Services interface, this library was created entirely on SData. As a consequence it can be used with any product that supports SData.

As part of our Sage 300 ERP 2012 development we tested Argos on our SData feeds and produced a sample mobile application.

As part of this development we fixed a couple of bugs and made sure our SData support works well with the Argos SDK. I’ll write a future blog posting on more details on the Argos SDK and how to write mobile applications for Sage 300 ERP. However if you are interested in Argos, you can check it out, since the whole project is open source and available on github:

• SDataJavaScriptClientLib: https://github.com/sage/SDataJavaScriptClientLib
• argos-sdk: https://github.com/sage/argos-sdk
• argos-saleslogix: https://github.com/sagesaleslogix/argos-saleslogix
• argos-sample: https://github.com/sagesaleslogix/argos-sample

E-Tags

We finished implementing e-tags with this version. These allow proper multi-user control when you have multiple sources updating records. Basically when you read a record, it returns an e-tag which has the date and time the record was last modified. When you update the record this e-tag is included with the update message and then the server can see if the record has been modified by someone else since you read it. This detects the multi-user conflict. Sometimes the server will just merge the changes silently and everyone is happy, sometimes the server will need to send back the dreaded “Record has been Modified by Another User” error response.

Using Detail Fields in Select Clauses

In 6.0A, if you read an SData feed with header and detail components (like O/E Orders), then you got back the header fields and links to the details. Even if you specified detail fields in a select clause. This meant if you wanted both the header and detail lines you needed to make two SData calls. Further this was annoying because it meant the format that you get back reading records is different than how you write the records, so you would need to combine the separate results from the header and details back  together to do an update or insert. Now if you specify detail fields in the select clause you will get back all specified fields in the XML package, which will likely be a header with multiple details all returned for the same call. This saves an SData call, but further it’s much easier to deal with, since now you have an already correct XML template for manipulating to do future inserts and updates.

Define Your Own Contracts and Have Your Own Classmaps

In version 6.0A, the only contract we supported for SData feeds created by Sage 300 ERP was the accpac contract. Now in the classmap file you can specify the contract your feeds belong to. This has always been in the classmap files, it just didn’t work. This means you can ensure any feeds you define yourself won’t conflict with anyone else’s.

Another problem in 6.0A was that to create your own feeds, you either needed to be an activated SDK application with your own program id, or you needed to edit one of our existing classmap files. This was annoying since your changes could well be wiped out by a product update. Now you can copy your own classmap file into an existing application (like O/E), you just need to name it classmap_yourownname.xml and it will be added to the defined SData feeds.

Further all the feeds that make up the accpac contract were oriented to the Orion UI work. They weren’t necessarily a good fit for people doing general integration work. So we are creating a new contract which will have the main master files and documents in it that is oriented towards integration work and stateless operation.

Summary

SData continues to be a foundation technology that we are building on. Quite a lot of work has gone into improving our SData support in Sage 300 ERP for the forthcoming 2012 release. This will allow us to leverage many other technologies like Argos to accelerate development.

If you are interested in learning more about SData check out my learning SData videos which are located here and which I blogged about here.

Introduction

A couple of weeks ago I blogged on Learning over the Web, in this blog I mentioned that I really like the Khan Academy and their video method of training. I’ve now started experimenting with making Khan Academy style videos. I’ve now done three, so far, as an introduction to SData. I plan to make more of these going forwards. Once I have a larger set of videos on SData, I may try branching out to other topics. Below is a picture of Sal Khan working on such videos:

New Video Page

I’ve added a Video page to my blog which will provide links to all the videos I produce. To start with there are three videos, which aren’t very many. However I hope to make a new one every week or so and then if I can keep that up, after a year there will be fifty or so videos. The first three videos are:

Introduction to SData
How to Practice with SData
SData Queries

The best way to learn something is by doing. So I recommend playing with SData and experimenting with the various items described in the video. To this end you can play with a locally installed version of Sage 300 ERP (or another Sage product) or you can access our demo server at http://sage300erpdemo.na.sage.com. The user id and password are ADMIN/ADMIN, make sure you enter them in upper case if prompted from the Browser or other client software. If you type the URL: http://sage300erpdemo.na.sage.com/SDataServlet/sdata/sageERP/accpac/SAMINC/arCustomersFinder into the Chrome browser and enter ADMIN/ADMIN for the userid/password then you should get back a large amount of XML containing the first 10 customer records in the SAMINC database. For information on how to perform other querying, see the third video.

If you want to try these with a different Sage product, then you might need to run Fiddler to see the exact form of their SData URLs. Once you have this, you can be up and running. Fiddler is a very useful tool for spying on HTTP requests made from your computer. You can spy on any program or website to see what it is doing.

For more background information on SData, see the SData Website or some of my previous blog posts: SData in Sage ERP Accpac 6, More on SData and Sage ERP Accpac 6, Stateful SData, On the Sage GCRM Contract, Fun with SData, A Roadmap for SData Integration to Sage Products or Defining SData Feeds for Sage 300 ERP. Jarett Smith has also started a blog on SData which is well worth checking out.

Production

I find creating videos more time consuming than writing, mostly because it’s harder to jump around in videos and harder to edit them. I’m hoping I can get better at creating videos with more practice and time. Partly getting used to the process and learning by doing. I hope that as I keep doing these, they will get better. It certainly takes some practice to use the writing tablet for drawing (hopefully my handwriting will improve) and at the same time I need to watch myself to not say “Um” so much. So I consider these first three videos the first three steps on a longer journey.

For producing the videos I pretty much copied what they use at Khan Academy. It’s neat that you can create videos these days with very little equipment or post production software. I used entirely either open source or free software and a very inexpensive writing tablet. The items I used:

• YouTube to post the videos to. Seemed the easiest and the URLs are easy to circulate.
• SmoothDraw 3 for drawing. I start with a black rectangle 854×480 pixels (which is a preferred YouTube resolution that fits well on my monitor).
• BB FlashBack Express for screen recording.
• Wacom Bamboo Connect pen input tablet. Cost around $80CAD.
• A Blue Snowball Microphone. This works a lot better than the microphone built into my laptop and we already have a few of these around the office. Original cost was about $65CAD.
• Windows Live Movie Maker for video editing. Not a very good solution, but it has done what I need. Suspect I might need to buy Camtasia eventually.

It took me a bit of trial and error to get things to work right. I tried a couple of free screen recording utilities like Camstudio, which didn’t work for me. They either crashed or didn’t produce good results. Then in editing, for MovieMaker you need to change the project from 4×3 to 16×9 or it produces something that doesn’t work right on YouTube.

Generally handling video files is a bit of a pain since they are so large. Uploading from work is ok. Uploading from home is very slow, I suspect because cable modem is optimized for downloading content rather than uploading it. Either that or Shaw decided that uploading videos is a no no and throttled my connection.

I’m still undecided on whether I want to add vlogging to my blogging. This requires a camera, but web cams are cheap and for that matter both my phone and camera both take really good videos, certainly good enough for YouTube. When I’ve tried this in the past, I haven’t been happy with the results and found that much more video editing is required. But then again hopefully with some practice, I can get better over time.

Summary

I hope you find my new Video page useful. Hopefully over the coming month I’ll add quite a few videos and start to branch out to other topics.

Introduction

We introduced SData support with Sage ERP Accpac 6.0A; however, the product as it shipped only defined a few SData feeds that it needed to support the new Web Portal, Data Snapshots, Inquiry Tool and Quotes to Orders features. But Sage 300’s support for SData is based on converting SData Web Service requests into View calls. So it is possible to expose any View (or collection of composed Views) as an SData feed.

In a future version of Sage 300 ERP we will expose all the relevant Views via SData, but in the meantime if you want to use SData with Sage 300, then you need to provide XML files to define the feeds you need.

All the feed definitions are XML files, which means you can read all the existing ones that come with Sage 300 using a normal text editor. Hence you can use the existing ones either as examples or templates for the definitions you need.

One thing to be careful of is that most of the fields in these XML files are case sensitive. This means they must match exactly or they won’t work. When things don’t work, it’s worth looking in the Tomcat\log folder at the relevant SDataServlet.log as this will often point out errors when parsing the XML files.

Class Map Files

The classmap files are a series of XML files located in the sub-folders under: C:\Program Files (x86)\Common Files\Sage\Sage Accpac\Tomcat6\portal\sageERP. The feeds for a given application are stored under the application’s program id and version id directory such as oe60a. Note that these need to be in a folder for an activated application to be read, but within an application you can define feeds that access Views in any application.

All the configuration XML files are loaded into memory by the SDataServlet on startup. So if make any changes to these files, you need to restart the “Sage Accpac Tomcat6” service for your changes to take effect.

You can use these to define custom Java classes to process the SData requests, I’ve covered this a bit in other blog postings, but won’t go into that here, since this article is only considering what can be done by editing XML files.

The classmap defines each SData feed and specifies the class to handle the feed and then a detailed feed definition file called a resourceMap file.

Example – Currency Codes

The currencyCodes resource is implemented by the Java class: ViewResourceKind and is defined by the resource map file: currencyCodeViewMapping.xml. ViewResourceKind is a system provided Java Class for generically converting SData requests into View calls. You can use this to expose most Views (that have data) as SData feeds.

Classmap.xml

<classmap>
<contracts>
<contract name=”accpac”>
<resource name=”currencyCodes” className=”com.sage.orion.sdata.servlet.accpac.ViewResourceKind>
<parameters>
<parameter name=”ResourceMapFile“ value=”currencyCodeViewMapping.xml”/>
</parameters>
</resource>
</contract/>
</contracts>
</classmap>

If you aren’t programming server classes then this is all you need to know about the classmap files.

Resource Map File

Maps an SData resource to a backing family of Accpac Views and fields. These are stored in the resourceMap folder under the folder that holds the classmap file.

By default all fields from the view are exposed as SData Resource Elements.

Has the ability to exclude or include or overwrite Sage 300 fields from the SData resource.

Example – Currency Codes

The currencyCodeViewMapping.xml resource map file contains the following:

<resource name=”currencycode” description=”Currency Codes”>
<viewID>CS0003</viewID>
<pluralName>currencycodes</pluralName>

<includedFields>
<resourceViewField viewFieldName=”CURID” />
<resourceViewField viewFieldName=”CURNAME” name=”Description”/>
<resourceViewField viewFieldName=”DECIMALS” />
<resourceViewField viewFieldName=”SYMBOLPOS” />
<resourceViewField viewFieldName=”THOUSSEP” />
<resourceViewField viewFieldName=”DECSEP” />
<resourceViewField viewFieldName=”NEGDISP” />
<resourceViewField viewFieldName=”SYMBOL” />
</includedFields>
</resource>

The key points are the viewID that maps this feed to the currency codes view CS0003. The URI of the feed is the plural name, namely currencyCodes. Then you can specify the list of fields you want included in the feed. You might specify a shorter list of fields to keep the size of the feed to a minimum. The includedFields section is optional.  I tend to prefer using an excludedFields section to just list the fields I don’t want.

Example – Single Level Resource

SData resource “customer” is defined from the view AR0024.

<resource name=”customer” description=”AR Customers”>
<viewID>AR0024</viewID>
<pluralName>customers</pluralName>
</resource>

Example -  Multi-Level Resource

SData resource ‘arInvoiceBatch’ is defined from a set of composed views – AR0031, AR0032, AR0033, AR0041 and AR0034.

<resource name=”arInvoiceBatch” description=”AR Batches”>
<viewID>AR0031</viewID>
<pluralName>arInvoiceBatches</pluralName>
<resources>
<resource name=”invoice” description=”AR Invoice”>
<kind>arInvoice</kind>
<viewID>AR0032</viewID>
<resources>
<resource name=”detail” description=”AR Invoice Details”>
<kind>arInvoiceDetail</kind>
<viewID>AR0033</viewID>
<resources>
<resource name=”optional” description=”AR Invoice Detail Optional Fields”>
<kind>arInvoiceDetailOptional</kind>
<viewID>AR0401</viewID>
</resource>
</resources>
</resource>
<resource name=”schedule” description=”AR Invoice Payment Schedules”>
<kind>arInvoiceSchedule</kind>
<viewID>AR0034</viewID>
</resource>
</resources>
</resource>
</resources>
</resource>

Resource Map File Details

Resource Mapping File Attributes / Elements:

name: name of the resource

description: description of the resource.

viewID: The ViewID of the resource. Remember you can get further information on the Views from the Sage 300 ERP Application Object Model (AOM).

pluralName: plural name of the resource.  If undefined then the pluralName = name +”s”.  This will be the URI of the resource.

resources: Collection of sub-resource elements

kind: resource kind name of the resource. Top resource of the resource tree the resource name and the resource kind name should be the same. However for a sub-resource the resource name reflects the name of the property that refers to it in the parent while the kind name is the name that it appears as at the top level of the schema.

includedFields: list of resourceViewFields that are to be included in the resource (by default all fields are included)

excludedFields: list of resourceViewFields that are not to be included in the resource

overridenFields: list of resourceViewFields that are to be overriden in the resource (usually done to change the SData name)

virtualFields: list of resourceViewFields that are to be added to the resource. Note: virtual fields requires extending ViewResourceKind with a custom class that implements these virtual fields.

lookupFields: These are a 6.1A feature that allow you to add fields looked up from another view like to get a description.

Summary

Hopefully this article gives an idea of how to setup additional SData feeds for Sage 300 ERP, without requiring any programming.

Introduction

I’ve blogged a few times about using SData to feed data into Sage products and about the general ideas behind SData. In this blog posting I want to look at some benefits to integrating into Sage products using SData and the general reasons behind doing so. A lot of this article is forward looking and so each Sage product is somewhere along this journey and no product has reached the end yet.

Imagine that you are an Independent Software Vendor (ISV) and that you have a great product that satisfies a useful specialized business need for many customers. However it requires some data that is typically stored in an ERP system and generates some financial results that need to be entered into this ERP. Say you need to read all the Orders from the system and then you produce a number of G/L entries that need to be imported. Wouldn’t it be great if you could automate this process by being notified automatically whenever an Order is entered and being able to write the G/L entries directly into the ERP? However, you find that your customers are running all sorts of different ERP systems. As you look into doing these integrations you find that you need to join an SDK program for each ERP and have someone learn and program in all these different ERP’s SDKs and program you integration in all sorts of strange languages from VB to C# to JavaScript to FoxPro. Then you have to maintain all these integrations from version to version as your product progresses and all these ERPs evolve. This is a lot of work. There must be an easier way.

You could take the attitude that you will provide an API and SDK for your product and then, since your product is so wonderful, that all the ERP vendors will write an integration to your product using your SDK. After a few years of waiting for this to happen, you’ll probably give up on this.

Sage has many ERP systems and combined has a very large market share. However all these ERPs have their own heritage and their own SDK. How can Sage make it so this ISV can create their integration easily to a large set of Sage products using their own development tools and programming knowledge?

This is where SData comes in. SData is a REST based Web Service interface. SData is based on internet standards and easy to call from any programming system that supports Web Service calls. SData is a standard being implemented in all Sage Business Solutions. So how will this work in the scenario above?

Method One – Poll for Orders

In this case the ISV application will periodically poll for new Orders on the Orders SData feed, process them and then write any G/L entries to the G/L SData feed.

This method isn’t that different than using the ERP’s built in API in whatever technology that is implemented, perhaps COM, .Net or Java. So at least now for the various ERPs we are using only one API technology to access several ERPs.

Standard Contracts

Generally each ERP has its own heritage and hence its own database schema. Most Sage ERP’s will expose their built in native schema as their native SData schema. So although you can use SData to access a range of different ERPs, you still need to handle multiple schemas to set and return in the XML payloads associated with the calls.

The solution for this is standard contracts. These are specifications for standard SData calls that define a common schema. It is then the job of each Sage ERP to transform their native schema into the schema of the standard contract as part of their job of offering standard contract feeds.

With standard contracts implemented the ISV can then access multiple Sage ERP’s using the same API technology, namely SData, and using the same payload schemas, essentially making the integrations the same.

Method Two – Connected Services

Connected services refer to integrating products whether from Sage or an ISV being located in the cloud and then integrated to an on-premise installed ERP.

Currently there is a concern that on-premise customers don’t want to expose their ERP system to the Internet, meaning they don’t want to manage a firewall and worry about the security concerns associated with an Internet facing web server. This means they do not want to expose their ERP’s SData feeds to the Internet at large. So how does the ISV integrate if they can’t call the ERP’s SData feeds?

Although the cloud application can’t call the on-premise application, the on-premise application has no problem calling the cloud application. This means it’s up to the on-premise application to push all necessary data to the cloud application. Generally this is handled by a synchronization engine that synchronizes some ERP data up to the cloud. The SData specification contains a data synchronization specification for this purpose. The synchronization algorithm is very robust and will catch up if the Internet is down, or the cloud service is unavailable.

We could also provide notifications when things happen, but this is less robust, since what happens when the Internet is down or the other application is unavailable (perhaps offline for maintenance).

Notification Configuration

The above method for connected services assumes that SData synchronization will need to be configured. The core ERP will need to configure a list of SData services to synchronize with (hopefully down the road there will be many), then the ERP will need to ask the service which feeds it wants to synchronize.

Initially this configuration will probably be done by configuration screens in the ERP. However over time it will be nice to move this all to dynamic discovery type services. This will reduce error in setup and allow connected services to evolve without requiring setup changes in all their integrated ERP systems each time something new is added.

If the ERP and ISV solution are both installed together (either on-premise or in the cloud), then it would be nice to configure some direct notifications, basically call-outs from the application when various important things need to be done. This might be for things like requiring a calculation be performed or to notify that an order’s status has changed. These sorts of things will only work when both applications are live and connected, but there are many useful scenarios for this.

Summary

Initial SData implementations are either shipping or forthcoming across all the main Sage business products. We are starting to see integrations of the method one type emerging. We are actively developing connected services using the SData synchronization mechanisms. Then as we continue to roll out SData technology you should see any missing pieces start to fill in and greater uniformity emerge on how you integrate with all Sage products.

As Sage studies the market and determines useful profitable connected services, Sage can publish an SData contract that all our ERP systems will support. Then the chosen ISV can provide this service for all Sage products by supporting this SData contract. Often for connected services we need different providers in different parts of the world, and again it’s easy for Sage since for this one contract perhaps we have one provider for North America, another for Africa, another for Asia and another for Australia.

Sage has standardized on a REST based Web Services protocol called SData. This protocol is documented at http://sdata.sage.com/. As a first step this will standardize the technology that you use to communicate with Sage applications. You will be able to issue SData requests to multiple Sage products such as Accpac, Abra and SageCRM. As opposed to using a different technology to talk to each, such as COM, .Net, Java, Soap, DLL, etc. All the products will still maintain their existing APIs, but they will all add SData to their repertoire. Even if you are a reporting type application, crossing multiple Sage products can be difficult as they tend to use different databases or have very different schemas. SData is a nice way to hide these differences and to provide a single consistent programming interface.

The next step is to standardize the actual data that is exchanged via SData. Towards this end, Sage is releasing a number of “contracts” that can be used to interface to a given application. For instance the Sage Global CRM (GCRM) Contract (http://sdata.sage.com/sdatacust_GCRM.html) is a standard contract supported by all Sage CRM products including Act!, SalesLogix and SageCRM. If someone integrates to one of these products using SData and the GCRM Contract, then it will be very easy to integrate to any of the other CRM products.

The GCRM contract was created by Sage for mostly selfish reasons. Sage owns dozens of ERP packages in dozens of regions around the world. Having each of these create a point to point integration with one of our CRM packages was proving to be a huge amount of development effort. Plus the results weren’t consistent, some of the integrations were better than others. This started as an attempt to standardize what was being done repeatedly over Sage again and again. However as the work progressed we realized there are a lot of other benefits. Not only does this save Sage some development time and money but it will:

• Provide a standard interface for others to do similar integrations. For instance an ISV could integrate another CRM package to multiple Sage ERP’s by taking advantage of the GCRM contract.
• ISVs will be able to integrate to multiple Sage applications at once (usually part of a suite of products) using only 1 technology.
• The various application contracts are well documented at http://sdata.sage.com/sdatacust.html making life a bit easier.
• Using REST based web services is an efficient way to produce a nice lightly coupled integrations.

Each application will still have a native SData interface where you can get at the full functionality of that product. But these standard contract interfaces will make it far easier for ISV’s to integrate their vertical solutions to multiple Sage products.

Many applications have a component of their data that is shared and needs to be synchronized. For instance much of the data that CRM maintains for its companies is shared by the ERP package in its customer’s information. In most CRM to ERP integrations a large part of the integration is keeping this company/customer data synchronized in the two applications. Similarly for an HR application to ERP integration, typically the employee database tables need to be synchronized in the two applications. SData provides a standardized protocol to allow multiple applications to synchronize data in a uniform manner (http://sdata.sage.com/sdatasync.html).

At the core of the GCRM Contract is the specification of how to synchronize the tradingAccount (company/customer), contact, postalAddress, phoneNumber and email tables. The GCRM contract specifies the fields that make up these standard contract definitions (how CRM and ERP expose their native tables) and what the applications need to do to provide the synchronization. In this case the hard part is handled by the CRM application which hosts the synchronization engine. The ERP package just needs to provide an SData feed for each of these tables in the correct format. Then CRM will periodically ask ERP if anything has changed and if so to provide the updated records. The protocol then has the details of how to handle conflicts and how to catch up if one application was unavailable for a period of time.

The GCRM Contract really consists of three parts:

1. The base customer/company tables that need to be synchronized and the protocol to handle that.
2. A number of real time data SData feeds in a standard format to access much of the data in the CRM product.
3. A set of SData feed definitions that allow CRM to provide a basic Order Entry screen, to query pricing information from the ERP and feed orders into the ERP package.

These three parts can be used independently. For instance Accpac could use the GCRM contract synchronization contract to replace the current method and start incorporating CRM data into its own UIs that is retrieved by the real time SData feeds, but then rather than using the CRM Order Entry screen, surface the Accpac Order Entry screen inside of CRM (the quote-to-order feature: http://smist08.wordpress.com/2009/12/17/sage-erp-accpac-6-0-quote-to-orders/).

As Sage moves ahead with adopting our REST based Web Services, SData, you will be seeing more and more new technologies based on this. Generally Sage is looking to develop many SData based technologies that can be shared across all Sage products and can be utilized by all Sage development partners.

SData is the new Sage standard for Web Services. It is a REST based protocol similar to GData. Typically REST based protocols are “stateless”, meaning that any request doesn’t rely on knowledge of other requests. An SData request might be something like add an order where the order is provided as an XML payload; update a customer record where the new fields are in the XML payload. This is a very efficient way to do things. Each document entered just requires one round trip request from the client to the server.

“Stateless” processes are very desirable in large Web based applications. It means each request can be directed (load balanced) to one of a set of servers in a server farm. History doesn’t matter; each request can go to any server. If a process is “stateful” then requests from a given client have to be directed always to the same server. “Stateless” is also desirable since the server doesn’t keep any knowledge of things in progress, so no server resources are committed between calls. Also if the server is restarted, or a process on the server is restarted, besides perhaps a slight delay, the clients never know, since no info was kept in the process that was stopped.

However this typically isn’t how we enter orders from a User Interface form. From the User Interface we build up an order a piece at a time. First we input the customer number or an order template, from which we fill in a number of field defaults. Then typically we update some order header fields, and then enter a number of detail lines. As we enter things they are validated and often entering one field like a currency code or item number causes a number of other fields to be updated.

One way to do this would be to send the order to the server each time something happens and have it updated and sent back. This would keep things stateless, but would tend to become unwieldy as the order gets large. Another consideration is that we want to keep using the Accpac Business logic more or less unchanged.

The Accpac Business logic builds up the order inside the Business logic (View) as it goes along. Then when you say save, it commits everything to the database as one transaction. In this model as parts of the order are completed they get applied to the in-memory order, defaults are looked up and validations are performed. So typically an UI will issue a command to create a new order and receive back the default values and states for all header fields. Then the UI will enter the customer number or a template code, this will go to the business logic which will then validate this value and lookup the defaults, apply them to its copy of the order and pass them back to the UI. At each step the UI is really updating the server memory part of the order. The UI doesn’t need to keep all entered details in memory since it can ask the server for them at any time. When the order is saved, the UI just sends a save request (with no data) and the server commits the order from memory to the database in one database transaction.

So how do we reconcile stateless SData with stateful Accpac? Generally we try to keep as many things stateless as possible, but for some operations, mostly entering documents, we need to introduce state to SData. We add an SData call that will start “stateful” processing on a given SData resource like an order. This means then that SData will keep state on the resource, which under the hood means it will tie a View instance to the client and preserve it for the client. Then the UI rather than send the whole order, just sends the parts that the user changed (the UI has to know when to trigger this) and then SData applies these changes to the Accpac Views, and then gets all the fields that changed as a result (either in value or in state) and returns them as the return payload to the SData call. This then goes on until an SData call like a header navigate or save happens to clear the state. This means that while the Order is being compiled server resources are being used and all requests for this client have to be routed to this particular server.

In a Web Application like Facebook with say 500,000 users online at once, keeping any state on the server is incredibly expensive. Plus things need to be load balanced in the most flexible manner possible. But I think Accpac has a bit of breathing room before we reach this kind of volume. At least we are moving in the right direction, having many things stateless such as inquiries and simple file updates/inserts. We just keep state during document entry of header/detail type documents. But if Accpac does start having hundreds of thousands of users online then we will need to start evolving our business logic to become truly stateless. But for now we have a nice compromise of using our existing business logic, but becoming stateless in a lot of what we do.

SData is Sage’s new Web Services standard. It is documented in full at http://sdata.sage.com/. For Sage ERP Accpac 6.x, SData is the interface used by the browser based UI pages to communicate with the Server. Like in Accpac 5.X where the Accpac COM API was the interface used for all UI to Business Logic communication, this role is taken over by SData in 6.x. This means that there is a complete robust REST based Web Services interface to the Sage ERP Accpac application. Anything that can be done from an Accpac User Interface Web Page can also be done via any other program making SData Web Services calls.

REST based Web Services are very simple in how they are constructed, they are built using current Web Standards such as HTTP and Atom. For instance the screen shot below shows typing:

http://bcraccdemo/SDataServlet/sdata/sageERP
/accpac/SAMINC/Customers(’1200′)

into the URL field for the Chrome Browser.

This URL is an SData request, it is sent to the server bcraccdemo and the application SDataServlet running there. The rest of the URL is then processed by the Accpac SData provider to return the correct data for customer 1200 (Ronald Black) in XML Atom format, which is then displayed in the Browser window. As you can see, no special infrastructure or programming libraries were required to make this Web Service call. All that was needed was the ability to send an HTTP request and receive the response.

SData is also self describing. Below is a screen shot of asking for the schema of the XML response given above.

When doing update or insert calls you need to provide an XML payload of data similar to the response above of the data to be updated or inserted.  There is also a standard interface for doing services which in Accpac includes things like reporting, day end and posting.

The example above was for customer 1200, however if the (‘1200’) part was removed, SData would have returned all the customers in A/R. In this case you can specify a filter to say return all customers greater than some point or specify which fields to sort the returned data by. SData will also page the data returned, it will return a block of records with links to the previous and next blocks.

Basically SData is a fairly simple protocol to use that should be accessible to almost any programming system. It is rich enough to support all UI operations required by Accpac. And ISV’s can be confident that it is complete enough, since all Accpac UIs rely on it. It is standards based on HTTP, Atom and other Web Standards allowing standards based tools to be used. Generally a very powerful Web Based Interface to greatly enhance Accpac’s intergratability and extensibility. Since SData is being adopted by all Sage Business Applications, learning and using SData with Accpac will teach you how to use SData for any SData equipped application, such as Sage CRM.

One of the big features in Accpac 6 is SData support. What is SData? SData is a web services protocol based on REST. What is REST? REST is a web services protocol that is very popular among Internet companies. There are basically two main competing standards for web services. One is SOAP which is promoted by Microsoft, Oracle and IBM; this is a fairly heavy duty protocol which requires a fair bit of infrastructure (which is provided by the companies promoting this protocol). The other is REST which was invented as a University Research project and basically uses existing Web Technologies such as HTTP and RSS to implement a web services protocol. The nice thing about REST is that it doesn’t require any extra middleware. You are basically doing everything via standard web URLs. The infrastructure that supports this is basically just the standard Internet. SData is just an extension to REST, similar to Google’s GData which is also based on REST. REST protocols are used by Google, Amazon, eBay, Yahoo and all the main Internet companies. Gartner estimates that 75% of web services in use on the Internet are REST based. REST/SData also provides a standard mechanism for performing CRUD (create, read, update, delete) on all records.

From Accpac, every Accpac View (or Business Logic Object) will be exposed via SData. This provides third party ISVs a complete, efficient Web Services interface to communicate to Accpac with. Also all Sage applications are being made SData aware, meaning that all Sage applications can manipulate SData Web Services (or feeds). For instance here is some Accpac A/R Customer and G/L Account data displayed within a Sage CRM Dashboard using SData.

From a simple wizard you can select any Accpac SData feed and once you’ve chosen a feed, you can select which columns you wish to display. Additionally Sage CRM and Accpac will be converting to use SData for our integration needs, this transition will occur over several versions of both products. In the same way you will be able to access CRM data from Accpac using SData.

All the new Sage ERP Accpac 6 screens are written entirely using SData. This means the screen can access any Accpac Views via SData, but additionally they can access any SData feeds from any product. In Accpac 5.x, the screen controls were mapped via datasource controls to views and view fields. This was great, but it limited you to only connecting our standard controls to fields in Views developed in our SDK. Now that the controls are mapped to a standards based SData fields, it means they will be able to be connected easily to much more data than just Accapc data. Look for other Sage products to start providing SData interfaces in their coming versions.

Sage is publishing SData as a publicly available standard. You can read all about it at: http://sdata.sage.com/.