Archive for November 2011
The pace of development of Web based applications has been frenetic. The pace of development of tools to program the Web is also frenetic. There are new programming languages, libraries, servers and systems announced daily. Whatever happened to the good old days where you got a new version of Visual Studio and an updated C compiler every three years or so? In this blog posting, I want to look at the problems that people are trying to solve and the innovations they are trying to introduce to address them.
One nice thing about working at Sage is that we have so many products; someone will be using one of the systems mentioned in this article. For instance SageOne and Sage Billing Boss are written in Ruby on Rails. Sage 300 ERP and SageCRM use GWT extensively. Other products use JQuery and Node.JS. Several of the mobile offerings were produced with JQuery Mobile. So within Sage we have some opportunity to compare these on real non-trivial projects.
The Perfect Programming System
The real goal of all this is delivering real value to customers (end users). All other goals are in support of this. Nearly everyone is adopting some sort of Agile development process centered around delivering value (or even delights) to their customers. Once you are delivering customer value, then the next question is always: how to do that faster? Everyone are looking for a software development toolset that delivers customer delights and improves productivity.
That’s the high level goal, but what are the more detailed goals:
- Quick to enter changes, ideally just edit some code, save it and refresh the browser to see the results.
- Easy to test. Meaning there are good ways to write unit tests that can run quickly every build. That the system promotes test driven development.
- Easy to maintain. The system produces readable code that is easy for another programmer to figure out and change.
- Little impedance between problem domain and language. When thinking about what you want to do, it isn’t a huge change of perspective to write the code; somehow the code reflects what you want to do naturally.
- Easy to learn. You don’t need to know 3 programming languages, 5 scripting languages and 27 runtime libraries all of which interact in a complicated way to get up and running.
- Large ecosystem – lots of tools and libraries to save you work.
- Produces correct code. You don’t have to worry about unexpected syntax errors or other strange unexpected runtime errors that testing might have missed. By the same token you don’t have to spend a lot of time testing for and removing these sorts of things.
- Easy to install and setup. It’s easy for programmers to setup the development environment and get up and running quickly.
- Easy to deploy. It’s easy to transfer changes from the development environment to a live environment.
- Ability to integrate with (consume) modules that are part of legacy systems.
- The generated code is very small, fast and scalable.
- Great development tools for debugging, code analysis, etc.
In life, there are many tradeoffs to make. The same in programming. You might want three things, but you can only have two. Which do you pick? Which do other people pick? Usually with a large set of goals, some practical tradeoffs need to be made. Usually there will be a lot of disagreement in a group of people as to which goals are the most important and which ones can be lived without. Often the group will break up into different camps each with the conviction they have made the correct choice and theirs is the correct way to do things. So too with programming and hence all the different camps of web developers. Here we’ll look at a number of the camps and the tradeoffs they are making. Obviously I have my own opinions and preferences and the members of these various camps will probably disagree with my assessments.
For instance would you want to sacrifice several of the items for scalability? Everyone would like to be as popular as Facebook, but if you are a new web site, then perhaps it’s better to go live quicker to get feedback rather than spend the extra time engineering scalability. Often it’s more important to get feedback quicker so you can adapt your plans rather than over-engineer and be slower to market and slower to react. This fits in with the “fail fast” methodology of innovation where you are trying lots of things to see what works, you want them live as fast as possible to see what is good. Then you can take the good ideas and engineer them for the projected growth once you know what that will be.
Client Side Programming
Server Side Programming
You control the server and you control what runs on it, so you can theoretically use any programming language that can receive HTTP requests via a network socket. But usually you want something that integrates with a Web Server like IIS or Apache. There are good server frameworks to create the server programs in Java, .Net, C or any of a thousand scripting languages. Nearly every programming language has been adapted to write Web server applications with, even COBOL.
Client and Server Together
Although it is possible to program the Browser and the Web Server completely separately and independently. Perhaps only communicating through a well defined RESTful Web Services protocol. Usually we would like to leverage the skills of one to help with the other. We would also like a unified debugging and problem solving system. We would also like to reduce the number of specialized tools that people need to learn. Wouldn’t it be nice if the separation between client and server is largely transparent to the programmer? How well the client and server environments are harmonized is often a critical deciding factor in decision making.
Some Leading Development Platforms/Libraries
Which to Choose?
The choice comes down to which items are most important to you in the list of what makes a perfect programming system. Which one gives you the best vibes, or feels right for what you want to do. Often this is heavily influenced by your current experience; what technologies are you most fluent in already; which new ones will you need to learn. This usually goes beyond a single person and you have to consider the expertise and experience of your development team.
A big influencer is whether you are writing a new system completely from scratch or you have to extend or integrate to a large existing system. If you need to extend or integrate then often your choices are limited to things that can easily do that. If you are writing a new system then you are much free’er to choose. For the new system, often it depends how much you want to rapidly prototype, versus how much the requirements are set and you want to implement quickly.
You also have to consider the longevity of the tools you are choosing. Perhaps you don’t care since you just want to rapidly prototype and start from scratch every few months. Or perhaps you know the system you are developing will need to be maintained and extended for the next ten or twenty years. Then you have to look carefully at the health of the open source community around the tool, or the commitment of the company behind the product to keep developing it.
There are many more systems in use. As I mentioned, nearly every programming language ever invented has a Web Development framework. People fuss quite a bit about which one to use, but I think once you bite the bullet and dive in, you can’t go that wrong with any of these. Also most of these can interact and interoperate at different levels, so usually you can build a composite system using many different tools (you just have to learn them all). Anyway the Web continues to be a platform that promotes innovation and fosters continuing learning.
Previously I blogged on where I think ERP is going in the next ten years, recently I was thinking about where we have come from over the last twenty years. I started working on the Accpac product at Computer Associates back in 1992 to work on Accpac Plus for Windows*. At that time our customers installed our software from 5¼” floppy disks onto PCs running MS-DOS connected to Novell Networks. Usually the Accounting department had a separate LAN from everyone else due to security concerns. In QA we validated the hardware and BIOS we would run on, supporting only certain IBM and Compaq models. Going online usually meant having a CompuServe or AOL account connected via a modem on your phone line.
Evolution of ERP
The core of an ERP system is the double-entry accounting method which has been around since it was codified by Luca Pacioli in the 15th century. The basics of how this works has largely remained the same since then. Efforts to “modernize” double-entry accounting have repeatedly resulted in financial fiascos like Enron. Much of the work over the past twenty years has been expanding the ERP system beyond basic financial accounting. Expanding to the front office through CRM, expanding to the HR department with HR and Payroll functionality, expanding into planning with advanced operation modules and project and job costing.
Jump ahead 20 or so years to the present day. The product name is now Sage 300 ERP. We’ve progressed from Windows 3.1 through 95, 98, NT, 2000, XP, Vista to the present day Windows 7. We’ve progressed through 5¼” to 3½” floppies to CDs to DVDs to now downloading and installing the product from the Internet. Not only is the Accounting department typically on the main corporate network (usually a Windows Server network), but horror of horrors they are even connected to the Internet. We now support any PC, Workstation or Laptop that runs Windows regardless of manufacturer.
A lot of these changes are due to the industry maturing, buying a new computer isn’t such an adventure as you hope its BIOS is compatible with what you need to run. You can now safely buy computers from thousands of manufacturers running various CPUs from multiple companies like AMD or Intel. You can trust that you can use any USB device with that computer. Very rarely do you ever install a new hardware piece in a drive bay or motherboard slot. Most people have a dedicated high speed link to the Internet.
Another advance that we’ve seen is the rise of virtualization. Who would have thought twenty years ago that you could run another copy of your operating system in another window? Twenty years ago, multi-tasking barely worked. Now we can run all sorts of virtualized operating systems on our laptop at once. We have servers hosting dozens for virtualized servers that can be accessed from anywhere. Virtual images even integrate with the host environment so when you run a program you don’t know if it’s native or virtualized.
Twenty years ago, Windows was strictly single user. Now with Terminal Services and Citrix, Windows is truly multi-user. Users can access a central Windows server using a thin client to see their desktop. This allows easier centralized installations and management, since only one central server needs to be maintained. Additionally there is client software from all sorts of computers and devices to access your Windows session. With this you can access Sage 300 from a table like the iPad or from a non-Windows PC like a Macbook. Certainly UNIX/Linux fans aren’t impressed with this, but it did bring multi-user back into the mainstream.
Utilizing the multi-user and virtualization technologies allowed us to offer a cloud version of Sage 300 ERP with AccpacOnline.com. With this, Sage manages the datacenter and handles all the tasks like backing up the data, keeping the hardware and software up to date. Then the customer just has to use our ERP package, accessing it from a Citrix client. This then gives our customers a choice between buying or subscribing to our product. We have been doing this for ten years or so now. The diagram below shows the choices this gives you.
Today we enhance your ERP system with “connected services”. These services all run in the cloud and your ERP package connects to them for additional functionality and services. A prime example is the Sage Payment Solutions connected service which allows you to take credit card transactions right in your application and then have it call the online service to process the transaction in real time. The diagram below shows our connected service web strategy, how our on premise or hosted business applications access all these cloud services.
We’ve come a long way in twenty years from MS-DOS character based applications to 16-Bit Windows applications to 32-Bit Windows applications to Web applications. We’ve had many improvements in hardware, operating systems and connectivity. All this generally keeps things from getting boring or stale. As we move forwards we are seeing ERP spreading to devices beyond PC to tablets and phones and much great utilization of connectivity, sharing and social interaction that this enables.
* You can still see remnants of the old names in the product files. For instance there is the runtime\plus.ini which is still left over from the original name. In the original 16-bit version all the DLLs started with capw (CA Plus for Windows) and later were changed to start with a4w for the 32-Bit version. Generally if you look through the file names, or the SDK API documentation you can tell when something was introduced by the naming convention used.
As we roll out Web versions of the Sage 300 ERP (Accpac) accounting modules, we need to ensure that the system can run for a long time without requiring a server re-boot or other manual corrective action. This is far more important in a Web environment than client/server. In client/server the Accpac Business Logic typically runs on the client and if it crashes (which is bad), it only affects one person. In the Web if something crashes then it could affect all users of the system. Many regular GPFs can be captured and handled just causing one person to redo their work; however for more severe errors they could crash a server process or even the server itself. Rebooting a server is a long time for people to wait to continue their work (especially if there is no one around to do the reboot).
As we develop Sage 300 ERP we want to ensure that the system is completely stable and can run for months and months under heavy multi-user load. Hopefully the only time you need to reboot your server is when it installs the monthly patch Tuesday Microsoft Updates.
Why Do Things Crash?
In both client/server and web computing we need to deal with basic GPFs which are usually caused by programming mistakes like memory overruns. In the early days of Windows these were very prevalent, but in more modern times these can often be caught and handled within the program. Plus modern programming systems like Java and .Net work to prevent these sorts of problems by not supporting the programming paradigms that lead to them. Even for C programs, there are better static code analysis programs and methods to test and eliminate these sorts of bugs. Generally these sorts of problems are fairly reproducible and easily diagnosed and solved in a source code debugger.
The harder to reproduce and diagnose problems usually involve running the system for a long time with a lot of users before something bad suddenly happens. The problem can’t be reproduced with just one user. It’s not known how to reproduce the problem in a shorter time. Often once we know what the problem really is, then we can construct an easier way to reproduce it, but of course now we don’t need it, on the other hand throwing in a unit test to make sure this doesn’t happen again is a good idea.
These sorts of problems usually start to occur when the system is stressed and low on resources. Sometimes the problem is that our program has a resource leak and is the cause of the system being low on resources. Another cause is multi-user. We run as a multi-threaded process on the server where the individual threads handle individual web requests. If these aren’t coded properly they could conflict and cause problems. The Java and .Net environments don’t really help with these situations and often make things harder by hiding the details of what is going on making these harder to solve. These are the problems we really need to avoid.
How to Find These Problems
How do we solve these problems? How do we test for them? We use a combination of manual and automated testing to find if we have these problems and then employ a number of tools to track what happened.
Usually every Friday we get a large number of people to participate in manual multi-user testing where everyone accesses a common server and performs transactions, either as part of functional testing that needs to be done anyway or following scripts that concentrate effort in areas we are concerned about. Then anything weird that happens is recorded for later investigation.
We also use automated testing. We extensively use Fitnesse and Selenium as I blogged about here and here. We can string all these tests together, or run them repeatedly to create long running tests that the system is stable and that nothing bad happens over time. Another very powerful tool is JMeter which performs load testing on Web Servers. Here we record the HTTP requests that the browser makes to the server as users perform their tasks, then we build these into tests that we can run for as long as we want. JMeter will then play back the HTTP requests to the server, but it can simulate any number of users doing this. This gives us a way to easily automate multi-user testing and run with far more users and for longer periods of time than manual testing allows. Below is a screen shot of JMeter with one of our tests loaded. It might be hard to read the screen shot, but notice under the Runtime item is a Pick One Randomly item. This item will pick one of several tests randomly and run it. Then we simulate a large number of users doing this for long periods of time.
How to Fix These Problems
The key to fixing these sorts of problems is diagnosing what is going on. Usually if you can find the correct place in the code where things are breaking down, then fixing the problem is usually fairly easy. The real trick is finding that one line of code that requires fixing. Though major refactoring’s are required now and then, and then you need to re-test to ensure you have really fixed the problem and not introduced any new problems.
At the most basic level we include a lot of logging in the product, so when we run these tests we have logging turned on and can study the logs to see what happened. This only gives clues of places to look and often we have to add more logging and then re-run the test. Logging isn’t great, but it’s very tried and true and often gets you there in the end. We use log4j for our server logging.
For memory leaks, the Java JDK gives a number of good tools to diagnose problems. The Java JDK includes the tool JConsole which you can connect to the JVM used by Tomcat and monitor the memory usage. It also records the number of classes loaded, CPU utilization and a few other interesting statistics. But one really cool feature is the ability to generate a dump file of the Java Heap at any time. From the MBean tab in JConsole you can choose com.sun.management – HotSpotDiagnostic – Operations – dumpHeap. The p0 parameter is the name of the dump file and should have an .hprof extension. By default the file will end up in the Tomcat folder. After you invoke this you get a binary dump of the Java heap, now what to do with it? Fortunately the JDK includes another tool called jhat which is used to analyze the heap and then run a web server that lets you browse the contents of the heap from a web browser. From this you can see all the Java objects in the heap, and hopefully determine what is leaking (usually one of the objects with a very high instance count). You can also see the contents of each object and who references it.
One good tool for solving these problems is ReplayDirector from Replay Solutions. This is a commercial product that records everything that happens in the JVM. You can configure your Tomcat service, so that ReplayDirector records everything that happens as you run. It doesn’t matter if you call outside of Java since it will record the call and the result. Then you can play back the recording in the regular Eclipse Java debugger as often as you like to figure out the problem. When testing, the testers can put marks in the recording to indicate where and what happened. You can set breakpoints on any REST or other HTTP call into Tomcat, you can expand log4j messages to see them even if they aren’t logged. Below is a screenshot of the main ReplayDirector Eclipse plugin. This was recorded on a server being driven by JMeter. The green line is CPU usage; the blue line is memory usage. From the looks of the blue line, you can probably guess that I’m looking for a memory leak. I can set a breakpoint at a later SData request and then break into the program and look around in the debugger to see what is going on.
Ensuring a Web Server can keep on running reliably for long periods of time can sometimes be quite a difficult task. Fortunately the tools that help with this are getting better and better, so we can monitor, record and reproduce all the weird things that testing reveals and ensure that our customers never see these sorts of things themselves.
Last week I blogged about using MVC to program the new Sage 300 Web UIs, this week is an update on how we lay out the forms for the new UIs. I blogged on the previously here; but, this article is quite outdated now, everything mentioned will still work, it just isn’t the preferred way of doing things. This week’s post is an update on how we are now laying out forms and applying CSS to best effect.
In the early days of the Web there was only HTML. HTML was designed to structure content, mostly for scientific papers. Over the years, HTML was extended to allow more content types and to add tags for formatting. HTML has always had a set of table tags which were intended to display tables of numbers, usually produced by various scientific experiments. Clever web designers then figured out how to layout quite sophisticated layouts by using these table tags, by putting tables inside other tables and such. These ideas were extended for a while until it was finally realized that HTML was getting over-burdened and that we really needed to separate the content from the formatting. This was when Cascading Style Sheets (CSS) were introduced.
The intent with CSS was that people would stop using various formatting tags in HTML (like the font tag) and would instead do all their formatting in the CSS. Then in HTML tags would only be used to specify content like paragraphs, sections, copyrights, etc. Then tables would only be used to represent proper tables again. However for compatibility all the old HTML tags had to be left in place, so now we have a situation where you have to know which tags in HTML to use and which not to use, because you should use CSS. If you want to layout your pages using CSS then you need to use the HTML div and span tags which don’t do anything except give CSS a way to control your layout, then you want to avoid the older layout tags like the table ones.
CSS is used both for layout and for controlling the detailed look of content elements. This blog posting is concerned with the CSS features to do with layout and not the features to control things like color and font style. That would be a large blog posting in its own right.
GWT provides access to both the HTML way of formatting and laying out pages as well as the CSS method of laying out pages. Some GWT layout controls like the grid, horizontal panel and vertical panel wrap HTML table tags. Some GWT layout controls like the flow panel and simple panel wrap HTML div tags. This can be tricky since the GWT documentation isn’t always clear on what DOM elements are being created by a given control. But generally if you aren’t laying out the form via CSS then use all the panels, if you are using CSS then you will only use a couple of GWT’s layout panels.
CSS lets you control layout in a liquid or flowing manner, or control the layout by using fixed absolute placements and sizes (usually in pixels). Generally we want a liquid layout, so that our pages can be resizable and automatically adapt to different screen resolutions and still look good as various parts are hidden or shown depending on application options.
If you read a book on CSS (and there are many excellent examples), you quickly notice that all the terminology is from typography and magazine layout. Sometimes it takes a bit of getting used to this, so you can see past the terminology and apply the ideas to the layout of your form. This just takes a bit of patience and time to get used to.
How Sage 300 Uses CSS for Layout
The main way that you control form layouts with CSS is through controlling things in HTML div tags. These were designed to have CSS attached to them and then you move them by floating them around and controlling their size and dimensions. First let’s consider a simple form with labels, textboxes and sometimes a menu icon to the right of the text box. Since we want to control the placement of each control we wrap it in a Simple Panel and place them in flow panel’s as the diagram below indicates:
We then build up rows of the controls we want. Then to break each row we have a simple panel with the clear style which then acts as a line break. As you can see from the diagram this leads to a fairly simple setup, but it doesn’t look like it will lead to much of a page. But this is how we lay it out in HTML, and then we will leave it to CSS to make this look pretty.
To build up more complicated layouts we nest these. So for instance if we want a two column display then we put two flow panels side by side and then build the above idea in each one. The below diagram shows how this level is setup:
The labels in the diagram are the styles applied to these flow panels. Now you might think that we are just doing what we did with HTML tables at this point, having to nest them in complicated ways to get our effect. This is true to some degree, but we have more control. For instance in a table control if you want to hide one cell, then the cells below will not move up to fill it, because this is a logical table. Cells only move up if you hide an entire row of cells. In the CSS scheme things will move up whenever there is room (unless you tell them not to), so this keeps forms much cleaner looking when parts are removed.
Another thing that controls layout is the CSS box model. This is the mechanism where you can put margin, padding and border around any element that you apply CSS to.
These elements give you a lot of control around how you position elements on the page. If you make the left padding of a flow panel large then you push the things contained in it right, etc. How to really control the fine details of layout using the box model can be a bit confusing at first, but it is easy to look for examples on the web that show all the tricks.
In the Sage 300 ERP SDK we provide a SwtDefault.css that contains all the styles you need. Then if you create your declarative layouts as we indicate and follow our style guidelines, you should get nice looking adaptable screens fairly easily. So basically it contains a lot of styles that are intended to be applied to simple and flow panels. These are intended to setup various page design patterns that we document in our UI guidelines. They then have the various floats to format the rows and to setup multi-column displays. They also include all the correct settings for the box model so that everything is spaced and sized correctly. This gave our interface designers a very fine grained level of control over the look and placement of all the controls.
Hopefully this gives you a flavor for how we are using CSS to layout our forms. CSS is a large topic, but for an application developer it’s really a matter of having a basic understanding so you can use the palette of styles that are provided by default effectively. If you are really advanced you can change the CSS files to completely change the way Sage 300 screens look.