Stephen Smith's Blog

Musings on Machine Learning…

Posts Tagged ‘lean startup

Sage Innovation Process

with 4 comments

Introduction

Sage is always redefining and working to improve its software development methodologies. We’ve transitioned from using Waterfall to using Agile development. We’ve incorporated User Centered Design into all our development. We’ve spend much more time having everyone in the organization connecting and talking to customers. We bring customers into our offices for “coffee with the customer” chats, where the whole development organization can hear what is working well and what is causing our customers pain.

We are now endeavoring to transition our development process to be more upfront User Centered Design and to introduce much more creativity into our processes. To some degree having creativity and processes in the same sentence sounds contradictory; however, we do need to have organized creativity that results ultimately in shipping software. The goal is still shipping software and not producing PowerPoint’s. What we want to do is ship software that aligns very closely with what customers require and which delights users in how appealing, friendly and easy to use it is.

What we are really trying to avoid is:

http://www.dilbert.com/2012-05-07/

which unfortunately happens in far too many products.

Generating Ideas

We started this process with a number of “idea jams”. We ran these as all day events at our various development center campuses. We picked half a dozen product categories, like manufacturing, and created a team for each one of these. Participants then volunteered for a team. The team then had the day to generate ideas (usually 100) then narrow them down to the top three and prepare a business case for their ideas. These were then presented to the whole group with some very original and animated presentations. Then everyone voted on which they felt were the best.

We don’t only take ideas from our idea jams, but also from customer feedback, competitive analysis, disruptive new technologies, business partners, development partners and any other source we can think of.

So out of these processes we accumulated hundreds of great ideas. Now what?

Narrowing Our Focus

Now we want to pick the best ideas to actually implement. So how do we pick and choose? As a first pass, the Product Management teams picked the twenty or so best ideas.

For the second pass we set up interviews with the CEOs of companies (generally of companies that already run our products). We hold these interviews as Webinars and generally for each interview run three ideas past the CEO using artist’s conceptions, mock-ups and verbal descriptions. Basically we want ideas that will excite the CEO of a company. After all the CEO has the ultimate buying power and if the CEO wants our product then we have the best chance of successfully selling to that company.

As you can imagine, getting an hour of a CEOs time for this sort of interview, can be quite difficult. CEOs are very busy and often barraged by huge amounts of sales pitches and spam on a regular basis.

Call to Action: If you are a CEO or know a CEO who might be interested in these sort of interviews, please contact me, by e-mailing me at Stephen.Smith@sage.com.

The following diagram shows this narrowing down process. We are now moving a number of ideas into the “Experience Testing” phase. Where we really want to focus on the overall experience to ensure we are delivering great business value to our customers in a very pleasing package.

Overall Process

All this initial idea generation, customer validation and experience design then become the first steps in an overall Software Development Process as shown in the next diagram.

This diagram emphasizes the initial phases, so the usual Agile Product Development then is the black box between “Experience Testing” and “Early Adopters”. However with modern development techniques all the boundaries in the diagram are very blurry and there is quite a bit of iterative improvement in each phase.

Basically we want to keep getting continuous customer feedback through the whole process. We never really know how well something will be received until the customer is running the real product on real production data in their real environment. We need to get to that phase as quickly as possible. Development needs to be oriented to delivering a minimum viable product as quickly as possible to get it to the early adopters. This then leads to real feedback and to the all-important “persevere or pivot” decisions that need to be made in any innovation process.

In any iterative process with lots of feedback, it’s very important that we achieve lots of “validated learning”. Where all the truths from all this feedback are documented and incorporated going forwards. So we keep moving forwards and don’t just iterate in circles. The Lean Startup people have very good processes for doing this, so you really do learn from your mistakes and don’t just keep repeating them.

Summary

The goal of this initiative is to make Sage’s product development process more innovative and creative. To incorporate much more structured design into all aspects of development. To really encourage far more stakeholder involvement in all parts of the software development process. To really get and incorporate feedback as validated learning’s that greatly enhance our products.

And remember anyone that is or knows a CEO that might like to participate in our concept testing, please let me know.

Written by smist08

May 12, 2012 at 6:13 pm

Doing Work in Small Batches and Limiting Work in Progress

with 5 comments

Recently Software Development has been borrowing a lot of ideas from Japanese Lean Manufacturing. In a way this mirrors the progression from the Detroit way of building cars pioneered by Henry Ford to the modern way pioneered by Toyota.

The original Ford production line approach was that a car was assembled on an assembly line by going through a series of specialized stations that were optimized to do one things really well and efficiently, so the car would go from perhaps the attach doors stations to the add Window glass station. Each station was optimized to do its job really well and the priority was to keep the assembly line moving no matter what. If the assembly line ever stopped then the hundreds of people working on it would be idled and this was considered a disaster. If problems occurred then they were noted and fixed post-assembly line. This process was then improved by optimizing each station to be quicker and more automated. Generally this produced a very good system of producing a very high volume of identical cars.

From the outside a Toyota assembly line looks nearly identical. But it operates very differently. Toyota didn’t have the same large market as Detroit and didn’t need the volume. They needed to produce more specialized cars and needed to switch their assembly lines quickly from one run to another. Their emphasis wasn’t on making very specialized machines to do one task quickly; it was on producing general machines that could do a number of different things and could be switched quickly from doing one job to another. Toyota then does small runs on their assembly line rather than one massive long run. The big benefit to doing things in small batches was that quality problems are found much sooner. Hence they can stop the assembly line on the first car and fix a problem, rather than producing thousands of cars with the problem and then possibly not fixing the problem because it’s too expensive at that point. This also re-enforces the practice of fixing problems right away rather than letting them pile up and that it’s more cost effective to have the production line idle while the problem is fixed than having to fix it later.

Another example from the book “Lean Thinking” by James Womack and Daniel Jones recounts a story of one of the authors stuffing envelopes with their two young children. The children wanted to take the approach of folding all the letters, then putting all the folded letters in the envelopes then sealing all the envelopes then putting stamps on all the envelopes. This seems like an efficient and intuitive way to approach the problem. However the author insisted they do it the less intuitive way of one complete envelope at a time, i.e. folding one letter, stuffing it in an envelope, sealing it and putting a stamp on and then going on to the next one. Which way is faster? It turns out that researchers have timed groups of people performing this task and it turns out the one at a time approach is faster! It has a couple of other advantage as well. One is that if there is a problem, you find out right away; if the envelope is the wrong size, you find out on the first letter, not after you have folded all the letters, this could save you then having to refold all the letters to fit the envelopes. Similarly if you take a break you know how far along you are, if you’ve completed 40 out of 100 letters then you are 40% done and you can estimate your remaining time easily. You can’t do this when you do each step completely. This process is called “single piece flow” and is an example of doing a task as lots of small batches rather than one large batch. This is a general principle you want to apply in everything you do, resisting the old Ford type assembly line type mentality. You also receive the first finished product off the assembly line quicker, so you can validate it with your customer before too many more are produced to ensure you are producing the right thing.

Kanban vs Scrum

To some degree batch size is at the heart of the difference between the Kanban and Scrum methods of developing software. Scrum tends to batch things into sprints (usually two or three weeks long) which is much smaller than traditional waterfall development, but Kanban wants to take it further. In Kanban you only work on one or two small things at a time and finish one before starting the next things. You operate on lots of small batches, but you also have to limit how many batches (or tasks) you can have in progress at a time. This is called limiting work in progress (WIP). In Kanban you don’t worry about batching things up into sprints, just keeping the jobs small and limiting WIP. Kanban requires a lot of discipline and often if a development team hasn’t successfully progressed from waterfall to scrum, then adopting Kanban is a disaster.

The goal here is to produce small high quality increments that are immediately validated by customers. This way if you are going in the wrong direction, you won’t get too far before discovering it and not too much time will have been wasted (only a couple of small batches).

How do you get customer feedback, one small increment at a time? Obviously you can’t have dozens of customers download, install and play with a new version of your product every time you add a small increment and then interview them about it. But there are techniques you can use to do just this. The first is to produce your software via “continuous deployment”. This means that every build of your software is automatically deployed to customers whether that means pushing it to a web server or loading it on an automatic software updater service. You need to be able to control which customers get it and you need to be able to roll it back if something goes wrong. To be confident with this sort of procedure you need really good automated testing in place to ensure you haven’t accidentally broken something. Then to find out whether your increments (or batches) provide customer value, you need really good instrumentation in your product to report back on customer usage, whether they use the feature, how long it takes to perform tasks and any other useful statistics.

Software products are large and complicated systems. Managing that complexity is a major challenge. Breaking large tasks down into many small tasks is a powerful tool to reducing complexity. Combining many small but high quality parts results in emergent complexity, but again it is much easier to manage by adding one small thing at a time rather than suddenly throwing several large items into the mix at once.

Summary

Most of the ideas in this blog posting come from my reading the “Lean Startup” by Eric Ries over the holidays. Quite a good book with many useful ideas on product development and innovation. Reducing Batch size and limiting WIP appeal to me to help make Sage more agile and innovative as we move forward.

Written by smist08

January 8, 2012 at 12:35 am

On Customer Adoption

with 3 comments

The Problem

Traditionally with ERP systems, a new version is released every 18 months or so. Typically customers will only upgrade every few versions for a number of reasons discussed below. This leads to a major problem, namely if a customer requests a new feature and we jump on it, then it still could be four years before they see this feature. How this usually works is shown in the following timeline.

In an agile on-line world, this seems like rather a slow process to provide value to customers. Most Sage products have an on-line suggestion web site, such as https://www11.v1ideas.com/SageERPAccpac/Accpac for Sage 300 ERP. However customers tend to give up on making suggestions to these once they enter a number of suggestions and don’t see any movement in the next few months. As more and more people are part of the MTV Generation (or later) then instant gratification becomes the expectation.

Traditionally this is all part of a fairly large and involved Product Development Life Cycle. Product Managers spend their time talking to customers and partners as well as evaluating various industry trends to come up with a list of features for the next version. Usually a number of themes are developed and implemented as major new features. R&D then takes as many such themes and features that they think this can develop in 18 months. If all goes well, these are implemented, extensive testing is performed and an updated version of the product ships after these 18 months. Then if Product Management has done their job properly and R&D has faithfully implemented what was laid out, then customers should be fairly happy with the new version.

Of course things aren’t quite that rigid anymore. Most development groups (including Sage ERP) have adopted the Agile Software Development Lifecycle where the development team is implementing features in small batches taken from a product backlog. The product backlog is in priority order so the higher priority features are implemented first. A side effect of this is that for lower priority items in the backlog, Product Management can swap these out with other higher priority features as business needs or understanding changes. This then leads to greater flexibility and does allow some newer customer suggestions to make it into a product sooner. I blogged on some aspects of this here and here.

Often we are tasked with the goal of shortening our release cycles; say from 18 months to 12 months or even 9 months. However we get a lot of resistance from customers and partners on this. The main reason is that traditionally upgrades are fairly costly and time consuming activities that can often be quite disruptive to a business. As a result customers typically don’t want to upgrade more than every three years, so there is no increase in customer value by providing more frequent releases. Even then the main reason customer’s site that they upgrade is to stay in support, since we only support the two most recent versions or to stay compatible with upgrades from other vendors, notably Microsoft. If you need to know you work well on Windows 7, then you need a more recent version of your products. A large part of this problem has been caused by us software vendors, when presented with a choice to add a new feature to a release or to make upgrading easier, we consistently choose to add the new feature and leave any difficulty with upgrading to the Business Partners to deal with; who, of course, pass this cost on to our customers. Right now 18 months works well because by supporting two versions, customers only need to upgrade every 3 years.

So out of this we have two significant problems to address:

  1. Making upgrades easier so it isn’t a huge inconvenience and cost to upgrade.
  2. Make the features and functionality we add to our products more compelling so customers cite these as the reason for upgrading and not just keeping up with Microsoft compatibility problems.

Frictionless Upgrades

To address the first point, we’ve been working hard to make the upgrade process easier. This is Sage’s Frictionless Upgrade initiative. We’ve just begun on this journey, but in a number of products we are already seeing results. Such as in Sage 300 ERP we’ve combined all the separate module installations into one master product installations, allow you to activate multiple applications to the new version in one go and to minimize database changes so Crystal Reports and Macros don’t need modifications.

Going Online with services like www.accpaconline.com can help, since now you don’t need to install the software yourself, but you still have the work of making sure custom reports still work and training staff on any changes to their workflow. Generally the process of upgrading can be done well or badly whether you are online or on-premise. Certainly we’ve seen cases where overnight changes to web sites like Facebook have been very disruptive.

Many products such as Windows or Apple iTunes have an auto-update feature, so even though these are on-premise installed on the local computer type applications, they can still be upgraded frequently with little fuss. Expect to see this sort of functionality come to more and more business applications allowing the software to be upgraded frictionlessly without requiring a major database conversion or requiring changes to customizations.

Making Features Compelling

The intent with all the customer visits, customer interviews and other research that Product Management performs is that the features we add to the product are compelling either to existing users or to attract new users. A lot of work goes into researching and collecting all the data that goes into this decision making. Generally this yields better results than being driven by technology trends or by what competitors are doing; however, we tend to not being seeing the results we would like. What is missing?

With the scientific method, like what you see in Physics, Chemistry or Biology, people come up with theories, but these aren’t really believed until they are backed by experimental evidence. In software development we tend to do a lot of work doing research and creating theories and spend a lot of time discussing these theories with customers, partners and analysts. However we don’t take the next step and scientifically test these theories in real customer situations.

We tend to rely on doing studies on customers using prototypes or mockups. But when customers aren’t in their real work environment, doing real work, the results tend to be suspect. Is the customer just telling you they like your new feature to make you feel good, even though they will probably never use it? Are the questions being asked at interviews slanted to get the answers the researcher wants?

I recently read “The Lean Startup” by Eric Ries. This book, among other things, heavily recommends real scientific experiments with real customers in their real environment. No more separate studies with proof of concepts or mock ups. In a way we are doing this now. The only problem is that we don’t know the result until after four years and then it’s very hard to remove something that really didn’t work.

So how do we perform these experiments on real users in their real workplaces doing real work? Sure it’s easy for Facebook to try a new feature in say one town to see what the reaction is, but here we are talking about real business software that people rely on to run their businesses.

The real trick to this is to keep features small and only try small changes at a time. This is often referred to as a Minimum Viable Product or MVP. Only the absolute minimum functionality to do the job is produced. Any additional functionality is only developed (and tested) due to customer demand. This tends to greatly reduce complexity and unnecessary bells and whistles that we see in so much software. This then allows features to be developed and tested quicker. In manufacturing this is referred to as keeping batch size small.

This requires a great deal of good instrumentation in the product, so we know if a feature is being used and if it is helping (are users really more productive entering invoices?). Creating a full release with all the changes from the work produced by a large team over a year is a major undertaking. But adding small changes that have been fully QA’ed is fairly easy. In fact we already do this with hotfixes. In a way each hotfix is an experiment as to whether we have fixed a defect (hopefully mostly we have). So why are small new features any different than hotfixes?

You might argue that SaaS vendors can do this easier, since they can introduced the change for just a few customers by routing them to a different application server. However with auto-update we could do the same for on-premise customers. Give them the new feature and then based on the results, if they are good, roll it out to all customers, but if they are bad, then roll it back for anyone that received it.

These techniques have produced great results in a number of well documented cases, but will they be accepted for business software? I think right now there will be a fair bit of resistance to these sort of practices, but I think it is really a PR exercise to prove to customers that by participating in these sort of experiments, that they will get the features they are asking for sooner and that the new features they receive will be far more valuable. One of the main advantages of these techniques is reducing waste, by detecting things that won’t work quickly and discarding them quickly. This way some really dumb idea can’t make its way all the way through development using all those resources, to just annoy users in the end. Chances are this will start with new connected services or new on-line features, but I think eventually this could become more mainstream for even large ERP and CRM systems.

Summary

New techniques are being developed to develop customer value quickly and to get it into customer’s hands much faster than we have traditionally. However some of these techniques like experimenting on customers running live systems are going to take a bit of good PR to prove their value to get the necessary participation. If this all sounds crazy, give “The Lean Startup” a read, it paints a fairly compelling picture.

Written by smist08

January 1, 2012 at 9:49 pm