Stephen Smith's Blog

Musings on Machine Learning…

Archive for April 2009

On Teamwork

leave a comment »

Forming teams and having them work effectively together can be a big challenge. Some teams are necessary just since a job is too big for one person to do, so a team is formed to break the job up and solve/implement it together. Often these sorts of teams comprise of members with the same background like programmers or QA.

Another sort of team is the cross functional team. These are often formed from people with diverse backgrounds. The mandate of these teams is often to add creativity to solve a problem. To find unique approaches that perhaps a single function team or individual would never consider. Often these sorts of teams are locked in a room for extended periods to brainstorm. To work collaboratively together to come up with a new type of solution to the problem. Often the team has a facilitator rather than a leader to keep the discussion going. Whether this team eventually implements the solution or passes it off to a more traditional implementation team is determined once the solution is known and the task breakdown is known.

In agile devleopment, developers are broken up into “scrum” teams. Small cross-functional teams that stay together and move from problem to problem as a team. Usually these teams consist of one or two people from each of: Product Management, Programming, Business Analysis, User Centered Design and Quality Assurance. Here the Product Manager usually acts as the “customer” from agile terminology. The idea of keeping the team together is to allow it to jell. It is important that for the team:

1. It is every member’s day job to be on the team. This is their primary job responsibility and their number one priority.

2. Every member wants to be on the team. They aren’t forced or strong armed to be on the team.

3. That the team can collectively replace members that aren’t working out.

4. That the team is self-managing. There isn’t an external manager managing them. All management comes from within the team.

The hopefully this can become a very high performance team, that works together to solve a great many problems over a long period of time. That they can be a creative team that comes up with innovative solutions and that they then can work together to implement the solutions they have come with. A teams develop, what they are good at can be observed and then the jobs most suited to them can be assigned (or they can lobby for). Generally this can lead to a very dynamic exciting work environment that is also very rewarding since all the team members get to participate in the creative input and share in the reward of completing the job.

Written by smist08

April 20, 2009 at 12:03 am

Should We Still Use the C Programming Language?

with one comment

Within Sage Accpac ERP, most of our Business Logic (our Views) are written with the “C” Programming Language. C was originally developed to write the original Unix kernel in. It was intended to be nearly as efficient as assembly programming but way more productive for programmers. C is still in wide use today. The Linux kernel and many open source projects are still developed in C. Many Microsoft products are still written and developed in C. Most video games are developed in C.  The original idea of C was to be a portable language where you can compile your programs for different platforms, like Linux, MacOS or Windows; and in fact we can compile our Views for either Windows or Linux.

We often get asked whether we will re-write this Business Logic in a more modern programming language like C# or Java. These are object oriented derivatives of C with many of the bad things in C removed (unlike C++ which keeps all the “bad” pointer stuff in). Part of the question is whether there is any Business case for doing so? Would C# or Java make a difference for end user customers? Is this a necessary step, no matter how expensive, to keep the product maintainable and capable of delivering the features and functionality required for tomorrow?

While contemplating how old C is, I ran across a news article celebrating COBOL’s 50th birthday and pointing out the 70-80% of business transactions in the UK are still processed by COBOL programs and that 32% of enterprises still employee active COBOL programmers. I guess this shows how persistent technology can be. How hard it can be to re-write. And how much mileage you can get from an older technology. The ROI on these original COBOL programs would turn out to be huge, given how long they have been in service.

For us C has been very powerful and we’ve gotten a lot of use out of it. But questions remain. Universities tend not to be teaching C programming anymore, so can our ISV partners as well as ourselves continue to find enough C programmers to meet our needs? Is our C code getting old? Is bit-rot setting in? Is it worth re-writing in a new technology for the benefit of re-factoring all the code during the re-write to make it more maintainable and efficient alone. There is also all the logistics of transitioning from the old technology to the new; managing things when half the system is in the new technology and half in the old technology. Currently there are thousands of Views and hundreds of thousands of lines of C code; transitioning this will be a major undertaking.

Written by smist08

April 11, 2009 at 5:11 pm

On Adopting Agile Development

with 2 comments

We are slowly moving from a traditional waterfall software development process to a more agile one. We are looking to adopt scrum without going all the way to extreme programming (XP). As a first step we are looking to go to an iterative waterfall model. In this case we adopt agile iterations, but then do a mini-waterfall in each iteration. This would be a first step to learn some of the ideas of agile, rather than biting off too big a process change at once and risking chaos.

The waterfall method of software development has been around a long time, and many successful products have been developed using it. However it does have a number of drawbacks:

1. It relies heavily on all the requirements being defined upfront and then complete accurate specs being developed before coding begins. Practically speaking this has turned out to be humanly impossible. Requirements change as market conditions evolve. No one can review a large spec and ensure that something hasn’t been missed, or gone in a bad direction.

2. The handover from programming to QA leads to huge scheduling problems. You can schedule how long to code something (at least relatively) and how long to perform functional tests. But how do you schedule how many bugs will be found? How long to fix those? How long to retest  these? How long QA is delayed because they are blocked by these bugs?

3. Combining 1 and 2, practically speaking the worst design defects are found in QA when it is quite expensive to fix these. Again this tends to be un-sheduleable work and usually is the cause of the biggest product delays.

Agile looks to fix these problems (and many more besides). The idea is to keep units of work manageable without requiring super-human powers. Another idea is that you don’t need the requirements up front, not do you need the specs up front (if at all). Some of the key ideas:

1. Break down requirements into small easy to manager “user stories”. This makes the requirements more customer focused and really forces functional decomposition right from the beginning. It then lets you prioritize these requirements and tends to limit designer gold plating.

2. All team members are involved for the whole project. There are no hand overs from one group to another. Traditionally these hand overs lead to delays (due to people not being available at the right time), defects due to misunderstandings, and lack of teamwork. With agile the design analyst, programmer, ucd specialist, QA person and product owner are always working together as a team through the whole process.

A lot of the rest of the Agile Development Process is then executing these ideas in a best practices type environment using the experience of previous teams. From the experience of others, leads to the scrum principles:

1. Short fixed length iterations (usually 2 or 3 weeks).

2. The product is “shippable” quality at the end of every iteration.

3. How to break down the user requirements,  and pick the ones to implement in each iteration.

4. How to prioritize the requirements and handle things that go wrong, like too hard user stories, key team members on vacation, etc.

Generally many product groups have discovered that Agile Development Process’s are more productive (more features or products are developed), quality is higher and the process is more predictable. Probably the hardest thing with adopting Agile is just to forget all those practices taught in older Software Engineering courses in University and accept the newer methods.

Written by smist08

April 4, 2009 at 4:42 am