Stephen Smith's Blog

Musings on Machine Learning…

Archive for March 2014

On Accelerating Projects

with 3 comments


Usually projects start with a POC (Proof of Concept) and then move on to a real project using some subset of existing resources and then if the project looks exciting, senior management will want it done faster and is willing to throw more resources at it.

In this article I’m looking at some of the problems this causes and some of the solutions to the problem.

Just Ship the POC

In the interest of Internet time, many companies are following the Lean Startup philosophy of shipping a MVP (Minimum Viable Product) early and then quickly iterating on this based on feedback from users. This has worked well for a number of companies, but it has also failed miserably as well. The big mistake is usually the Viable part of MVP and that the work to make the product viable is greatly underestimated.

This approach often leads to products shipped too early which receive a black eye or lead to very major re—writes (often called refactoring) that amount to starting over from scratch. Often this just starts you on a much bigger project that you need to ramp up for. This approach also tends to do away with a project management framework which then leads to various project dependencies blowing up into major problems.

Brooks’ Law

Brooks’ law is a principle in software development which says that “adding manpower to a late software project makes it later.” This is from Brook’s book: The Mythical Man-Month. The corollary of Brooks’ Law is that there is an incremental person who, when added to a project, makes it take more, not less time. And generally people that have worked on large projects know who that person is.


Some of the causes of Brooks’ Law are:

  • Training Time. It takes time to bring people up to speed. Even if they are experts in the technology you are using, they still need to learn your product and understand the work that went before they joined the team. This also diverts productive resources from the project to conduct this training and then extra supervision and oversight are required to ensure they are up to speed.
  • Communications Overhead. As the number of people increases the number of possible communication channels increases dramatically. Keeping people in sync and moving in the same direction becomes a real management challenge.


Another common comparison is that the way we put resources on a project is like hiring nine women and demanding that they produce one baby in one month.



This might make it look like completing a large project on time or accelerating a large project are impossible. This conclusion is usually due to a number of misinterpretations of Brooks’ Law and to a number of older bad management practices. But ramping up a large project, although not impossible is still highly challenging.

First Brooks’ Law applies to projects that are already late. Ramping up the project earlier certainly helps. Then there is less code already developed to learn and we don’t take away resources to babysit and to train people later in the project. Having real schedules also helps, just having a schedule with no basis in reality is a problem with the schedule not with the project.

The quantity, quality and roles of the people added to the project is a major factor as well. One approach is to over hire for a project to compensate for the time taken for training and communications overhead. Adding high quality people and removing underperformers early can really help as well. Adding some roles like QA and documentation later in the project is not so harmful.

Modern development practices and supporting technologies can really help as well. For instance continuous integration, agile development and test driven development all help greatly by keeping projects in a complete state.

Good design patterns simplify the distribution of work, since then people know where they fit into the design framework and can do their part independently. Similarly if the project is well segregated so that small groups can work on independent parts of the project to greatly decrease the communications overhead.

Another aspect is the social and political climate of a company can have a big effect. Often small companies or projects can rely on a hero to carry the project to success. This model doesn’t scale to large projects, where lots of people doing repeatable work is required. Once projects get to a certain scale, all the work needs to be de-centralized and all the work needs to be delegated. Tightly controlled organizations with a lot of control freaks are going to have problems once they scale past a certain point.

A good Project Management framework within a company really helps as well. Things have progressed quite a bit since Gantt charts and if they have sufficient tools for good tracking and the power to organize dependencies, this can help greatly.

Things to Watch Out For

As you scale your project here are a few warning signs.

  • Different groups doing the work differently. This indicates that the design patterns aren’t laid down properly or that people aren’t following them. This will lead to major problems quickly.
  • Training time and individual ramp up time not being factored into the schedule. Schedules that are made around resources that are assume to be fully productive are going to slip. As people are added to a project, people need training, acclimatization and ramp up time. The schedule also has to allow for the time of trainers, mentors and supervisors of the new people.
  • Watch out for high people turn over. If people are leaving the project or company in higher than normal problems, it usually means a project isn’t being run properly and people are unhappy working on it. Further replacing these people is expensive in recruiting, training and individual ramp up.
  • Miscommunications. Often if a lot of communications problem are happening it means people are avoiding the communications overhead problem by not communicating and this causes lots of problems on its own. Managers and team leaders need to stay on top of this and make sure the necessary communication happens. There are a lot of new good tools for this like Wiki’s and group virtual meetings that can help.
  • Watch out for bottlenecks. Often when you ramp up a project, one group might have been neglected and starts to be a bottleneck. Eliminating bottlenecks has to be a key role of the Project Management Organization (PMO). Especially watch out for self-imposed bottlenecks where people are taking on too much and refusing to delegate or distribute work. These sorts of practices just don’t scale.
  • Watch out for project dependencies. The more dependencies in a project, the harder it is to manage. If the project isn’t designed to scale out independently, it isn’t going to work. Lots of dependencies between groups is a real red flag and a huge indicator that the project needs to be restructured.


Many companies want to take on ambitious large scale projects. With the current demands to do things in Internet time, there are a lot pressures to really accelerate projects. No one wants a train wreck like in the picture below, but we do have to use past experience from these to guide current projects away from the obstacles and to keep them on track.


Written by smist08

March 29, 2014 at 6:08 pm

On Retaining Employees

with 3 comments


In a few previous blog posts I’ve been talking about attracting new employees whether through office design, advice for someone starting their career or corporate mobility. In this article I’ll be looking at some ideas on how to keep existing employees. Generally the value of a high tech company largely depends on the IP contained in the heads of the employees and growth prospects depend on their ability to execute.

High Costs of Hiring and Training New People

Hiring new employees is quite time consuming and a slow process. Especially in todays job market which is very hot with all the venture capital that is freely flowing right now. Is this a bubble that will shortly burst? Either way hiring is fairly slow right now. Then any new employee has to take quite a bit of time to learn your ways of doing things and to become familiar with your existing programs and systems.

On the converse new employees do being new ideas, new experiences and new perspectives that greatly help an organization. Having a stream of new employees is very beneficial, but when it becomes a torrent then things get tricky.


To retain employees, it isn’t just a matter of higher salaries (though that works well for me), but understanding people’s motivations which may not be intuitive. A good video on people’s motivations is this one. Motivations are really quite complex and much more is involved than just money. This video’s thesis is that you need to pay enough money to take money off the table as an issue, then the priorities become:

Autonomy: people want to be self-directed, they want control over what they do. This is one of the reasons that unstructured time is so successful at so many organizations.

Mastery: people want to have mastery at what they are doing. They need time to learn and practice what they are doing in order to raise their work to a higher level. Often in technical organizations, this is why frequently moving people between projects causes so much dissonance. People aren’t just cogs that do repetitive work that are all interchangeable. This is often confused with resistance to change which is something quite different.

Purpose: People want to make a contribution. They want to see their work being used by happy customers. They want to see their work making other people’s lives better. Putting out poor quality products that annoy people will cause employees to want to leave an organization. Having corporate policies that violate customer’s privacy or do other semi-legal immoral corporate activities will disengage the workforce.

If a company pays a competitive salary then these items will be very important in engaging and retaining employees. But there are still other factors.

Golden Handcuffs

One of my favorite ways to be retained by an employer are golden handcuffs. These are benefits like stock options or future bonuses that you have to remain an employee to collect. Often these can become quite valuable making it a very difficult decision to leave. For instance stock options vest over five years and you can retain them for ten. If your company is growing and its stock is going up then these can become very valuable and walking away from them is as difficult as getting out of handcuffs. Even if you company isn’t public, having these in the hope of going public is a great retention tactic.


Challenging Work

Technical employees like programmers value challenging work where they get to use newer technologies. This keeps people interested via continuous learning and people feel secure in their profession since they know their skills are up to date.


A lot of times technical people leave an organization because they feel their skills are getting dated and that it’s hard to learn and practice newer practices.


When performing employee surveys, often the key answers given to the question of why people stay is that they like their co-workers and/or they like their boss. To some degree this comes down to having a very positive work environment. Ensuring everyone treats everyone else with respect and that bad behavior to other people isn’t tolerated.

Another key aspect is when hiring to consider how people will fit in to the current teams and often to give team members a chance to participate in the job interview process to give their input on this.

Probably the most important relationship is between an employee and his boss and this means that ensuring managers are properly trained and that you have good managers is extremely important.


Having good vertical communications in an organization is critical. A lot of times when people are having problems or not fitting in, they are saying so, just no one is listening. Many times people leave due to misunderstandings or frustrations that they aren’t being heard. Having good clear communications channels is crucial.

Also an organization needs to ensure that all the employees know what the corporate priorities are and also what is the reasoning behind these. People won’t be engaged if they don’t understand why a company is doing something and in fact will often act against it.

Another good practice is to have good coaching and mentoring programs within the organization. These can really help with communications and employee development.

Don’t Reward the Bad

On the converse, you don’t want to retain people at any cost. If people aren’t performing, aren’t engaged or exhibit bad behavior, don’t reward them. Often company’s give out bonus’s anyway because they are worried about losing the employee. But I think in some cases it’s better for everyone if the employee finds a different opportunity. You especially don’t want to do this year after year or people just won’t have confidence in your rewards system.


Retaining employees doesn’t have to be hard. Generally employees are motivated by things that are also good for the company like pursuing innovation, pursuing learning and staying up to date. Generally a healthy happy workforce is also a productive workforce, so many of these items are in everyone’s interest. When companies lose sight of this, they get themselves into trouble.


Written by smist08

March 22, 2014 at 4:15 pm

The Umbrella Ceiling

with 4 comments


My wife, Cathalynn, and I were recently discussing issues with people moving to other cities to pursue their careers and the hard decisions that were involved in doing this. My nephew, Ian Smith, is just starting his career and when choosing where to work has to consider what it takes to grow in the role he eventually accepts. When I started at Computer Associates, if you wanted to move up in the organization past a certain point, then you had to move to the company headquarters in New York. Similarly, when Cathalynn was working at Motorola, the upwardly mobile had to relocate to Schaumburg, Illinois.

From Cathalynn Labonté-Smith

Recently, Vancouver hosted a Heritage Classic hockey game at BC Place as have many cities across Canada. An outdoor rink facsimile was made inside an indoor venue to recreate a 1915 game complete with original uniforms and “snow”. The plan was to retract the ceiling on the dome but a torrential downpour kept the giant umbrella deployed. Despite the nostalgia of the game the Vancouver Canucks and Ottawa Senators were playing for real—this game counted for NHL points, so the integrity of the ice had to be maintained.


We’ve all heard of the glass ceiling. Indeed, yesterday (March 8th) it was International Women’s Day—a day to reflect on all aspects of women’s’ equality and well-being. In the corporate world, how are we doing? According to Catalyst only 4.6% of Fortune 1000 companies have women CEOs (

We’ve all heard of hitting the glass ceiling, however; living on the West Coast working in the high technology sector we have what I call an umbrella ceiling that applies to both genders. Umbrella in the down or sun position–you are blessed with a lifestyle that promotes health and well-being with a year-round outdoor playground and cultural diversity. Umbrella in the up or rain position—you are blocked from moving on to a top job within any corporation that has a head office outside of British Columbia you have to leave. We’ve been to many a tearful going away party. But then if you stay as the Smiths have where are roots and family are, you many spend your weekends hiking, snowboarding, cycling, gardening, wine-tasting, cross-border shopping to Seattle and in many other wonderful pursuits, so that’s cool too.

Does it have to continue to be this way? With all the technology like Skype, other teleconferencing software, cloud applications, mobile phones, portals, access to travel and other collaborative tools that are available why do corporations still tend to centralize top officers in one location? Or, can companies truly embrace the mobile workforce including more females at the CEO level. Are they missing out on or losing top talent for this-is-the-way-we’ve-always-done-itism?

I’m turning this over to the expert, Mr. Steve himself. Cat out.


Physical versus Virtual Offices

A lot of discussion comes down to how important is face-to-face interaction. How much can be done virtually via Skype, e-mail, telepresence, chat and other collaborative technologies?

My own experience is that there are a lot of communication problems that can easily be cleared up face-to-face. Often without direct interaction, misunderstandings multiply and don’t get resolved. Probably the worst for this is e-mail. Generally, programmers don’t like to talk on the phone and so will persist with e-mail threads that lead nowhere for far too long rather than just picking up the phone and resolving the issue.

But with video calls so routine can much be handled this way instead and physical meetings kept to a minimum? Another thing that limits interactions is living in different time zones and how much time you have to interact. For example, I have days bookended by early morning and late evening conference calls.

Generally, office design has improved over the years as well to better facilitate team work and collaboration. If you aren’t in this environment are you as productive as the people that are?

Tim Bray leaves Google to stay in Vancouver

A recent high profile case of this was Tim Bray who worked at Google but lives in Vancouver. He gave a quick synopsis on his blog here. Google has a reputation as a modern web cloud company, and yet here is a case where having someone physically present is the most important qualification for the job. If Google can’t solve this problem, does anyone else have a chance?

Though personally it seems that Tim accepted the position at Google with the assumption of moving to California, so it seems a bit passive aggressive, then staying in Vancouver and just pretending he would move.

Mobility of CEOs

The ultimate metric of all this is how mobile is the CEO of a company. Does the CEO have to physically be present in the corporate headquarters for a significant percentage of their time? Does the CEO have to have a residence in the same city as the corporate headquarters? Is even the idea of a physical corporate headquarters relevant anymore in today’s world?

Many top executives spend an awful lot of their time on airplanes and in hotels. To some degree does it really matter where they live? After for modern global companies often to have the necessary face to face time with all the right people can’t be done from the corner office. Is the life of an executive similar to the life of George Clooney in Up in the Air?

I think if the CEO is in a fixed location then the upwardly mobile are going to be attracted to that location like moths to a flame. I think there is a strong fear in people of being out of the loop and for executives this can be quite career limiting.


I tend to think that face-to-face interaction and working together physically as a team has a lot of merit. Just breaking down the barriers to communications in this sort of tight knit environment can still be challenging.

I find that working remotely works very well for some people. But these people have to be strongly self-motivated and have to be able to work without nearly as much direct supervision or oversight.

I’m finding that the tools for communicating remotely are getting better and better and that this does then allow more people to work remotely, but at this point anyway, we can’t go 100% down this road.

If you have any thoughts on this, leave a comment at the end of the article.


Unstructured Time at Sage

with 6 comments


Unstructured time is becoming a common way to stimulate innovation and creativity in organizations. Basically you give employees a number of hours each week to work on any project they like. They do need to make a proposal and at the end give a demo of working software. The idea is to work on projects that developers feel are important and are passionate about, but perhaps the business in general doesn’t think is worthwhile, too risky or has as a very low priority. Companies like Google and Intuit have been very successful at implementing this and getting quite good results.


Unstructured Time at Sage

The Sage Construction and Real Estate (CRE) development team at Sage has been using unstructured time for a while now. They have had quite a lot of participation and it has led to products like a time and expense iPhone application. Now we are rolling out unstructured time to other Sage R&D centers including ours, here in Richmond, BC.

At this point we are starting out slowly with 4 hours of unstructured time a sprint (every two weeks). Anyone using this needs to submit a project proposal and then do a demo of working code when they judge it’s advanced enough. The proposals can be pretty much anything vaguely related to business applications.

The goal is for people to work on things they are passionate about. To get a chance to play with new bleeding edge technologies before anyone else. To develop that function, program or feature that they’ve always thought would be great, but the business has always ignored. I’m really looking forward to what the team will come up with.


We are still doing Hackathons, Ideajams and our regular innovation process. This is just another initiative to further drive innovation at Sage.

Crazy Projects at Google

Our unstructured time needs to be used for business applications, but I wonder what unstructured time is like at Google where they seem to come up with things that have nothing to do with search or advertising. Is it Google’s unstructured time that leads to self-driving cars, Google Glasses, military robots, human brain simulations or any of their many green projects. Hopefully these get turned into good things and aren’t just Google trying to create SkyNet for real. Maybe we’ll let our unstructured time go crazy as well?


I’m a big fan of Neal Stephenson, and recently read his novel Anathem. Neal’s novels can be a bit off-putting since they are typically 1000 pages long, but I really enjoy them. One of the themes in Anathem are monasteries occupied by mathematicians that are divided up into groups by how often they report their results to the outside world. The lower order reports every year, next is a group that reports every ten years, then a group that reports every 100 years and finally the highest group that only reports every 1000 years. These groups don’t interact with anyone outside their order except for the week when they report and exchange information/literature with the outside world. This is in contrast to how we operate today where we are driven by “internet time” and have to produce results quickly and ignore anything that can’t be done quickly.

So imagine you could go away for a year to work on a project, or go away for ten years to work on something. Perhaps going away for 100 years or 1000 years might pose some other problems that the monks in the novel had to solve. The point being is to imagine what you could accomplish if you had that long? Would you use different research approaches and methods than we use typically today? Certainly an intriguing prospect contrasting where we currently need to produce something every few months.

My Project

So why am I talking about Anathem and unstructured time together? Well one problem we have is how do you get started on big projects with lots of risk? Suppose you know we need to do something, but doing it is hard and time consuming? Every journey has to start with the first step, but sometimes making that first step can be quite difficult. I’ve had the luxury of being able to do unstructured time for some time, because I’m a software architect and not embedded in an agile sprint team. So I see technologies that we need to adopt but they are large and won’t be on Product Manager’s road maps.

So I’ve done simple POC’s in the past like producing a mobile app using Argos. But more recently I embarked on producing a 64-Bit version of Sage 300. This worked out quite well and wasn’t too hard to get going. But then I got ambitious and decided to add Unicode into the mix. This is proving more difficult, but is progressing. The difficulty with these projects is that they involve changing a large amount of the existing code base and estimating how much work they are is very difficult. As I get a Unicode G/L going, it becomes easier to estimate, but I couldn’t have taken the first step on the project without using unstructured time.

Part of the problem is that we expect our Agile teams to accurately estimate their work and then rate them on how well they do this (that they are accountable for their estimates). This has the side effect that they are then very resistant to work on things that are open ended or hard to estimate. Generally for innovation to take hold, the performance management system needs a bit of tweaking to encourage innovation and higher risk tasks, rather than only encouraging meeting commitments and making good estimates.

Now unlike Anathem, I’m not going to get 100 years to do this or even 10 years. But 1 year doesn’t seem so bad.


Now that we are adding unstructured time to our arsenal of innovation initiatives, I have high hopes that we will see all sorts of innovative new products, technologies and services emerge out of the end. Of course we are just starting this process, so it will take a little while for things to get built.