When I first started working as a mechanical engineer my boss would routinely double any estimate I gave him. On average my recollection is that he was pretty much always right. I usually had time to do a little more than what needed to be done so I had time to pursue some diversions.
Most of the work I did during those years was writing software to support the team. It was with some surprise that years later when I was working in software development groups time estimates were always taken at face value.
To me this is intuitively wrong, but for some reason most people don't seem to see this. There seems to be an assumption that an engineer will pad his estimates. The problem is most people don't account for the obvious. Things like vacations, and sick time are so obvious that you would think they wouldn't be missed. Given industry averages for sick time, vacations and holidays subtract 1 month per year.
Other things like meeting overhead go equally unnoticed. Typically you are going to spend, if you are lucky, 4 hours a week in meetings. That's about 25 man days a year. Subtract 1 more month.
Then there is the unknown nature of certain tasks. Let's you estimate that a task will take 4 days. Let's assume for a second that you missed something and it takes 2 days longer. Let's also say that you overlooked a quicker way of doing it and it takes only 2 days. So on average the estimate of 4 days is good. Wrong!
Let's say you were way off and it takes 16 days. There is no way that it could take -12 days. The problem is that you need to view time passage as doubling or halving of the estimation. This means that you need to do estimation on more of logarithmic scale.
So assuming an engineer estimated correctly on average to plus or minus 2 times his estimate you need to calculate the actual average of (x/2 + 2x)/2. Or x * 1.25. If they are off typically by 4x then it becomes (x/4 + 4x)/2 or x * 2.125. Or if they are off by only 10% typically (x/1.1 + 1.1x)/2 or 1.0045. Even someone that is very accurate will tend to be on average under. I've never met anyone that was nearly that good. Someone who is accurate by 2x is rare.
The thing is that the longer the time estimate, the more likely that it will be off. I have read in some places that in the early part of a project the time estimates may be off by as much as 4X. Given that the longer a project takes the more likely there will more things missed in the estimation this makes sense. Using the 2 times rule of thumb for estimating should give you good results most of the time assuming you have also accurately accounted for the obvious overhead of vacations and such. The overhead we estimated above amounts to about 17%.
Revisiting overhead a little more we realize that there is more. How much time in a day is lost to idle conversation. How much time is lost due to other factors such as computer failures, upgrades, recruiting, answering questions for support, etc. All told this adds up to a fairly significant percentage. Since most engineers don't estimate the time required for maintenance, bug fixing, documentations etc. this can easily extend the 17% to as much as 50%.
There are many other factors as well. For example the motivation factor. While it is true that people work harder when they have a deadline that is looming than when they don't most managers will overestimate the effect of this motivation. My observation is that it is never more than a small benefit and often runs the wrong way. When an engineer is faced with a deadline that they know they cannot meet, they don't work harder. They give up.
Since most of the time estimates are on the wrong side when the deadline gets to where it is looming it is almost always a foregone conclusion that the engineer will know it cannot be met. And even if the estimate is horribly short but the time frame is more than a month or so, there is no motivation from this at all. Bottom line, forget about it. It is a transitory effect that those who do good estimation enjoy toward the end of a cycle that you pay back at the beginning of the next cycle. If you don't do good estimation you never enjoy it and you have a net negative impact.
On the other hand excessivly padding a schedule doesn't work either as it you will suffer from the problem that someone else will beat you to market.
The best choice is to get the estimate right in the first place and to correct for mistakes as you go. It a terrible disservice to employees to have a schedule slip as they may schedule a vacation based on original estimates so they aren't gone during a critical time only to find out that because of slippage the vacation they scheduled falls at a very inopportune time. This makes them look bad and feel bad if they take the time, but they should take the time because it was planned for.
Other things I see happen all the time is the unplanned release. Someone in upper management sees a business opportunity and takes it. Unfortunately, the development team is already behind schedule on their current tasks. This can terribly demoralize a team. Even if the new opportunity pans out, which often as not it doesn't, one or the other or both suffer. If on the other hand you plan for this, yes I said plan for the unexpected opportunity, you will be ready to leap and can have success on all fronts.
If the unplanned never materializes, what happens? You ship early! This is a morale booster in everyones book. A team that consistently hits it's targets will start taking pride in that and will become even more productive. A team that consistently fails to meet it's targets will remain unproductive and those that are on the team that take pride in their work will start looking for a successful team.
One problem that I have never seen on any team that I have worked on, although I'm sure it exists somewhere, is an engineer that pads estimates so much that they can spend most of their time playing games. If you hire good people you don't need to worry about this. They are competitive and if one engineer thinks they can get the job done faster than another they will challenge an unrealistic estimate.
Another factor is the generally optimistic nature of people. Of course this is what you want. I was just telling a junior engineer the other day to ignore the advice of a more senior engineer that said "you don't want to touch that code". While that senior engineer may be right, many really great things have been invented by those that didn't know better. If you have a good group of engineers they are optimistic and that optimism will be reflected in their time estimates as well.
When it gets down to it estimation is an as much art as it is science. The problem I see is that problems occur most often when the science is ignored. Here I have clearly shown that logic dictate multiplying all estimates by some factor. 2 Seems as good as any and as we have seen is probably still optimistic. However, it is much closer to reality than 1.
Saturday, August 11, 2007
No One Knows How to Build Software Right
I was reading an interview with Joel Spolsky who writes Joel on Software this AM in the ACM Queue. Much of the time I agree with what Joel has to say, but something he said in the article really rubbed me the wrong way.
He was going on and on about how there isn't any real science in all the software engineering books and articles. Things like programmers having offices being based off of one weekend study done years ago and simply parroted by others over time. While he is right to say that the software engineering texts aren't giving us the best advice, he is very, very wrong to say that we don't know anything about this.
Every time I watch a large construction project or watch a show on TV about how some huge project was done and that it is either ahead of or behind schedule, I know that those projects face exactly the same kinds of issues as software development.
Someone changed the plans again, or the welds on the beams from a supplier are failing. We just released a product to manufacturing yesterday. It was quite a successful project so far, hopefully it sells well when it hits the stores. It was a project that had a very tight and short schedule due to a hard deadline.
While we pulled it off there were several things that occurred along the way that were classic problems for any business. At one point someone suddenly realized that one of our key people was going to be on vacation for a week and all of a sudden the whole project slipped.
This actually resulted in us devoting more time to development which made for a better product. However, it would have been an even better product if we hadn't switched gears in the middle. The new features that were added had a domino effect of requiring a user interface redesign. If we had known we had more time to start with we could have planned the UI better and most likely would have had time for another feature or two.
We also had last minute problems due to a misunderstanding about how we needed to branch our software to avoid problems. This resulted in a lost day due to a failed automatic build that occurred because of the side-effect of fixing an entirely unrelated bug in another product that is only loosely related to the failed product. Lesson learned.
The point of all this is that every complex project out there works under the same fundamental principals. We are dealing with people of various skill sets and they make mistakes. The projects that are completed on time are the ones with the best managers that understand principals of time management and anticipate the unexpected by putting time into the schedule for these items.
The software industry could learn a lot from the construction industry in the area of project management. Certainly, there are differences in specifics. It's a heck of a lot easier to change a broken foundational class than it is to fix a badly poured concrete foundation in a building. The ramifications of the change in the software and the resulting issues in the building can be far reaching. It's way better in both cases to get it right at the start.
Because it seems so easy to change the design of the software at any point along the way, we can get complacent about building things well. The problem is later on you have all of these things that adapt to the bad design and in a sense come to depend on it. When you go back months or years later and need to fix the bad design you are also faced with needing to identify and fix all the places that depend on it. Faced with an impending release date you may end up opting not to fix it and add yet another band aid fix that makes fixing it next time around even harder.
In any case, the next time someone says we need to study the problem of software engineering more, or that there is no science that helps us do this, slap them soundly and point them at the nearest large construction project.
He was going on and on about how there isn't any real science in all the software engineering books and articles. Things like programmers having offices being based off of one weekend study done years ago and simply parroted by others over time. While he is right to say that the software engineering texts aren't giving us the best advice, he is very, very wrong to say that we don't know anything about this.
Every time I watch a large construction project or watch a show on TV about how some huge project was done and that it is either ahead of or behind schedule, I know that those projects face exactly the same kinds of issues as software development.
Someone changed the plans again, or the welds on the beams from a supplier are failing. We just released a product to manufacturing yesterday. It was quite a successful project so far, hopefully it sells well when it hits the stores. It was a project that had a very tight and short schedule due to a hard deadline.
While we pulled it off there were several things that occurred along the way that were classic problems for any business. At one point someone suddenly realized that one of our key people was going to be on vacation for a week and all of a sudden the whole project slipped.
This actually resulted in us devoting more time to development which made for a better product. However, it would have been an even better product if we hadn't switched gears in the middle. The new features that were added had a domino effect of requiring a user interface redesign. If we had known we had more time to start with we could have planned the UI better and most likely would have had time for another feature or two.
We also had last minute problems due to a misunderstanding about how we needed to branch our software to avoid problems. This resulted in a lost day due to a failed automatic build that occurred because of the side-effect of fixing an entirely unrelated bug in another product that is only loosely related to the failed product. Lesson learned.
The point of all this is that every complex project out there works under the same fundamental principals. We are dealing with people of various skill sets and they make mistakes. The projects that are completed on time are the ones with the best managers that understand principals of time management and anticipate the unexpected by putting time into the schedule for these items.
The software industry could learn a lot from the construction industry in the area of project management. Certainly, there are differences in specifics. It's a heck of a lot easier to change a broken foundational class than it is to fix a badly poured concrete foundation in a building. The ramifications of the change in the software and the resulting issues in the building can be far reaching. It's way better in both cases to get it right at the start.
Because it seems so easy to change the design of the software at any point along the way, we can get complacent about building things well. The problem is later on you have all of these things that adapt to the bad design and in a sense come to depend on it. When you go back months or years later and need to fix the bad design you are also faced with needing to identify and fix all the places that depend on it. Faced with an impending release date you may end up opting not to fix it and add yet another band aid fix that makes fixing it next time around even harder.
In any case, the next time someone says we need to study the problem of software engineering more, or that there is no science that helps us do this, slap them soundly and point them at the nearest large construction project.
Friday, August 10, 2007
Flexible vs Inflexible User Interface
Today while reviewing a project to add customizable hot keys to our product that a couple of our summer interns were working on a suggestion was made that got me to thinking about flexible vs. inflexible user interfaces and what is better.
The suggestion was that we not allow the arrow keys to be reassigned to other commands. On the surface this seems like a bad idea. Why would we prevent the customer from making the commands that are currently associated with the arrow keys be reassigned to something else?
To answer that question is to understand a bit more about what makes a good user interface.
A flexible user interface is nice because it allows you to do pretty much anything you want. On the other hand it may make doing very common simple tasks harder to do. An example of this is what makes computers work. Machine language is a very flexible user interface. You can use it to create any software that you want. On the other hand it can be nearly impossible to deal with for a reasonably complicated system.
A higher level language is less flexible but you can be much more productive. In C++, for example, a single line can generate more instructions than a room full of monkeys.
This gets to main point, simple interface may be the best interface. A great example of this is the iPod. It does only a very few things, but it does them very well and simply.
It is tempting to take the easy way out and create a general interface for a particular task. In this case we are actually doing just that. In fact, it is easier to generate the interface to allow someone to customize hot keys than it is to come up with a set of hot keys that is really good. That's why we are doing this in the first place as it allows multiple different viewpoints of what make a good set of hot keys to exist within the same application.
However, in the case of the arrow keys we have a modal interface that allows different behavior depending on the mode based on an arrow key being depressed. These are all commands that affect navigation in a way that is intuitively associated with arrow keys.
Of course now we get into the realm of trying to predict what the customer wants. This is like trying to predict whether it will rain on Sunday or not. Pretty much whatever you decide is the desirable outcome will not be good.
The case of the arrow keys isn't nearly as compelling as many other cases. For example the tools for the blog I'm using now are quite limiting from a general word processing point of view. However, because they are so simple, even my wife can figure them out. Actually, she knows them far better than I as she has her own blog. In any case the point is that an interface is should be as simple as possible but no simpler.
The suggestion was that we not allow the arrow keys to be reassigned to other commands. On the surface this seems like a bad idea. Why would we prevent the customer from making the commands that are currently associated with the arrow keys be reassigned to something else?
To answer that question is to understand a bit more about what makes a good user interface.
A flexible user interface is nice because it allows you to do pretty much anything you want. On the other hand it may make doing very common simple tasks harder to do. An example of this is what makes computers work. Machine language is a very flexible user interface. You can use it to create any software that you want. On the other hand it can be nearly impossible to deal with for a reasonably complicated system.
A higher level language is less flexible but you can be much more productive. In C++, for example, a single line can generate more instructions than a room full of monkeys.
This gets to main point, simple interface may be the best interface. A great example of this is the iPod. It does only a very few things, but it does them very well and simply.
It is tempting to take the easy way out and create a general interface for a particular task. In this case we are actually doing just that. In fact, it is easier to generate the interface to allow someone to customize hot keys than it is to come up with a set of hot keys that is really good. That's why we are doing this in the first place as it allows multiple different viewpoints of what make a good set of hot keys to exist within the same application.
However, in the case of the arrow keys we have a modal interface that allows different behavior depending on the mode based on an arrow key being depressed. These are all commands that affect navigation in a way that is intuitively associated with arrow keys.
Of course now we get into the realm of trying to predict what the customer wants. This is like trying to predict whether it will rain on Sunday or not. Pretty much whatever you decide is the desirable outcome will not be good.
The case of the arrow keys isn't nearly as compelling as many other cases. For example the tools for the blog I'm using now are quite limiting from a general word processing point of view. However, because they are so simple, even my wife can figure them out. Actually, she knows them far better than I as she has her own blog. In any case the point is that an interface is should be as simple as possible but no simpler.
Subscribe to:
Posts (Atom)
