Vista Woes

Yeah, I know, everyone has been whining about Vista. After more than a year of sitting on the side running XP 64 I finally decided it was time to start fixing some bugs that only occur under Vista. So I had my IT guy install Vista 32 and 64 bit along with the XP 64 so I could boot to any of the systems.

After 4 days I finally had to insist on getting at least my old XP 64 back up and running so that I could finish off some work that I had started there. At the moment I'm told that there are two versions of Vista on the machine that I can't boot to. 

I'm not sure why things took so long but it seems that a large percentage of the time was spent trying to get the installation CD Keys accepted by  Vista.

I don't know about anyone else but 4 days to add a hard disk and install the two systems seems a tad bit excessive. 

I say this while happily blogging from my Mac which I upgraded to Leopard a week after it was released without any trouble at all.

Recruiting -- Did I Miss the Dress Code Memo?

Last week I was a the University of Washington recruiting computer scientists. When we go recruiting we tend to wear what most would consider business casual. No ties, suits, etc. but definitely not jeans and t-shirts. When I showed up I felt like I had walked in on a competition for who could find the rattiest jeans contest. The thing is that some of the clothing I saw was in worse shape than what I wear working in my yard on the weekend.

While I understand that software engineers don't want to work in an environment where they need to dress in anything more than jeans and t-shirts, it is common courtesy to dress with some respect for the notion that you are representing your company and not a bunch of yahoos that just walked in from a weekend of yard work.

The thing is that this attitude of disrespectful dress basically tells me that these people aren't professionals and don't expect professional work from the students they are recruiting. While I wouldn't make a decision about hiring someone based on how they dress, I have noticed that those that dress well fall into two camps. Those that know they need an edge and those that are truly professionals. The smart ones that don't dress well tend not to be well rounded. These are people you don't want interfacing with customers.

Since a good portion of software engineering is actually getting the right requirements from customers it makes sense that you need at least some engineers that can do this. While we would probably hire someone who didn't have these skills if they were very smart in other areas and worked well with the team, we would rather have more well rounded people as they will tend to solve the right problems.

In any case it was astounding to me that so many big well respected companies seem to feel that software engineers will be attracted to a company that sends under dressed recruiters.

The "What's Important Today" Problem

In a typical day I have at least one person walk in my office and say something along the lines of "This feature is critical to our customers and we need to do something about it now." The problem is the same person will stop by the next day or next week with another really important problem. I'm almost always working on something or another and so can't just drop everything and start a new project. If I did nothing would get done.

So what to do about this? The problem is that we all have a tendency to make the issue that we ran into today a top priority. But then tomorrow, something else is a top priority. So it kind of becomes a time averaging problem. If you create a list priorities then re-prioritize, add to and remove items on it on a daily basis, then over time the highest priority items will float to the surface.

This works, as long as you aren't putting today's problem at the top of the list. So it makes sense to put today's problem in the middle of the list and then let it float. If it really is a high priority item then it will float to the top quickly.

Time Estimation Classic Mistakes

When I first started working as a mechanical engineer my boss would routinely double any estimate I gave him. On average my recollection is that he was pretty much always right. I usually had time to do a little more than what needed to be done so I had time to pursue some diversions.

Most of the work I did during those years was writing software to support the team. It was with some surprise that years later when I was working in software development groups time estimates were always taken at face value.

To me this is intuitively wrong, but for some reason most people don't seem to see this. There seems to be an assumption that an engineer will pad his estimates. The problem is most people don't account for the obvious. Things like vacations, and sick time are so obvious that you would think they wouldn't be missed. Given industry averages for sick time, vacations and holidays subtract 1 month per year.

Other things like meeting overhead go equally unnoticed. Typically you are going to spend, if you are lucky, 4 hours a week in meetings. That's about 25 man days a year. Subtract 1 more month.

Then there is the unknown nature of certain tasks. Let's you estimate that a task will take 4 days. Let's assume for a second that you missed something and it takes 2 days longer. Let's also say that you overlooked a quicker way of doing it and it takes only 2 days. So on average the estimate of 4 days is good. Wrong!

Let's say you were way off and it takes 16 days. There is no way that it could take -12 days. The problem is that you need to view time passage as doubling or halving of the estimation. This means that you need to do estimation on more of logarithmic scale.

So assuming an engineer estimated correctly on average to plus or minus 2 times his estimate you need to calculate the actual average of (x/2 + 2x)/2. Or x * 1.25. If they are off typically by 4x then it becomes (x/4 + 4x)/2 or x * 2.125. Or if they are off by only 10% typically (x/1.1 + 1.1x)/2 or 1.0045. Even someone that is very accurate will tend to be on average under. I've never met anyone that was nearly that good. Someone who is accurate by 2x is rare.

The thing is that the longer the time estimate, the more likely that it will be off. I have read in some places that in the early part of a project the time estimates may be off by as much as 4X. Given that the longer a project takes the more likely there will more things missed in the estimation this makes sense. Using the 2 times rule of thumb for estimating should give you good results most of the time assuming you have also accurately accounted for the obvious overhead of vacations and such. The overhead we estimated above amounts to about 17%.

Revisiting overhead a little more we realize that there is more. How much time in a day is lost to idle conversation. How much time is lost due to other factors such as computer failures, upgrades, recruiting, answering questions for support, etc. All told this adds up to a fairly significant percentage. Since most engineers don't estimate the time required for maintenance, bug fixing, documentations etc. this can easily extend the 17% to as much as 50%.

There are many other factors as well. For example the motivation factor. While it is true that people work harder when they have a deadline that is looming than when they don't most managers will overestimate the effect of this motivation. My observation is that it is never more than a small benefit and often runs the wrong way. When an engineer is faced with a deadline that they know they cannot meet, they don't work harder. They give up.

Since most of the time estimates are on the wrong side when the deadline gets to where it is looming it is almost always a foregone conclusion that the engineer will know it cannot be met. And even if the estimate is horribly short but the time frame is more than a month or so, there is no motivation from this at all. Bottom line, forget about it. It is a transitory effect that those who do good estimation enjoy toward the end of a cycle that you pay back at the beginning of the next cycle. If you don't do good estimation you never enjoy it and you have a net negative impact.

On the other hand excessivly padding a schedule doesn't work either as it you will suffer from the problem that someone else will beat you to market.

The best choice is to get the estimate right in the first place and to correct for mistakes as you go. It a terrible disservice to employees to have a schedule slip as they may schedule a vacation based on original estimates so they aren't gone during a critical time only to find out that because of slippage the vacation they scheduled falls at a very inopportune time. This makes them look bad and feel bad if they take the time, but they should take the time because it was planned for.

Other things I see happen all the time is the unplanned release. Someone in upper management sees a business opportunity and takes it. Unfortunately, the development team is already behind schedule on their current tasks. This can terribly demoralize a team. Even if the new opportunity pans out, which often as not it doesn't, one or the other or both suffer. If on the other hand you plan for this, yes I said plan for the unexpected opportunity, you will be ready to leap and can have success on all fronts.

If the unplanned never materializes, what happens? You ship early! This is a morale booster in everyones book. A team that consistently hits it's targets will start taking pride in that and will become even more productive. A team that consistently fails to meet it's targets will remain unproductive and those that are on the team that take pride in their work will start looking for a successful team.

One problem that I have never seen on any team that I have worked on, although I'm sure it exists somewhere, is an engineer that pads estimates so much that they can spend most of their time playing games. If you hire good people you don't need to worry about this. They are competitive and if one engineer thinks they can get the job done faster than another they will challenge an unrealistic estimate.

Another factor is the generally optimistic nature of people. Of course this is what you want. I was just telling a junior engineer the other day to ignore the advice of a more senior engineer that said "you don't want to touch that code". While that senior engineer may be right, many really great things have been invented by those that didn't know better. If you have a good group of engineers they are optimistic and that optimism will be reflected in their time estimates as well.

When it gets down to it estimation is an as much art as it is science. The problem I see is that problems occur most often when the science is ignored. Here I have clearly shown that logic dictate multiplying all estimates by some factor. 2 Seems as good as any and as we have seen is probably still optimistic. However, it is much closer to reality than 1.

No One Knows How to Build Software Right

I was reading an interview with Joel Spolsky who writes Joel on Software this AM in the ACM Queue. Much of the time I agree with what Joel has to say, but something he said in the article really rubbed me the wrong way.

He was going on and on about how there isn't any real science in all the software engineering books and articles. Things like programmers having offices being based off of one weekend study done years ago and simply parroted by others over time. While he is right to say that the software engineering texts aren't giving us the best advice, he is very, very wrong to say that we don't know anything about this.

Every time I watch a large construction project or watch a show on TV about how some huge project was done and that it is either ahead of or behind schedule, I know that those projects face exactly the same kinds of issues as software development.

Someone changed the plans again, or the welds on the beams from a supplier are failing. We just released a product to manufacturing yesterday. It was quite a successful project so far, hopefully it sells well when it hits the stores. It was a project that had a very tight and short schedule due to a hard deadline.

While we pulled it off there were several things that occurred along the way that were classic problems for any business. At one point someone suddenly realized that one of our key people was going to be on vacation for a week and all of a sudden the whole project slipped.

This actually resulted in us devoting more time to development which made for a better product. However, it would have been an even better product if we hadn't switched gears in the middle. The new features that were added had a domino effect of requiring a user interface redesign. If we had known we had more time to start with we could have planned the UI better and most likely would have had time for another feature or two.

We also had last minute problems due to a misunderstanding about how we needed to branch our software to avoid problems. This resulted in a lost day due to a failed automatic build that occurred because of the side-effect of fixing an entirely unrelated bug in another product that is only loosely related to the failed product. Lesson learned.

The point of all this is that every complex project out there works under the same fundamental principals. We are dealing with people of various skill sets and they make mistakes. The projects that are completed on time are the ones with the best managers that understand principals of time management and anticipate the unexpected by putting time into the schedule for these items.

The software industry could learn a lot from the construction industry in the area of project management. Certainly, there are differences in specifics. It's a heck of a lot easier to change a broken foundational class than it is to fix a badly poured concrete foundation in a building. The ramifications of the change in the software and the resulting issues in the building can be far reaching. It's way better in both cases to get it right at the start.

Because it seems so easy to change the design of the software at any point along the way, we can get complacent about building things well. The problem is later on you have all of these things that adapt to the bad design and in a sense come to depend on it. When you go back months or years later and need to fix the bad design you are also faced with needing to identify and fix all the places that depend on it. Faced with an impending release date you may end up opting not to fix it and add yet another band aid fix that makes fixing it next time around even harder.

In any case, the next time someone says we need to study the problem of software engineering more, or that there is no science that helps us do this, slap them soundly and point them at the nearest large construction project.

Flexible vs Inflexible User Interface

Today while reviewing a project to add customizable hot keys to our product that a couple of our summer interns were working on a suggestion was made that got me to thinking about flexible vs. inflexible user interfaces and what is better.

The suggestion was that we not allow the arrow keys to be reassigned to other commands. On the surface this seems like a bad idea. Why would we prevent the customer from making the commands that are currently associated with the arrow keys be reassigned to something else?

To answer that question is to understand a bit more about what makes a good user interface.

A flexible user interface is nice because it allows you to do pretty much anything you want. On the other hand it may make doing very common simple tasks harder to do. An example of this is what makes computers work. Machine language is a very flexible user interface. You can use it to create any software that you want. On the other hand it can be nearly impossible to deal with for a reasonably complicated system.

A higher level language is less flexible but you can be much more productive. In C++, for example, a single line can generate more instructions than a room full of monkeys.

This gets to main point, simple interface may be the best interface. A great example of this is the iPod. It does only a very few things, but it does them very well and simply.

It is tempting to take the easy way out and create a general interface for a particular task. In this case we are actually doing just that. In fact, it is easier to generate the interface to allow someone to customize hot keys than it is to come up with a set of hot keys that is really good. That's why we are doing this in the first place as it allows multiple different viewpoints of what make a good set of hot keys to exist within the same application.

However, in the case of the arrow keys we have a modal interface that allows different behavior depending on the mode based on an arrow key being depressed. These are all commands that affect navigation in a way that is intuitively associated with arrow keys.

Of course now we get into the realm of trying to predict what the customer wants. This is like trying to predict whether it will rain on Sunday or not. Pretty much whatever you decide is the desirable outcome will not be good.

The case of the arrow keys isn't nearly as compelling as many other cases. For example the tools for the blog I'm using now are quite limiting from a general word processing point of view. However, because they are so simple, even my wife can figure them out. Actually, she knows them far better than I as she has her own blog. In any case the point is that an interface is should be as simple as possible but no simpler.

Egoless Programming

The concept of egoless programming has been around for years. The thing that really surprises me is how much ego seems to creep into the development process. Unfortunately, it seems that some of the smartest software engineers out there tend to have very large egos associated with the code they write. Of course this is a generalization as I have encountered engineers that weren't very good that had large egos and those that were really smart that don't have much of an ego.

The real question is how important is the concept of egoless programming? In reality the question should be how important is teamwork. Egoless programming is all about team work. That reminds me that there is no "I" in team work...

Analysis Paralysis

Sometimes we as engineers fall into the trap of searching for the one best solution to a problem. Algorithmically speaking I suspect this is probably NP-complete. This places it in the same category as the traveling salesman and knapsack problems in terms of complexity to solve.

For anyone who is familiar with these types of problems you should realize the futility of trying to find the best solution. Instead what we should be doing is attempting to come up with a measure that says that a solution is good enough.

Almost every problem has multiple qualitative variables such as ease of implementation, speed of execution, simplicity of maintenance, etc. And many problems have qualitative solutions, such as deciding whether one user interface design is better than another.

When you are working on a problem a good guideline to follow is to ask yourself whether the solution at the moment is good enough to satisfy minimum requirements in all areas. Then ask yourself if you can see an obvious modification that significantly improves one or more of these areas that doesn't significantly erode any other areas. If you are only seeing marginal gains in any area or the work to make it better is difficult, then you are probably done.

Knowing Your Customer

One of the hardest things to do in software is to really understand what your customer wants. Most software engineers make the mistake of assuming that what they like is what the customer will like. While this is often the case it is often not the case as well.

You can probably make the assumption that if you don't like something that some customers won't like it either. You can also assume that if you like something that some customers will also like it. However, whether this is a large or small percentage is hard to tell without spending some time researching the problem.

Another thing to keep in mind that many very creative people are terribly disorganized. Engineers like to have nicely orthogonal solutions. Everything lines up nicely and is consistent. The problem is the world isn't this way. It is often haphazard, disorganized, inconsistent etc. Creative people can often be difficult to deal with and make conceptual leaps that don't make any logical sense.

A foolish consistency is the hobgoblin of little minds. -- Ralph Waldo Emerson

Don't get me wrong. Consistency is often a very good thing as it tends to simplify the learning process. But on the other hand it may get in the way of efficiently doing your job.

My brother runs a support group and one of the things he focuses on is an accurate statement of the problem. An accurate problem statement is key. It's as obvious as the nose on your face that if you solve the wrong problem the customer won't be happy. So you must first understand the problem.

Here we get into requirements gathering. If you state a problem incorrectly you will not be able to produce an accurate set of requirements. If you think about it a little you will realize that requirements are simply another way of expressing a problem. Rather than stating what isn't happening, the problem, you are stating what should happen, the requirements.

A single problem may result in multiple requirements. And a single requirement may partially or totally solve many problems.

Once you know what your customer wants, you aren't really done yet. Let's take a simple example. When working with CAD there are quite a few large format paper sizes. For example Arch A, B, C, D and E. D is very commonly used. So if you talked to several customers you might come to the conclusion that you only need to support D. Or if you are having a particularly lucid moment, you might say that we should support all these sizes.

You would be wrong. The fact is that there is also an ANSI set of paper sizes. An ISO set of sizes and a variety of others. You will not find out about these by talking to your customers. You must become an expert in paper sizes in order to design a system that will work well for pretty much any customer.

The point is that when you start working on software you need to know more about the topic you are working on than most of your customers. When you do this you will be able to write software that not only does what you customer asked for, but also does what your customer doesn't know they need yet.

Now you are probably saying that you don't have time to do all this research. The problem is that if you don't do the research, you will be revisiting your design later. The less you know about a problem the more likely it is that your design will require significant rework. For anyone that has maintained legacy code, you will realize that not only will you need to make things work write, you will also need to make sure that the way it did work is still supported or at least provide a migration path.

The redesign may take as much time as the original and the migration path will probably be more work on top of that. If it took you a week to do the original project and a week or so to do the rework plus half a week doing research to figure out what you should have done then spending half a week doing the research to begin with and not having to do the rework will save you at least a week later on.

If on the other hand you fail to do the research the second time you are likely to repeat the process several times until you get it right. Each time incurring additional overhead to support all of those legacy migration paths. Eventually, you will have spent at least a week in the school of hard knocks learning about why your original design was wrong.

Not only that, you will suffer from less commercial success because the customers aren't happy with what you provided them.

So bottom line talk to your customers. Really understand what they want and how they want to do it. Read about what they do from whatever sources you can obtain or get training from others. Perhaps even a course at your local college would be in order. In any case, when you get done with all of this you should be in a position to not only know what and how your customers work, but also be able to teach them a thing or two as well.

Nothing makes a customer happier than having a solution that anticipates their needs. Nothing frustrates a customer more than having to jump through hoops to do very common tasks.

So, know your customer better than they know themselves and you will never fail to produce top quality solutions.

Subversive Programming

I program mainly in C++. As such I know the language fairly well. If you are beyond the beginner stage of learning C++ you know that const correctness is important. It is a contract between you and the reader of the code that if they call this method they will not modify the data of the object in an observable way using the public API of the class.

Observable is important here because in some cases it is important to be able to have data that is modifiable in a class. The case that I find most compelling is when you need to cache a calculation for performance reasons.

Now there are ways around this using perfectly legal C++ syntax. I once saw some code where this was done and even proudly commented as "fooling the compiler". The problem is that the compiler isn't the only one being fooled. When this sort of thing is done you are basically lying to the poor maintainer of the code. If this happens frequently then code maintenance is more about navigating a field of land mines.

Most of the time engineers aren't intentionally subversive. In the example I cited this resulted from not knowing that this was subversive. A great engineer will actively learn what is subversive and endeavor to avoid it.

Be a better engineer by learning about subversive coding techniques and expunging them from your vocabulary. The following link is a good start http://mindprod.com/jgloss/unmain.html.

Endurance

Often software development is more about endurance than being smart. While it really helps to be smart, there are many things that don't come easily. I often see young software engineers wanting to give up on a project because they aren't making the kind of progress they would like to see. They feel like they are wasting time and especially for someone who is used to being successful, it makes them feel like they are failing.

I like to ride my bicycle, often for several hours at a time. In a lot of cases I end up overextending myself and the last few miles on the road back can be very difficult. The same can be said of software. Especially, if you are working on a project and you get down toward the end and you find that there is a very difficult problem to solve that you didn't anticipate.

It is during these times that often the successful engineer isn't the smartest one, but the one with the most mental endurance.

When I ride my bike I like to contrive to make the ride back the easy half. So I'll ride uphill to start or into the wind. Sometimes this doesn't work out, but if you can tackle the hard stuff first, you will be in a better position to finish the project. And, if it turns out that the hard stuff can't be solved, you call it quits and you have lost less time.

Endurance isn't everything. Sometimes you need to know when to take a break. I will very often figure out a nasty problem on the drive home from work. The simple act of taking my mind off the problem or being distracted by other things seems to help my mind figure out problems. If you find yourself struggling with a problem well after everyone else has gone home, go home. You are not doing anyone any favors by continuing to beat your head against a problem when you are tired.

Just as 30 minute break to eat and drink and generally relax in the middle of a bike ride can make the rest of the ride go a lot easier, the same is true for your brain. It works better when it is rested.

Another way to make things easier to deal with is to break things into chunks that you can deal with. When I'm riding, I will set my sights on the next easily achievable goal. It might be the top of the next hill, the next mile, the halfway mark or so on. When you are working on a project you should keep the larger picture of where you are at in mind, but focus on the next thing. It may seem that months or years of time devoted to a project is insurmountable, but if you take it one easy step at a time you will soon start to see the results.

Sometimes I see engineers that are more like sprinters. These are the guys that are working really long hours in the last week or so of a project to get it done. My question is where were they in the first few weeks? You obviously don't want to get burned out early so you have to pace yourself, but there is no reason to work excessivly hard at the end of a project, unless you didn't plan well. I never liked cramming for tests either. I figured if I didn't know it by the time I took the test then I didn't know it and what's the point of taking the class in the first place.

So the next time you feel like the project you are working on is never going to get done. Take a step back examine the long term goals, look at where you have been and then refocus on the next step.

When to Worry About Performance

I spent the day wrestling with slow code. Most of it was very easy to spot where the slowness was. I like using the poor man's profiler to spot these. What's the poor man's profiler you ask? Basically it boils down to using the debugger to break already executing code. The theory is that in all probability it will break in or near the code that is taking a lot of time. It's quick and should work with nearly any debugger. The assumption, of course, is that you can make something slow enough to start it and then cause it to break before it gets past the slow code.

In any case that isn't the point of this post. The real question is when should you worry about performance. One developer I worked with many years ago said this. "Make work, make it work right, make it work fast." I like this quote but I add to it, when you are working on an algorithm take 5 minutes and pick a fast one.

Most of the time I encounter slowness when there is an O(n^2), O(n log(n)) or O(n) algorithm in use when an O(n) or O(log(n)) algorithm is easy to choose at development time. The problem is that often that slower algorithm is excessively difficult to replace. Often, the slow algorithm isn't discovered until the customer pushes the limits of the program.

I lost count many, many years ago of the number of times where the engineer came to the conclusion that no one would ever need more than n of these things. Then the customer comes up with a case that the engineer didn't think about where 10 * n is a small number.

Rule of thumb. Think of the largest number of items that you can imagine that a customer would ever need then multiply it by 100 or 1000. Then do your design based on that assumption.

Another idea that makes my blood boil is the whole Moore's Law thing. We have effectively seen the end of this about 4 years ago. That was when memory speed stopped keeping up with CPU speed. The words I hear are just buy a faster computer. Or in a year or two it won't matter.

The problem with this is that if computers double in speed every 18 months or so, you are talking about an algorithm that is O(n log(n)). If you are throwing O(n^2) algorithms at this they will swamp any CPU speed gains you get. Couple this with Moore's Law not keeping up with expectations and ever more efficient ways of increasing n and performance suddenly becomes a huge issue.

The current trend in multi-core processors is also interesting. However, if an algorithm is O(n) or worse on memory access with only 1 path to memory, no amount of additional cores will help.

So the bottom line is that if you write any code that is essentially unbounded on n, spend a few minutes coming up with an algorithm that is optimal for large n.

I leave you with one caution. If n is only very rarely large and is very frequently on the order of magnitude of 10, then that better algorithm may in fact be slow if you are working with a large number of items.

Accountability in Software

I must confess that I am a Mechanical Engineer by training. Everything I learned about software engineering has been on the job and a lot reading and a fair amount of trial and error. When I first started my career as a Mechanical Engineer the group I was working with had written some software as part of a NASA contract. Since I had an interest, and as it turns out, an aptitude in dealing with programming, I naturally migrated into supporting this software. This software was written in Fortran (ugh)... At the time I thought it was a great programming language. In comparison to using Basic on a TRS80 (a whole 4K of memory) this was pretty much a step up.

As a Mechanical Engineer I had to take the Fundamentals of Engineering exam when I graduated. Depending on the field that I went into, I would have eventually needed to take another exam to become a licensed professional engineer. As it turns out, I took a different route and never crossed that bridge.

Should software engineers be licensed? Theoretically, no. Practically, maybe. The issue is really accountability. Most good software gets that way because of several factors. Mainly by the simple fact that bad software doesn't usually make money. Or if it does, eventually someone else will come along and do a better job.

The other way that software gets good, is because of the person who wrote it made it good.

Engineers are licensed because of loss of life due to things like bridge failures. Even so, there are still failures that occur even though an engineer signed off on the design. The real bottom line is whether you can make money or not. However, licensing does leverage into the idea that an individual will not want to sign off on something if their neck or license is on the line. Given the large number of people out there that picked up a "Dummies" programming book and are now "experts", I wonder if maybe this would help to separate the true engineers from the code jockeys.

Don't get me wrong, just because I have an engineering degree doesn't qualify me as a software engineer, and just as before professional mechanical engineering degree programs existed there were many who were highly qualified engineers. I feel that we are in a similar time frame where good software engineering training is more the exception than the rule.

I do a fair amount of college recruiting. However, our local university doesn't have a software engineering degree. They offer computer engineering and computer science. The good news is that these programs are part of the college of engineering rather than the college of science. They have a really nice senior design program that emphasizes software engineering.

There are still gaps though. It never ceases to amaze me how little a CS student knows about practical software engineering when they graduate. Their grasp of the languages that they used is usually very weak. The ones that actually do know something about it spend a lot of time learning on their own. This is a double edged sword though.

If you learn on your own you can often build very bad habits that are hard to correct later on. In addition, most of the professors teaching CS haven't spent that much time learning languages. This leads to huge gaps of knowledge or misplaced emphasis on what is important in a language.

I happen to have spent many years writing in C++. I consider myself to be advanced but by no means an expert in C++. I am continually learning something new about the language. In any case I digress from the point of this blog.

The point is that even if we did decide that licensing is a good thing, which it probably is, I wouldn't trust the knowledge base that might be required for licensing. Perhaps the best solution in the short term is to require software teaching credentials to include 10+ years of industry experience. If we get the professors on the right page in terms of what is important knowledge in the industry, then the rest will follow.

This will naturally occur over time, but will probably take several generations for the profession of software engineering education to catch up with industry.

On the other hand, it will take some pretty hard knocks for some of the industry to catch up with what is important. When you have pressures to push a product out the door to capture market share, or just to keep a company from going under and no personal accountability, we get varying results.

This too will naturally get corrected over time as these products and companies fail.

The bottom line though, is that you as a software engineer need to take personal responsibility for what you write. Become a craftsman. Make your code and the end product a thing of beauty. Write code that is defect free. Make sure you know your customer. Take responsibility for your code.

I doubt that this will become the norm, so when licensing becomes necessary, and you can't pass the test, do us all a favor and go work at a fast food place where your sloppiness and lack of accountability will be expected and rewarded with the salary that you deserve.

On the other hand, if you really are a great software engineer, no worries...

Rule of Hand

I have a guideline that I call the "Rule of Hand". If a function takes up more lines than you cover with your hand it's probably too big.

I'm reminded of a post that I saw just the other day that was a very long paragraph. Probably about 4 times longer than this post. It was very difficult to read simply because it put too much information into a single spot.

The idea here is that small bits of information are much easier to read and understand.

One final comment is that this is a guideline and as a guideline there are always exceptions.

Code Comments

When I first started writing software the common mantra was to put in lots of comments. At times I did this and other times I didn't. During most of my career I have worked on code bases that someone else started. In that time I have rarely seen code comments that really helped. Often I see comments that basically restate in English what the code is already saying. Even worse the comments tell me that the code does something different than what it actually does.

In more recent years I have found myself reading comments as a last resort. Usually this is after I have determined that the code isn't working the way it seems like it should. At this point I'm looking for a clue as to what the original author, sometimes myself, was thinking when they wrote the code.

So, long story short, if you feel like you need to write a comment to clarify some code, you probably need to rewrite the code so that it is more readable. Every once in awhile I get someone who is asking me to put in comments to clarify what is going on. As it turns out, usually the problem is that what I thought was obvious when I wrote it wasn't. This makes it very difficult to comment code effectively.

Which gets to the point of my post. Learn to read code. No matter what language you are using, you need to be fluent in both writing and reading the language. To be really fluent you need to read code written by a large number of writers. What you will find is that there are certain dialects that creep into language. The more code you read the better you will be at figuring out what is wrong with a program.

Ultimately, the speed at which you can correct problems in your and others code will translate into how valuable you are as an engineer.

The other side of this is to write self documenting code. This usually involves reasonably descriptive variable names. Beware of too long names as this can also detract from readability.

Package up logical things into functions or methods that do only one thing. That way it is really clear what is happening.

Mostly just use common sense and keep things simple.

Software Engineering Analogies

I have seen many comparisons to software engineering that like to make software development fit into the engineering metaphor. Sometimes it goes like this, a software engineer designs the software and then passes it on to a programmer to implement. The analogy is that software is like constructing a building or a bridge or something like that. The architect or engineer doesn't do the construction of the building therefore the software architect doesn't do so either.

It sounds good and it actually does work. However, almost all software isn't developed using this "sound" method. Why?

Because it doesn't work that way. A whole industry of software engineering methodology is based off of the assumption that the analogy is correct. Which makes a lot of software engineers and non engineers who read this stuff think that the way we develop software is wrong. This doesn't do anybody any good. Often you get people outside of the development team telling you that your methods are wrong. When or if you try to apply these methods they tend to cost too much and don't get significantly better results. In some cases the results are worse.

I submit there is a better analogy. In order to understand the analogy, we need to first understand what the end product of a software engineer is.

Most people will state the obvious, the end product is the executable. It might be an embedded system that runs a cell phone, or a sales contact management database or a software program like the web browser you are using.

While this is the deliverable of the company producing the software, it isn't the deliverable of the software engineer. The software engineer delivers code that can be compiled into the final product.

So what does an architect deliver that is designing a building? Not the building. The deliverable is the working drawings. They stick with the process through construction and continue to fix the drawings as problems during construction are identified.

If you haven't guessed already analogy I like is that the software engineer is like an architect. Just as an architect interfaces with customers to define the requirements for a building, the software engineer also interfaces with customers to define the requirements for the software. Just as an architect will iterate through several designs working with pen and paper or a CAD system a software engineer does the same. The working drawings can be reproduced at will, just like software.

This analogy makes a lot more sense when you realize that most software is implemented by the designer of the software. This works really well when there is just one person and when you are working in a small group of engineers. It also scales pretty well to larger groups, but does rely on more documentation to keep everyone on the same page.

In any case, I think if you start your analysis of how to develop software with a faulty analogy, you get faulty results.

Assuming you accept my analogy that a software engineer is like an architect, an examination of how architects work would be in order. In fact when you look at this model you find that most of the stuff that an architect does a software engineer also does. You interface with your customer, you show them preliminary designs, you take feedback and iterate on them.

It is just as easy to change a CAD drawing as it is to change software so iterating on the software is something that can be done. Just as architects leverage building blocks like CAD details that they have placed into a library a software engineer leverages code reuse.

Another aspect of this is that an architect needs to make all the working drawings in a set look consistent even if several people are working on them. Have you ever had trouble with a team member not sticking to coding standards because they liked their style instead of the team standard?

It starts getting you to thinking, if the end product is the code, then there should be some pride of workmanship in the code. A mechanical engineer designing a high end car will spend a lot of time making the engine compartment easy to work on and to some extent it will be a work of art. If you ever get a chance it is interesting to take a look at the large steam engines that Henry Ford used in his factories. These aren't ugly, they are works of art.

If you look at code and it looks like it went through a blender, you can tell the person who wrote it was less concerned with the implementation than simply getting the job done. Back to a set of architectural drawings. A sketch on a napkin may suffice for a very simple project, like building a deck, as long as you are the one doing the construction. But the more complicated the design and the more people involved, the more accurate and clear the working drawings have to be.

Another consideration is the idea of writing in pseudo-code. This is useful if you are going to be writing in assembly, but these days our languages are more expressive, readable and precise than pseudo-code. I liken this to writing the outline of an English novel in French. The same idea applies to other tools like UML etc. While I do believe UML has value when communicating with someone who doesn't understand the language you are writing in, it really doesn't help a lot when communicating to a expert in the language.

Don't look to hard at this analogy or you will start to find cracks in it. As with all good analogies, they break down after a certain point.

So the next time you think about software engineering, think about the architect.