What do Computer Scientists Do?

Over the past couple of weeks I have had the opportunity to talk to some computer science students as well as a number of professors and others involved in a Computer Science (CS) education. This topic came up several times.

In one context it was an issue that some students get into a CS program based on some vision of what it is that a computer scientist would do, only to find out that it is quite different. The result is that these students drop out of computer science.

While learning that CS is not for you early is great, when we look at this problem from the flip side this may mean that many who would like to study CS think it is something else and thus don't even consider CS as an educational choice.

In recruiting interns I ask what they expect to get out of an internship. Often the answer is that they want to find out what it is like to work in industry.

The answer to this question is that it depends. There are many routes that one can take with a CS degree.

One can variously be a software engineer, do research, teach, be a software test engineer, be a user interface designer, etc.

Since my expertise is primarily in the area of software engineering I will limit this discussion to that area.

In a nutshell a software engineer is responsible for creating a software solution to solve a customer problem.

An example of this is the blog you are reading now. The problem being solved is to provide a simple way of posting a blog. A collection of tools that allow editing of the blog, the style, etc. have been created that make this easier than writing straight html.

In software engineering one is expected to become an expert in at least two knowledge domains. The primary domain is that of designing, writing and testing software. The secondary domain is in the problem that you are trying to solve.

For example at one point in my career I worked on printing for a CAD program. To do printing correctly I had to learn about many things including paper sizes. I ended up learning more about paper sizes for CAD than pretty much any of our customers that use the program.

Often software engineers don't spend enough time in the secondary domain learning what their customer needs. This can lead to poorly designed solutions to the customer problem.

Let's take an aside here and talk about customer need vs. want. It is a classic mistake to give a customer what they ask for. The problem is this, the customer is not an expert in the software domain. Thus when they ask for something they are asking from the standpoint of their limited knowledge of software.

By taking the time to really learn what the problem is that they are trying to solve you as a software engineer become an expert in that problem and can apply your expertise of software to come up with a solution that the customer may not have thought possible that is far better than anything they could have suggested.

On the flip side one shouldn't discount what the customer is asking for as it may be that their idea is better than what you would come up with.

Software is tightly coupled to technology. Often it is the technology that attracts students to CS. As a software engineer it is your responsibility to match the right technology to solving the customer problem.

There can be many competing technologies. For example there are many programming languages such as Java, C, C++, C#, Perl, Python, Ruby, Basic, Fortran, Cobol, Lisp, and dozens of others.

Choosing the right technology can be tough. That is where continuous education in what is available is important. It is also true that in some cases there is no best choice, or that there are many good choices. In some cases the right choice may be to use the technology you are most familiar with as that will allow you to finish the task soonest.

In other cases it may mean you need to learn something new. For example in the last year I've spent time learning more about XML, .NET, Qt, Ruby, CMAKE, and several other technologies.

Over time it becomes more and more difficult to be an expert in various technologies so often software engineers specialize in certain areas. For example I've invested a great deal of time in learning how to program in C++. While I can program in many other languages I spend most of my time using C++. As such it becomes the language I gravitate to when there is no clearly better choice.

So once you have made your choice of technologies to use to solve the problem we get into the real grunt work, writing the software. This is typically where most engineers spend the majority of their time. The decisions that you make in choosing the technology can be crucial to how hard or easy this is, but ultimately it gets down to making it all work.

Being able to write bug free, maintainable software is a skill that requires a lot of effort. Some that may be able to make good decisions about hardware or solving problems don't have the attention to detail that is required to be good at this.

Some people view software as a means to an end. They think that it doesn't matter what it looks like so long as it gets the job done. These are the guys that would be happy with a sports car that has a big engine, great suspension, transmission but the body has dents, bondo and rust.

The code you write should be a work of art. Most people who drive a car don't really care what the engine looks like so long as it is reliable and does what they expect. To a car enthusiast what the engine looks like is as important as the paint job, interior, etc.

One way to put this is that the ugly of software will eventually become visible to the customer. Usually this is in the form of bugs. Pride of workmanship, attention to detail, continuous improvement to the code are hallmarks of what a software engineer does.

This can be as trivial as formatting the code in a consistent way or abiding by a consistent naming convention or as complex as sweating the details in a very complicated algorithm.

Another aspect of software engineering is one of economics. As a software engineer you are expected to keep the bottom line in mind. Unless you are independently wealthy and can do whatever you like eventually you need to make some money. Whether you are working for yourself or a company the bottom line is important. The work you do needs to pay for the time you invest in it.

This is where you need to become good at time management. People who are poor at time management make poor software engineers or they require a lot more handholding from their managers. Managers don't like doing this in general so you need to be good at time management.

You need to know how long it takes to do stuff. I have found that most people tend to be horrible at this. I have lost count of the number of times that I have underestimated the time it will take to do a task. What this means is that you need to account for this tendency when you estimate how long it will take to do things. Here are some of my thoughts on time estimation.

There is obviously a lot more to software engineering that I can cover in this post, but the basic idea is that software engineering is about communicating between people. Sometimes people have a picture of a software engineer as someone who spends all night programming and is very non-social.

While there is a place for people that fall into this category in software engineering they need a lot of extra direction and management to keep them on track. Often these highly creative people come up with brilliant solutions to problems that no one cares about.

The really good engineers spend as much time working with people as they do with their nose in code. As a software engineer it is your job to bridge the gap between real life and technology.

Evolution and Software Engineering, "Survival of the Fittest Code"

The theory of evolution, as postulated by Charles Darwin, and further extended by Herbert Spencer, who coined the term "Survival of the Fittest" has an interesting parallel in software development.

Anyone who has worked on any code base for an extended period of time knows that some code is in a constant state of flux. Often that same code is where most of the defects occur as well. At the same time there is code that almost never changes.

In software development methods a lot of emphasis is place on things such as code reviews, design patterns, requirements, testing and many other concepts that are supposed to create good maintainable code that is relatively defect free. In practice these methods generally don't produce exceptionally better code.

I have seen a lot of well written, well designed code that is easy to read and maintain that is also in a state of flux. Usually this is only for a relatively short time, but sometimes things take a really long time jell no matter how much good design work was done up front.

I have also seen a lot of poorly written, badly designed code that is hard to read and difficult to maintain that hardly ever changes. And in many cases this code is core to the survival of the software program.

Often very good programers who take a look at the code label it as "bad code" because it is really hard to understand.

And to a large extent they are correct.

The problem is, it works, it does the job well, it's bullet proof, not by design, but because over time it evolved through many fixes to the point that it works really well.

This code survives. Often much longer than really well written code.

In the end who is to say that ugly code is bad code. Sure it may be hard to maintain, but if it does the job it was designed to do and doesn't require much or any maintenance then maybe it really is good code.

Now I'm not advocating that we write ugly code. However, this points out that the ultimate measure of good software is that it does what it was designed to do.

When we write code we should strive to make it maintainable and readable by others, but in the end if it doesn't do what it is supposed to do it is garbage that will get thrown away or refactored to do the job.

Another concept that "Survival of the Fittest Code" encompasses is that code evolves over time. In other words creating good software can be achieved by constant fixing and massaging of the code until the customer is happy with it. The fewer iterations to achieve this the more successful we will be.

Traditionally we call this the "Test and Fix" paradigm of software development.

When we look at DNA we find that a random or planned breeding produces results that are possibly good and possibly bad. The customer is the environment that the offspring has to live in. If the environment is really harsh to the offspring it may die before it reproduces. If on the other hand the environment is really easy for the offspring to survive in then it will thrive.

I have observed cases where an unintended "bug" became a useful "feature". In some cases this feature was unknown to the engineer and later on when they fixed the bug the customer gets really upset that the feature they were using is now gone.

This sort of interaction will more than likely result in the reinstatement of the feature. Since the engineer now knows about the use case of the customer they have the opportunity to evolve the software in another direction that the customer likes.

"Survival of the Fittest Code" can be an effective software development strategy.

In fact I believe it may be the single most effective software development strategy.

Just think how much more effective it could be if we combine it with just a little bit of planning and good coding habits.

Future of Operating Systems

I've been watching with interest how the operating system wars are playing out. The traditional war has been dominated by Microsoft with players like Mac and linux picking up a significant but minor role.

All of that is changing now. Today smart phones are more capable as computing devices than the hardware that these other operating systems were developed on. Which begs the question why don't they use one of these operating systems?

In fact they do. Google, Nokia and probably others are basing their operating systems on linux and of course the iPhone from Apple is based on the Mac OS which is based on unix.

They try to hide this as much as possible, but it gives us a peek at the future. The biggest reason that linux or unix work so well is that these operating systems were well designed from the start and run on very light weight hardware.

Of course Microsoft has their CE operating system and will continue to push that as well.

One thing that has puzzled me for awhile is why each vendor seems to want to create a closed system. Software developers who develop for one smart phone will no doubt want to develop for all smart phones. The typical way this is done is by supporting applications that run within the web browser. This is a great strategy, but I think it is a bit short sighted.

In talking to people and using online applications, such as the editor for this blog, I have found that while browser based applications seem to work, they have issues. The most notable is frequent updates that break things. This is fine for social networking or casual blogging, but it isn't fine for a mission critical application that you use to make money.

For example, if you are a reporter that has a daily blog, you want the software that you use for blogging to work the same today as it did yesterday. If one day you start using it and it's broken because a bug was introduced, you want the option to fall back to the version you used yesterday.

Some sites do a good job of this, but many do not.

Also, while many applications do require internet access, some do not. If an application doesn't require internet access or it only requires occasional access then it seems logical that it could be independent of the browser.

Most applications written up until a few years ago didn't have any internet connectivity.

What we have is a very large set of applications that could be potentially ported to a smart phone and possibly be very useful there. These applications are often only Windows based or linux, or Mac. Some have been ported to all the major platforms.

One of the portable application frameworks is QT. QT was recently acquired by Nokia and now Nokia is creating a smart phone operating system that supports QT. I believe this is a very astute move by Nokia. It is the kind of move that I would hope other vendors recognize and step up to supporting QT on their smart phones as well.

While every vendor would love to have their smart phone be the phone that everyone uses, the fact remains that the more applications that you can offer for your phone the better it will sell.

We saw Apple initially close the door on apps saying that they should be developed to run in Safari. They quickly changed that and we saw a huge number of apps written in a short period of time.

Now we are seeing all the smart phone vendors putting together their answer to the iPhone. If they want to beat Apple then they need to adopt a standard API so that software developers can write applications that can run on any phone. Open systems will ultimately win.

We have seen linux start to take over the OS market and ultimately I believe all operating systems will be based on linux. There is really no reason that even Microsoft can't move to linux and in fact there have been rumors that they are ultimately going that way.

Creating standard API's that are supported on all devices is reasonable and I believe necessary in order for hardware vendors to stay in business. Think of the operating system as if it were the road and the application as if it is the car. Every car can drive on every reasonably smooth highway. Certainly there are special cases, but in general if you were to build a car that could only be driven on half the highways you it wouldn't sell.

I will watch with interest the unfolding of the smart phone operating system wars. It is my belief that the winners will be those that embrace open standards, like linux and QT. The losers will be those that try to create a closed standard.

While Apple hit a home run with the iPhone if they want to remain a viable competitor they need to embrace an open standard so that developers can develop applications for all smart phones. While they clearly have an advantage today, if the combined market of other smart phone makers were to adopt many open standards they could be a minor player a few years from now.

Ooh! Shiny Thing

Over the years I have seen a lot of new technology come and go. Some of it sticks, some of it was just a bad idea. The problem is that often we as software engineers tend to get excited about some new bit of technology or a design pattern etc. This is the "Ooh! Shiny Thing" syndrome that many of us share.

While I'm a fan of new technology, it is a double edged sword. What tends to happen is that some new technology is used in one place in the code but other places that were using and older technology don't get updated. Now there are two ways of doing the same thing. Then someone else comes along and uses a third technology for the same task.

Maintaining a code base having several ways of doing the exact same thing means that you need to learn each method. While not a bad thing to learn, it takes time. Time devoted to maintenance can be costly. Given that pretty much any technology will have bugs, you are also taking on that problem as well.

Over the years I've found that the shininess tends to wear off of new technology pretty fast.

Use new technology, but be careful when doing so. In an ideal world the new technology should replace the existing technology that does the same thing. But only after it has been proven to be reliable and viable for the future.

One of your best gauges is the old-guy. Sure he may be old and set in his ways, but there are good reasons for this. It isn't that he can't learn new things. Certainly as we age it is more difficult to learn new stuff. Of course there are some that are unwilling to learn anything new. You should ignore them.

The ones you want to use as a gauge are those that are continually learning new stuff. Look at what they are learning. Let them know about what you are interested in. They may have some insight that say's it's good or bad. Take that input and use it to make a more informed decision.

Recently I've invested quite a bit of time learning stuff out of the boost library. Some of it is very good. For example shared_ptr is so essential that it has been pulled into the next C++ standard.

Other things are less compelling. Some of them while nice in theory will put a world of hurt on your compiler. For example boost::format while a really nice way of doing type safe formatting in a printf sort of way will produce a very large amount of template code. In many cases the much more lightweight stringstream is just as readable.

This isn't to say that you shouldn't use it, but what appears nice on the surface can have some very serious side-effects that are not at all apparent.

Another easy to abuse boost function is bind. In some cases it can make C++ into a write only language. Once you get a bit beyond the simple syntax to bind a function into a std::for_each the code can become almost unreadable and also is also susceptible to errors such as passing by value instead of by reference.

I'm sure that you can look at your own code base and find similar cases where someone was overly clever.

Just because it can be done doesn't mean it should.

Be careful with your new technology. Before you use new technology make sure the rest of your team is OK with it. Getting their buy-in is important and the act of doing so may change your mind about the viability of the technology.

Do What You Enjoy

While at some point in time we all need to get to the bottom line of making enough money to live and perhaps a bit more to do some of the things we enjoy, doing a job just because it makes you money is ultimately unsatisfying, unless making money is what excites you.

I happen to enjoy writing software. I also enjoy other things that are creative, such as the occasional remodeling job, blogging, cycling etc. The software thing is what currently pays the bills and as it turns out is enjoyable to me most of the time as well.

Every once in awhile you will get excited about something. Perhaps it's a new technology. When that excitement peaks you should follow it. Over the years I've seen many great ideas thrown around that eventually turned out to really big. Very often those ideas were implemented by others who also had that idea, but instead of pushing it into the background pursued it.

The reasons for this are varied. Often the idea seems impossible to implement. I'm reminded of an idea that some of us tossed around when I was in college studying mechanical engineering. We thought it would be cool to have a pen that would be able to write by itself. It would have tiny gyroscopes in it to control it's movement. We also thought that we could have the pen record what you were writing using accelerometers.

Interestingly the second part of that idea is now a reality in the "Pulse Smartpen". This pen uses a camera and tiny dots on the paper to record what you write. In fact it goes even further in that it knows where on the paper you are and so can play back stuff.

Obviously back in 1980 when we had this idea there were massive impediments to doing what we talked about and it was really only an idle conversation that none of us were even slightly motivated to try.

The thing is that there were many more practical ideas that could have been implemented given the technology of the day that I and others were sometimes excited about. But we often let the practicalities of life get in the way of pursuing them.

Or worse we knew they couldn't be done.

Many of the greatest inventions were done by those that didn't know it couldn't be done. Don't let someone else talk you out of doing something you are excited about. Push forward, even if it takes years of working nights and weekends to get the job done. While you are doing that you will learn something. And maybe that will lead to a breakthrough. When it's no longer fun put it away and do something else.

In any case always have something that you are pursuing that excites you.

America's Got Talent?

I recently watched the "America's Got Talent" show. Interestingly the winner was not the most talented person. Two singers got the top two spots, one country and western the other opera. The opera singer was in my opinion and others that know music a significantly better singer.

This got me to thinking about what makes software popular. In my experience the best written software from user interface design and from the point of view of software quality is not necessarily the most popular. The program that I work on now, "Chief Architect", is much more popular than the program I worked on before "HiQ".

The quality of the code I'm working on now started out significantly worse than what I had thought was average quality code in the industry today. Even now I don't believe the code quality is significantly better than average even with all the work our team has done to improve it. It has improved dramatically over the years as has been attested to by our customers and our software metrics.

The interesting thing is this poor quality code produced a consumer product that has several times been the #1 product in it's category. So clearly code quality isn't what makes a program popular.

The user interface design in this example is also interesting in that it had and still has numerous cases where the design is inconsistent, cumbersome and visually unappealing.

What is it that makes one product more appealing than the other? In the case of "America's Got Talent" there are a lot more people who like country and western music than those that like opera. The same idea applies to "Chief Architect" and "HiQ". "Chief Architect" appeals to anyone who wants to design a house while "HiQ" appeals mainly to engineers who are trying to solve some hairy computational problem.

This ties in with my previous blog post "Know Your Market". If your goal is to make the most profit then the idea of working on things that are really useful should be extended to include the idea of working on things that appeal to a wide audience.

Know Your Market

A recent article by Tom Demarco "Software Engineering: An Idea Whose Time Has Come and Gone?" is a must read. It makes the "obvious" point that we should be working on projects that are really useful and not working on projects that are relatively useless.

Knowing the bottom line is implicit in the article, but often ignored by some really smart people. Most of the projects I've worked on over the years, even very successful ones, have rarely done the leg work to get estimates of how much money a project could potentially make and even more rare have looked at what the project could realistically make.

Instead the project is often done because the person funding it thinks it's a good idea or the person selling the idea to them is really good at selling the idea.

I like giving engineers the freedom to create and work on things that they are passionate about. This is generally a good idea because they work on things they like and generally if one person likes it many will. However, sometimes this isn't true.

So before you start your next project spend some time figuring out the bottom line. If it's marginal go on to another idea.