Unicode anyone?

As you may have guessed from my previous post I've been digging into Unicode. One may think it odd that I haven't done this sooner, but when working on legacy applications it is hard to justify supporting more modern things when you spend all your time trying to make legacy software better.

Fortunately, I'm now in a position of writing fresh new code that can be built on a strong basis. As I thought through many of the things I wanted to do I fairly quickly came to the conclusion that moving forward I needed to write things using Unicode.

Should be easy. Unicode has been around for 20 years now so, unlike the mere 3 years since adoption of C++ 11, everything should work and the skids should be smooth and well greased.

Of course not!

Windows seems to like UTF-16. That's fine. Other operating systems seem to like UTF-8 or UTF-16.

Out of the box on Mac and Ubuntu I can do something like the following:

    std::cout <<  u8"Α-Ωα-ω\n";

Of course on Windows not only is the u8" syntax not yet supported, but even if you get a std::string with a proper UTF-8 encoding that won't work either.

Turns out that I can set the cmd.exe console encoding to use UTF-8, and it works great if I use printf for my string, but std::cout doesn't work. To top things off Microsoft decided to explicitly disallow UTF-8 in their std::locale implementation. So I can't tell std::cout to send things to the console as UTF-8. Instead it appears that I will need to use printf or find another obscure way of outputting Unicode in my unit test console based application.

I'm not sure what this means, but it does give hope to those worried that the machines will take over. It will likely take them several decades to figure out the mess we have made with software.

g++ is dead

Lately I've been working on a project that involves using C++ to develop libraries for Windows, Mac, Ubuntu, iOS and Android, with an eye toward quality and portability.

Recently I decided that I needed to build a Unicode class to support a forward thinking basis for things. Of course C++ is generally not Unicode friendly, but it isn't all that unfriendly either. Especially with C++ 11.

So I dig in and start learning and coding and came up with a first pass of my class on Windows using Visual Studio 2013. So far so good.

Now let's go over to Ubuntu where where we have compilers like g++ 4.8 and Clang 3.3 that are reportedly more compliant than Visual Studio.

The first thing I notice is a missing include . The next thing I notice is that the API for std::basic_string that I was trying to emulate is well, just plain wrong! Not even close! 

I start digging and find that the GNU standard C++ library is about 6 sigmas off of supporting C++ 11. How can you claim your compiler is C++ 11 compliant but the library is not?

Fortunately, I eventually got to the point of using the new libc++ library which Apple switched to awhile ago and things work, in clang. But when I try to build for Android I can't get the experimental clang support to work. Probably because of 2 or 3 really important steps that I missed, but nonetheless I gave up.

Working through the problem I manged to hack around the deficiencies in the GNU standard library with regard to std::basic_string, but got nowhere with trying to get the codecvt stuff working. Turns out that the other option of using iconv isn't built into Android. I would have needed to jump through several major hoops to compile it and hook it in.

I eventually wrote my own conversion routines between UTF-8, UTF-16 and UTF-32.

So to the point of my post.

It is 2014. C++ 11 was ratified, in March of 2011 and formally adopted in August of 2011. So why, 3 years later, is the GNU library languishing? I would not have expected full support on day 1 or even 1 year after adoption, but 3 years? And the thing that is particularly annoying is that some of the stuff that isn't supported is trivial to correct.

My sense is that the ideas behind GNU and the "Free" software thing are fallacious, which I speculate is a major reason the Apple decided to ditch GNU and that we now see that a more open "open source" implementation in Clang and libc++ is now displacing GNU.

While I will likely continue to keep an eye on the GNU C++ compiler for awhile, it is now a distant #3 in my compiler recommendation list after Visual Studio and Clang. This leads me to the prediction that G++ will be a footnote in the history of C++ compilers. Although they could turn this around, but I'm not going to hold my breath.

Software Quality Ceiling

Over the past 3 years I have worked on a project porting a large legacy C++ code base so that it will run on Mac, Windows and in the future other platforms. In doing so we used a number of third party toolkits including Qt.

Portable libraries are nice to use when they work right. The problem is that sometimes you run up against quality issues with the libraries. Things that work right on one platform and not on another or things that simply don't work right on any platform.

Qt is a nice toolkit, but it has many issues. We worked closely with the support team at Digia to correct many issues, but with every software update a new set of problems are added. This means a long and laborious process of first creating a reproducible bit of code, reporting it, sometimes needing to convince the support people that it really is a bug, getting a patch, trying it out and often iterating several times until the patch is correct. Then we hope that the problem is corrected in the next update, which it often is, but about as often has new issues.

Of course Qt is not the only problem. Yesterday I was investigating the new threading library for C++ 11 in Visual Studio 2013. Everything was going along just fine as I was working on a simple ThreadPool class. It worked great except that it would hang when shutting down. I finally did a search and ran across the following http://connect.microsoft.com/VisualStudio/feedback/details/747145. A bug report against Visual Studio 2012. Marked Closed Deferred.

I'm able to workaround the problem by ensuring that my thread pool shuts down before returning from main, but now I'm faced with how to write a class that logically would be global that will correctly shutdown in Visual Studio. I have yet to test it on Mac or Ubuntu, but I'm fairly confident that I won't have the same problem there.

Over the years I have encountered numerous code generation problems with compilers. The good news is that I haven't run across anything recently. But in the back of my mind I continue to worry that one may show up. This is really bad because unless you actually run the code affected in the manner that would exhibit the bug you may never know you have this problem.

Then there are things like the openGL drivers for video cards. In some cases something works fine on one card but not on another. At that point you are faced with answering the question, did I do something wrong? Sometimes I did. In some cases the fact that it worked on one card was wrong. But in other cases I used the API correctly and there is a bug in the driver.

Then there is the unspecified behavior clause in languages. In C++ there are many places in the standard where the behavior is listed as unspecified or compiler dependent. In these cases I would prefer that it didn't compile or at the very least it should crash when I try to use it. In my recent porting project I ran across several cases where the compiler on the Mac crashed on badly written code, but on Windows it didn't. I was happy to find and fix those issues. But they should have not worked in the first place.

All of these issues result in a quality ceiling. My goal is to write code that is 100% bug free. Occasionally, I achieve that for simple cases. More often I have a few bugs that testing should eventually uncover. But if I'm working with tools, compilers, libraries, operating systems, etc. that are buggy I will always be faces with the fact that my software will be of a lower quality than I'm theoretically capable of writing.

As an industry we need to figure out how to break through the quality ceiling.

With that I close, because I have no answers.

Optimizing C++ Build Times

I've been fighting improving build times under C++ for years and over that time I've managed to put together a few good practices to help.

1) Use good header inclusion isolation.

Use forward declarations as much as possible.

Use the PIMPL idiom to provide better isolation.

Avoid include only libraries. Some are marginally OK, such as most of the STL. Others, especially certain boost libraries, can create enormous burdens. Wrap these using PIMPL or "has-a" instead of inheritance.

Use good encapsulation by splitting your classes into reasonably small DLLs. This reduces link times and helps enforce other good design practices.

In theory this should be all you need.

But there are some other things that work well.

2) Use IncrediBuild if you are building under Windows.

There are other distributed build systems as well, but at the moment I've not had any luck improving over local builds on today's multi-core hardware.

3) Use multi-core machines.

The more cores the better.

4) Use SSD drives to eliminate disk bottlenecks.

Depending on the amount of code you are compiling you may not need an SSD drive. However, in my current project I've seen a 30% improvement in build times.

I also have found that when using IncrediBuild in combination with an SSD can really improve the throughput allowing you to use many more machines. I've recently done tests where I ran up to 100 agents seeing significant improvements between 50 and 100.

5) Figure out what the hardware bottlenecks really are.

Memory speed. This is still a potential bottleneck in today's hardware.

Memory amount. This is easy, if you are using more than about 3/4 of your memory typically while building, get more.

Disk speed. The disk cache can get into a swamped state where it takes as much as several minutes to finish writing all the cached data. This can lead to randomly long link or compilation times. Use SSD drives to take care of this.

Network speed. If you are using distributed builds then go to at least a gigabit network. Beyond that I'm unsure that there would be any gain. Below that it is clearly a bottleneck.

CPU speed. Faster is better unless the bottleneck is memory speed, disk speed, or network speed.

Number of cores. Since compilation can be highly parallel more is better, unless memory speed or disk speed is an issue. Usually more cores means faster memory so it is normally just disk speed that can slow you down.

6) Turn off active virus scanning.

This should be obvious, but every .obj, .dll, or .exe that you create would get scanned thus slowing your compilations considerably.

I find it amazing how often this is overlooked.

Does this work? Yes. I've taken a build that was pushing 15 minutes and reduced it to about 8 using the methods in 1). I've take a build that was pushing 10 minutes and dropped it to about 7 using and SSD drive. I further cut the same build down to about 4 minutes by taking the number of IncrediBuild clients from 20 to 100, 5 minutes for 50 clients.

Slow builds are productivity suckers. The slowest build I ever saw was 30 hours. I only did that once. At one point I worked on a project that went from 5 minutes to 1 1/2 hours in year.

Somewhere not far past 5 minutes will put you into a state where flow gets interrupted. If you find your flow getting interrupted often, you will be far less productive. Don't let that happen.

Ideally you should strive to keep your builds under 1 minute. I believe this is possible for typical builds with the right encapsulation of data regardless of the total size of your project. While an entire system built from scratch could still take days, your normal builds should not need to take long. If they do, you are wasting resources.

What is the Best Software Process?

In order to write good software you need some sort of process. As it turns out it doesn't really matter much which process you choose but there are a few key elements to a good process.

1) It must focus on your customer.

The first thing to get right is to solve the problem that is most needed by your customer. This can be a bit hard to determine because in most cases the customer will tell you what they think you can solve rather than what you can solve.

This means that sometimes they want you to do the impossible and you need to scale back their expectations or that they have already scaled back their expectations and what they really need to be solved is easy to do.

There is a real art in figuring this out but it is mostly about spending enough time understanding what it is that your customer does. This often means you need to learn something new outside the domain of computer science.

To me the chance to learn something new is the best part.

2) The process must be iterative.

No matter how you end up doing things there needs to be some iteration involved. This has led to many different process methodologies. All the best ones use some sort of iteration.

Basically this recognizes the fact that you can't create a great design without writing some code. Eventually you need to rewrite the code to deal with changes. Sometimes you get it right to start with, but usually that is for a trivial problem that you can already write the code for in your head.

3) The process should be collaborative.

Writing code by yourself is OK, but two heads are better than one. Code that is written by and maintained by multiple people will get better or worse over time depending on the team.

If it is getting worse then you have a problem with your team. Either you or your teammates need to learn to work better together. Almost always it is you that needs to improve. How can that be you ask?

Well what it boils down to is that most software engineers think their code is better than their colleagues. While that may be true in some cases it is almost always true that you can learn something from your teammates about writing better code.

Looking for that and adopting the best of others will make you a better engineer. When you get to this point then it is probably true that your teammates need to improve. They will be a lot more likely to adopt your good habits if you have first adopted theirs.

4) The process must include testing.

This is obvious, but there are some good and bad ways of testing. While you should test your own code it should always be tested by someone else. Testing should be independent and done by a team that doesn't also do development.

The testing team should develop somewhat of an adversarial role with respect to the code. However, it should be more of a friendly rivalry. The goal of the developer is zero bugs. The goal of the tester is to find many bugs.

Failure to achieve either of these should be a sign that the respective team has room to improve.

5) The process should include documentation.

The first documentation should be written by the software engineer. It should be written, no videos, and it should clearly state what the feature should do (requirements) and how it works (specification).

Each of these can be used to produce other documents, a test plan, user documentation, etc.

6) There needs to be accountability.

This can come in many forms. For example source control maintains a record of who did what and when, a bug tracking database records defect and is used to assign them to the appropriate party, a schedule gives management, marketing, and other stakeholders a prediction as to when they can expect delivery so that they can plan accordingly.

7) Return on Investment Must be Estimated.

Technically this goes under accountability, but it is so often ignored that I call it out specifically.

Software that costs more to develop than it saves should never be started. For example if it takes $100,000 to automate a shipping process where the savings is only $20,000 over the next 5 years then there is probably better things to do.

Not only is this a losing proposition, it is likely that the project will get canceled making it an even bigger waste of money and time. It will also piss off the engineers that work on it.

If you are an engineer starting work on a project like this then make sure management knows it is a waste of resources. If they don't listen, move on. You don't need the aggravation, and there are plenty of software development shops around that do work on profitable projects.

On the other hand if adding a feature for $100,000 will add $1,000,000 a year in revenue then it is most likely a good thing to work on.

Spend your development resources where they will make you the most money. It's good business and it's good for the team.

8) Focus on the End Game.

Often from the time of feature completeness to release can take several weeks. Often one thing, such as finishing documentation, web site, marketing materials, etc. can end up taking a surprisingly long time. These needs can often push back the date of feature completeness by a month or two for a given release date.

Gantt charts are very helpful in planning your end game to ensure that everything that has dependencies on the software being complete are done.

9) Use Technology Only When Appropriate.

There are some technologies that are pretty much a given. For example you need a compiler.

Other things should be a given such as using source control and a bug tracking database.

Then there are technologies that may or may not be helpful. Most things can be done manually. Such as managing requirements, specifications, etc. Tools exist to do these things, but are not necessary to write software. Use them only if they really help you.

If they are black holes where information is poured but never used then get rid of them as they are just taking up time.

10) The Process Must be Simple.

If you can't explain it in 10 minutes to your mother and have her understand it, then it's too complicated.

A complicated process will fail. It won't get buy in from your team. It will be interpreted differently by different team members. It can take a long time to be fully adopted.

11) Use Iteration to Improve the Process

With each release you can sit down and examine your process. Figure out what worked well and what didn't work so well. Focus on the areas that didn't work well and figure out how to make them work better.

One thing that happens frequently over time is that as things improve you hit a release where a the quality slips. This tends to come from complacency. You have to keep working to improve.

One way to deal with this is to keep raising the bar. Recognize that you can always improve some part of the process as there is not a perfect process.

12) Adapt Process to Your Needs.

Don't just adopt a process. No process is perfect and every team has slightly different needs.

For example, a young inexperienced team can greatly benefit from pairs programming while a highly seasoned team would only be slowed by pairs programming. Some would argue that point with me, but 30 years of development experience tells me otherwise.

Adapt process ideas to your team needs. Don't just blindly adopt a process as it won't work if you and your team don't understand why you are doing things in a particular way.

Conclusion

With a little effort you can write good software. With a lot more effort you may be able to write slightly better software. But if it doesn't make your company money, it's a bad process.

The traveling salesman problem is a great example of a problem where achieving the perfect solution can take so much compute time that the salesman could have gone with a quick less than optimal solution and be back from his trip before the computation of the perfect solution is finished.

Don't let your process become such a large cost that you can't get the job done.

Boost Threads and the Data Hiding Zoo

In C++ the concept of data hiding is implemented via private data. This is a fairly nice concept but it is really more like putting the data behind bars and telling the viewer not to touch it.

Very much like visiting animals at a zoo you can look at all the data and so long as you don't touch it everyone is happy.

However, this also means that we have to see the data even when we don't want to. This has a lot of detrimental effects, not the least of which is that it adds to compile times because of C++'s lame include model.

C++ and for that matter pretty much all languages give us the ability to not only hide the data but to make it invisible. The simplest and most effective method is to put the data that you don't need to know about into a DLL and then provide a simple function call to perform the action that you want to make public.

A good example of this is a dialog. Let's assume that you have a class that has an associated dialog that is designed to edit the class. All you need is a simple function call that takes a reference to the data that you want modified. Then the only public interface to the dialog is a plain old function call.

In some recent work I've been doing involving reducing the time it takes to compile our program I have run across cases where the simple function call prototype and the class for the dialog are in the same header file. This may seem logical since they are related, but by the time you count up all the lines that the dialog class needs to include from all the 100's of header files that it ultimately includes you can easily have 300,000 lines of included files to get at the 1 line prototype that you need.

The problem get's compounded in the file that is used to dispatch dialogs which can end up including dozens of similar headers. Certainly there is overlap so in the end the file may only bloat to about half a million lines of code when they are all included. But why pay the cost of half a million lines of code when a 100 lines will do?

I also spent part of last week encapsulating the boost thread locks and mutexes which on Windows end up pulling in 400,000 lines of code. I ended up with a nearly identical templated set of classes where the headers run less than 100 lines of code total including everything that they include. This was done by using the pimpl (Pointer to Implementation) idiom.

In talking to one of my colleagues I commented that it seemed wholly surprising to me that people as smart as the guys that put together the boost library would impose such a horrendous overhead when it was nearly trivial to avoid it. He commented that this is what you get when you have a header only library. To which I replied, but boost threads requires that you link against a DLL.

The absurdity of a 4 orders of magnitude difference in number of lines of code to implement a simple concept makes me wonder what the implementors were thinking. I can understand an inexperienced or even a somewhat experienced person making this sort of rookie blunder, but these are supposed to be experts.

I am becoming more and more conscious of the fact that many so called experts in the C++ community are really not. They are really smart, but that doesn't make them experts. That just makes them dangerous.

While there are some really good top notch open source software projects out there the vast majority, including boost, are putting out average or below average software.

The same can be said about the standard template library. While many implementations of the STL are good, I have yet to find one that includes less than 20,000 lines of code for something as simple as a string. Ask yourself why std::string has to be template. The correct answer is that there is no good reason.

Template library creators have lost sight or never grasped the basic programming concepts of data hiding, data invisibility, or the proper design of a library.

Rather than hide the implementation in a .cpp file they expose everything to your compilation unit through header includes and force the compiler to work excessively hard to compile the same thing over and over again.

Rather than give us the option of using the explicit instantiation model and do forward declarations of template classes they force upon us the one size fits all mentality of exposing our compilation unit to tens or hundreds of thousands of lines of code when a couple of dozen is all that is needed.

I don't get it. Why isn't the C++ community up in arms about this irresponsible imposition of insanely heavy compilation costs?

I suppose part of the problem is that adding 1 more straw to the pile isn't noticeable. Eventually you break the back of the proverbial camel or boil the proverbial frog alive.

The problem is that the library creators aren't trying to use their libraries in real world applications. Their trivial test applications compile in seconds and so they don't notice and don't have to pay the cost of the fatal flaw in their designs.

Demand better. If the "experts" aren't going to make the data invisible then we are forced to do it. That adds a little work for all of us that could be done once.

Code Invariance and C++ Templates

I have been working to improve compilation times recently using all the usual things, interfaces, pointer to implementation (PIMPL) wrappers, forward declarations, etc.

In doing this my biggest target is the boost libraries. These are orders of magnitude larger than you might think. In analyzing the amount of headers that a typical boost header includes I've found them to be 2-10 times the size of a typical STL header.

The STL headers are already bloated. For example <string> runs to about 30,000 lines typically.

First of all std::string is a template to produce 2 common variants std::string and std::wstring. While it could produce more I can't think of any that I would typically need or want.

Design Guideline:

Don't use templates when a non-template implementation is just as good or better.

If you look at the std::string API you discover that you should be able to prototype it in less than 200 lines of code. Now there are arguments for inlining for performance reasons that make inlining it a nice option.

However, most of the time I don't need the performance benefits. It is also very difficult to forward declare a string and the STL implementors don't provide us with one that we can use. They provide but no other forward declarations for any of the container classes.

The result is that any file, even a 10 line file, that I want to use a string in becomes effectively a 30,000 line file.

This is a huge price to pay when the code it generates is invariant. Now I know that compilers are smart and can cache this, and some do a pretty good job. And computers are a lot faster today. But when I end up spending several hours a day waiting for compiles, productivity suffers.

The point here is that templates are effectively programs that generate code. That code only needs to change when the template changes and the generated code needs only to be done once for each combination of types that you are using.

With the design of C++ this is done once for each compilation unit. It is redone anytime a compilation unit is invalid. This leads to an N^2 algorithm on top of needing to parse thousands of lines of code.

There are other invariants in compilation as well. In fact any code that hasn't changed and won't generate different machine code is invariant. Compilers and programmers move invariant code out of repetitive tasks.

We need a compiler design that does the same thing.

Today we see typical compilation times of minutes and hours when in reality they could often be milliseconds. Current if I change 1 character in a header that is included in all of my code then all of it needs to be recompiled. If that character results in a 1 byte difference in the generated machine code, it should be instantaneous.

Effectively this is the same problem as we used to have with typewriters. If I add 1 character I may need to retype the whole page. If I add a paragraph I may need to retype all the following pages. This is why we used to have many secretaries in businesses. Now we have word processors and we are far more efficient.

Language designers need to apply the lessons of complexity to the design of the language. Library writers need to apply the lessons of complexity to the design of libraries.

A small amount of work on the part of the STL library designers would allow us to have massive savings in the time it takes to compile.

The C++ language already gives us options for making the use of templates less invariant.

Let's take std::string as an example. While I don't think std::string as a template is even marginally a good idea, let's assume for a minute that it is. To give C++ users the most flexibility we need the following:

<string_fwd> - Provides a forward declaration of std::string in as few lines as possible so it can be used to reduce the header load in header files that only need a forward declaration.

<string_proto> - Provides the prototype for std::string without implementing any methods using as few lines of code as possible. This allows me to use std::string in compilation units without having to pay the price at compile time of compiling all the methods in string and then optimizing their inline use and then emitting the template code.

<string> - Provides the full std::string implementation as it stands now so that I can use it in cases where performance is critical and can also explicitly instantiate it where needed.

This pattern applied to all of the STL would be easy to implement and give us more ability to control how long compiles take.

While I suppose I could create a wrapper class for std::string, std::vector and the other containers are much more compelling at templates. I have been playing with doing a forward declaration and that seems to be feasible, but suffers from some extra maintenance.

With any luck someone will start listening and fix some of these issues.