Sunday 6 July 2008

gcc 4.4 breaks boost

Hello! This is an update. As i have just realized, the version from the trunk breaks a few Boost 1.35 headers. This is due to the way the new GCC handles preprocessor symbols in #if and #elif. This new way is more standards conformant, however it breaks at least the boost headers.

Look here to read the details: http://www.nabble.com/Patch-for-building-Boost-with-GCC-4.4.0-td17982932.html

But don't panick, you can still use the boost library with GCC 4.4. You will have to patch it using the patch provided in the link. To do so, download the patch, cd into the include directory that contains the boost headers (a folder called boost), and apply the patch with the GNU patch utility.

Here are the commands for my machine:
# cd /usr/include
# patch -p0 < path_to_the_patchfile/patchboost-gcc-4.4.0-iteration-depth.patch

Sunday 8 June 2008

C++0x with GCC from the SVN repository

Hi, that was rather fast that I wrote this blog, but don't get used to it, that was only because I got bored of the first one while writing, so I started this one and continued to work in parallel. Whatever, let's get going...

Introduction

If you want to try out the new features of C++0x, you'll need a compiler that supports at least a few of the features.
But there's a problem: C++0x is all new and hot. It's even so hot that you'll burn your fingers when trying to grope it to firm. What I mean by that is, that the standard is still developed and a few features are not yet completely defined and formulated. Yet, some compiler vendors have taken upon themselves the task to implement a few features of the new standard, to enable programmers to test the new features and get and idea of what the new standard is about.

The compiler landscape

The list of these vendors however is very short. There is:

Comeau C++ is a quite good but proprietary compiler. It supports few features, the most notable one is the "auto" keyword. The state of C++0x support can be viewed here: [4].
ConceptGCC is a fork of GCC, its main goal is to implement the Concepts feature and to rewrite the STL to use Concepts. ConceptGCC does barely support any other C++0x features, and was forked off because this feature requires fundamental changes in the Compiler. Right now effort is being made to merge the Concepts feature back into the mainline GCC compiler.
Starting from version 4.3, the GCC compiler adds experimental support for C++0x. This experimental support has now grown to a considerable amount of features. The current status can be viewed here: [5]. Unfortunately, my favourite, the "auto" keyword has not yet been implemented.
Of course, there are other compilers out there who support singular features as extensions to the language, but I don't know any examples I could mention here.


The compiler of choice

To test new features I chose to use the gcc 4.4 compiler. The reasons for this are simple: It's free (as in beer, as well as in freedom), it's good and it has the most support for C++0x.
You can for sure download a tarball from somewhere, compile it and use it. But I chose a different approach: get the most recent version from their svn repository. The advantages of getting the source from svn is that

  1. It's really up to date. When you check out the development trunk, you definitely get the latest version.
  2. Updates can be applied with low overhead: simply update your working copy and make will recompile only the updated and dependent files.
Of course there are disadvantages
  1. Where a tarball would achieve great compression performance, the sources here are downloaded uncompressed. That uses a lot of bandwidth.
  2. A working copy contains quite a lot of extra files, that are needed for versioning.
  3. Checking out the trunk repository will get you the up-to-date development version. The problem is, it can happen that this version is broken. That means when you check out this version, your copy will be broken too. But don't worry, if you wait a few days and then update the source, the problem will most likely be allready fixed.

Preparing to install

Allright, before we start we got do a few things.

First of all, install UNIX or a UNIX derivative, such as Linux. Ok, admittedly this is a bit unfair. But let me tell you one thing: the build process on Microsoft Windows is a pain in the ass. It is done by installing all kinds of tools, shells and compilers that let your system act as though it was Linux. If you really want to do that, you might want to read the instructions on building gcc 4.1 on a Windows host [6]. I'm pretty sure, the instructions for the current SVN version differ very little from the ones provided for 4.1. However, it's probably much faster and easier to install a GNU/Linux distribution such as Ubuntu [7]. This document only describes how to build the compiler on a GNU/Linux host, but I suppose with a few adaptions they will be also valid for other POSIX compliant systems.

Install a Subversion[8] (SVN) client. I won't discuss this, since installation of software is very specific to operating systems and most Linux/UNIX distributions provide easily installable packages. If you are on a Windows host, you might want to take a look at TortoiseSVN [9].

Now here comes the annoying part. You need to satisfy the build dependencies before you start doing anything. Here you can find the complete list of build dependencies for gcc: [10]. On a reasonably complete system that was allready used for development (which I assume, since you are interested in the new standard) you should allready have most of the stuff installed. The only items I had to install additionaly were

  • GNU Multiple Precision Library (GMP) version 4.1 (or later)
  • MPFR Library version 2.3.0 (or later)
Which was no problem, since my distribution [11] allready provided precompiled packages for these libraries. Of course, you will need a compiler to compile the compiler, or do you want to compile it with your word processor? ;-) I used gcc 4.1 for that, but basically any gcc version should work. (For compilers other than gcc, look at the build instructions [12])

Next, we need two directories. The first is the directory that contains the source code for the compiler. I will refer to this directory as $GCC_SOURCE. It will take about 2.5GB. "WHOA, WTF??? 2.5GB? Is that gigabytes?" you might ask. Yes. It is gigabytes. Two and a half. The reason for this is that first of all the SVN trunk of gcc contains a lot of additional files, such as tests, documentation and so on. Also, gcc is more than just a C/C++ compiler, its a compiler collection of several compilers. Especially the Java API takes a lot of space. (If you are smart enough to find a way to not download the whole collection but still being able to compile it, mail me, because I wasn't). And finally, don't forget that we have to actually build the compiler, so we need space for the object files and executables.
The second directory is the "installation directory", i will refer to it as $GCC_INSTALL. It contains libraries, header files and executables. If you really want to, you can "hard install" it into /usr/local or something, but I wouldn't recommend that. This directory uses about 100MB.

Last but not least, you need:
Time. The initial checkout downloads all the sources (and documentation and test and ...) since that's quite a lot, this step takes about an hour or so to complete, with a decent internet connection. The first compile also takes ages, that's about a 3/4 hour, on my good old Pentium D with 3 Ghz.

Get going

Now that we have all the prerequisites, we can start off. Verify that you have Subversion installed properly with svn help. This should display some help text. Change into $GCC_SOURCE, the directory you created earlier for the sources. Now is the time to stop watching videostreams and downloading porn, because you will need every bit/second you can get out of your internet connection. Find the subversion repository for gcc. This should be svn://gcc.gnu.org/svn/gcc/trunk

The big checkout

Start the initial checkout with
svn checkout svn://gcc.gnu.org/svn/gcc/trunk
If you want, you can add the -q option to the command line, this will cause to run the checkout to run quietly (without output). However, as the checkout takes so long I prefer to see that at least something is being done.
Do something else. This checkout takes a long time, so you might like to do play games, read ANSI/ISO papers about the new standard or talk with your wife. But don't do anything that consumes your bandwidth.

Configuring

When you have obtained all the sources, you can easily compile the source with

./configure
make
make install

Well, to be honest, it's not THAT easy. The above invocation installs the complete compiler suite into the default installation directory, which is /usr/local. That might not be what you want. To customize your copy of the compiler, we have to set a few flags. You can look up the flags by calling

./configure --help. These are the flags I used:

--prefix=$GCC_INSTALL
Set root directory of installation to the directory we created previously.
--program-suffix=suf
The suffix to the name of all binaries you will have. The compiler you have installed might have a suffix that looks somehow like "4.2". I used "svn" for that, to be clear that this is a svn version.
--enable-threads
Enable threads
--enable-languages=c, c++
IMPORTANT! This flag is used to indicate that only C and C++ compilers shall be built. If you don't specify this flag, you end up having all supported languages installed. Believe me, it takes long enough to compile these two compilers, I dont want to know how long it takes for all languages.
--enable-shared
This builds shared libraries that can be used by the programs you compile. If you omit this, you have to link those libraries to your programs -statically.
--disable-nls
This option disables Native Language Support. Although it's very nice of the Free Software Foundation to provide error messages for various different languages, I personally find it annoying to read the cryptic error messages in german (maybe just because I am not used to it). Leave this option out if you do like them.
Basically, it's a good idea to look at the flags your previously installed version of gcc was compiled with, and then adapt these flags to your needs. Invoke your old gcc version with gcc -v to retrieve this information. For all those who dont't have a gcc compiler, here are the flags of my vanilla pre- packaged compiler:

me@mybox:~$ gcc -v Using built-in specs. Target: i486-linux-gnu Configured with: ../src/configure -v --enable-languages=c,c++,fortran,objc,obj-c++ --prefix=/usr --enable-shared --with-system-zlib --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --enable-nls --with-gxx-include-dir=/usr/include/c++/4.2 --program-suffix=-4.2 --enable-clocale=gnu --enable-libstdcxx-debug --enable-objc-gc --enable-mpfr --disable-libmudflap --enable-targets=all --enable-checking=release --build=i486-linux-gnu --host=i486-linux-gnu --target=i486-linux-gnu Thread model: posix gcc version 4.2.4 (Debian 4.2.4-1)

Compilation

Now we are ready to compile. Now is the time to resume downloading porn or to watch videostreams, now you need the CPU, not your internet connection. Unless you have a multicore processor, stop all running programs, you will now need every CPU cycle you can get. type

make

and as soon as you hit the enter button the compilation will start. This step takes a LONG time (about 3/4 to 1 hour). You can leave your machine now, the build process should run automatically. Come back from time to time to check if the compilation finished or encountered errors.

When this step has finally finished, type

make install

into the command prompt to install the binaries into the directory you specified as '--prefix' flag for the configure script. Depending on wether you specified a system directory (e.g. /usr/local) or something in your home directory, you may have to run this command with root privileges.

You're nearly done. The only thing you need to do now is to reconfigure your dynamic linker to work with the libraries provided by the compiler. If you do not want that, you have to link those libraries statically to your programs, by adding the -static flag to the command line. To reconfigure your dynamic linker, find it's configuration file (maybe /etc/ld.so.conf), and add this line to it: $GCC_INSTALL/lib. (Note: on newer systems, the configuration file only includes the files in a configuration directory, like /etc/ld.so.conf.d/. In this case it is advisable to create an own file, e.g. gcc_svn.conf and put the line with the lib directory there.). Then run ldconfig and everything will be fine.


That's it! You're finished, finally. For convenience, you might want to add $GCC_INSTALL/bin to your PATH environment variable.

Testing the installation

To test this installation, open an editor and paste this code:


#include <iostream>
#include <vector>

static_assert(NULL == 0, "You live in a paralell universe!");

int main()
{
    long long a = 2; // new type
    decltype( a ) b = 5; // b is now long long    

    // binding rvalue reference to a temporary object
    std::vector<long long>&& vec_ref = std::vector<long long>();
    
    vec_ref.push_back(a);
    vec_ref.push_back(b);
    
#ifdef _HAS_AUTO
    // unfortunately, this doesnt compile yet...
    for (auto it = vec_ref.begin(); it < vec_ref.end(); it++)
#else
    for (std::vector<long long>::iterator it = vec_ref.begin(); it < vec_ref.end(); it++)    
#endif
        std::cout<<*it<<'\n';

}

Compile this code with the -std=c++0x to enable C++0x extensions:

$GCC_INSTALL/g++-svn -std=c++0x -o stdcxx09 stdcxx09.cpp

where $GCC_INSTALL is the directory you installed the new compiler, and the file stdcxx09.cpp is the file you pasted the above code in. If compilation succeeded, Congratulations, you made it!

How to handle updates

Now that you have obtained the binaries, you can use the compiler normally. But hey, what if two weeks later you read somewhere, that your favourite C++0x feature (maybe the auto keyword?) was added to GCC? All the other people would have to wait until a new gcc version was released. But now, we are special! Simply change back into the $GCC_SOURCE directory, and update your sources:

svn update

This will retrieve the updated files incrementally. That means that only new and updated files are retrieved. Even better, if a file was updated, only the diffpatch is transferred!
Next, recompile the sources, by issuing the

make

command. (but DONT do a ./configure, the previous configuration will be fine!) If you are lucky, make will only have to recompile the updated files and it's dependencies, then compilation will not take so long. When compilation has finished, do a make install and you're ready to go.


Allright, i hope this little howto helped you a bit in obtaining a compiler with C++0x support. Don't forget to have a look at the project status page [5] of the gcc compiler, and try out all features you find. They will definitely make your programming life easier.


[1]: http://www.comeaucomputing.com/
[2]: http://www.generic-programming.org/software/ConceptGCC/
[3]: http://gcc.gnu.org/
[4]: http://www.comeaucomputing.com/439features.html
[5]: http://gcc.gnu.org/projects/cxx0x.html
[6]: http://www.mingw.org/MinGWiki/index.php/How%20to%20Compile%20GCC%204.1
[7]: http://www.ubuntu.com/
[8]: http://subversion.tigris.org/
[9]: http://tortoisesvn.tigris.org/
[10]: http://gcc.gnu.org/install/prerequisites.html
[11]: http://www.debian.org/
[12]: http://gcc.gnu.org/install/

Friday 6 June 2008

Welcome to my blag!

Hello, boys and girls! Instead of writing my own wobsite, I came here... So it looks like today is the day when my blagging career starts. But before you dissolve in exalation, i must warn you: This is going to be a tech only blag! "What do you mean by tech only?", you might ask. Well, here is a list of things you should at least have heard of before trying to read it. (Ordered from least probably heard of to very likely heard of, bottom up:)
  • Electricity
  • Computers
  • Programs
  • Programming
  • C++
  • The upcoming Standard for C++

No idea of what I'm talking about at one of those points? Well, you're probably better off reading something else, otherwise you are in danger of serious injuries from your head banging against the keyboard when falling asleep momentarily! (-;

Sow, now that I'm the only reader of my own blag now, here's what I'm up to:
The International Standards Organisation (ISO) and the American National Standards Institue (ANSI) are currently working on an improved version of C++, that addresses many issues that came up after the first standard (back in 1998).
This new standard will probably be released about the beginning of 2009 [1].

And what I want to do is to write some kind of "introduction" for C++ programmers that want to become familiar with the new features of C++. Of course, before beginning to write about it, I should become familiar with the features myself.

Today I had nothing to do at work. And what does a geek do when he's bored and has access to a computer with an internet connection? That's right, he reads ANSI/ISO- Papers (-: Here's what I have been reading:
[2]: This paper describes a new feature called "variadic templates", that's going to ease template metaprogramming a lot. [3]: This document describes the changes to be made to the working draft and also contains a few revisions over the above document.

So what are "variadic templates"? Variadic templates are the possibility to pass a variable number of arguments to a template. Sounds familiar? Well, it's the compile time version of the varargs macro, which everybody instantly starts to hate at first sight. Where the varargs macros forces you to trust the users of your functions to pass the right types of arguments, and leaves a LOT of space for errors, the Variadic Templates feature lets you pass a variable number of arguments SAFELY! And by safely I mean a typesafe, completely compile time defined, zero overhead variable argument lists. Can you top that?

Well, I think it's time for code.


// overload a special case with no arguments
void printf(const char* s) 
{
    // stop on null character
    while (*s)
    {
        if ( *s == '%' && *++s != '%')
            // if we came here we ran out of arguments but not out of string
            throw std::runtime_error("Missing Arguments!");
            
        std::cout<< *s++;
    }
}

// case for variable arguments
template <typename T, typename... Args>
void printf(const char* s, T value, Args... args)
{
    while (*s)
    {
        if ( *s == '%' && *++s != '%') // if we encountered a formatting argument,
        {
            std::cout<<value; // print the value
            return printf(s, args...); // and call this function recursively
        }
        std::cout<< *s++;
    }
    
    // if code flow came to here, we ran out of string but not out of arguments
    throw std::runtime_error("Extra arguments");
}

This code is taken from [2], with a few adaptions and comments from me.

Allright, let's have a look at the declaration of the template: template<typename T, typename... Args> Here typename T is a regular template parameter, but typename... Args is the new stuff. This means that Args is a Parameter Pack. A parameter pack is a template argument that contains other arguments. The parameter pack has to be the last template parameter (of course), and can be also of an ordinary type.
Example: template <typename T, unsigned int... Integers> void do_something(Integers... ints) { ... }

If we again look at our printf example from above, we see that the ellipsis (...) is on the left side of the template Parameter. This means that we pack all template arguments into Args. Args is now a pack of Types. The ellipsis to the right of Args, in the parameter list of the function itself means that we unpack the types in the parameter pack. At the same time, we pack the arguments for the function into args. is now a pack of values. This pack of values is finally unpacked in the recursive call to printf.

As you see, we packed a set of Types and then we unpacked it. When unpacking the set of Types, we packed the actual values, that will be passed to the function. Later we unpacked the values, to pass them to another function.

But there is a problem. The only way to unpack Template Parameter Packs is either as Types for function arguments or as arguments for another template. Function argument packs can only be unpacked for calling another function. Although there is the possibility to count the number of arguments in the Template or Function Parameter Pack with the sizeof..() operator: template <typename... Args> void print_nargs(Args... args) { std::cout<<sizeof...(Args); } there is no way to iterate through the parameters with an ordinary loop like with the varargs macro. However you can still handle the arguments one by one: by using recursion.
We allready have seen how this is done above. The trick is performed by providing one fixed Template argument, and calling the same function again, with the remaining arguments. This will cause the first argument to be "stripped off", so you can use it. Depending on what you want to do, you might have to provide a specialization or overload for an instantiation with no template arguments (as you can see above).

So what's it for? This new feature can be used for functions with variable argument lists, such as printf, but also for passing on arguments to constructors, callback funcions and so on. Variadic templates are particular useful in conjunction with the new initializer lists. An important application for variadic templates is the area of template metaprogramming. Whereas you might not come in touch with it directly, it may well be that you will use one of the libraries written with this technique, such as various Boost libraries. Until now, variable arguments were realized either with templates which had a lot of default arguments or by (mis)using the preprocessor, which causes awfull and unmaintanable code. With this feature, library authors have now a great tool at their hands. We will see, what will emerge from that.

Well, that was a rather short Introduction into variadic templates. Look back here in some time, I will post a more detailed introduction in some time. Until then, you can read [2] and [4], as "accompanying literature".

I hope someone read this, and I welcome every (usefull) comment. If have questions, write me an email, maybe I'll answer it (-: Here are the links: [1]: http://www.open-std.org/JTC1/SC22/WG21/docs/papers/2008/n2565.html [2]: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2006/n2080.pdf [3]: http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2007/n2152.pdf [4]: http://www.osl.iu.edu/~dgregor/cpp/variadic-templates.html