Thursday, March 11, 2010


C++-201x Variadic Templates



C++-201x (or C++-0x as near sighted individuals like to call it) introduced a new feature called variadic templates, which allows a template class or function to recieve multiple parameters to it of various types as you would normally achieve with "...".

You could continue to use stdarg if you wish, but a limitation of it is that you have no way of knowing what type each parameter is to use with va_arg(), unless you just hard code it to only work with a specific type, or pass a type list how the printf() family of functions do it, or something similar.

I always wanted a way to do func(integer, string, float, whatever), and interchange that with func(float, whatever, char), and now you can.

If you try to research this topic online, basically everyone is just writing and talking about how to implement a tuple class, or how to make a type safe printf. Lets try something a little more interesting and perhaps instructive, yet simple.


#include <iostream>
using namespace std;

//Output function to output any type without type specifiers like printf() family
template <typename T, typename ...P>
void output(T t, P ...p)
{
cout << t << ' ';
if (sizeof...(p)) { output(p...); }
else { cout << '\n'; }
}

//Since variadic templates are recursive, must have a base case
void output() { cout << '\n'; }


//Compute sum of all parameters
template <typename T, typename ...P>
T sum(T t, P ...p)
{
if (sizeof...(p))
{
t += sum(p...);
}
return(t);
}

//Since variadic templates are recursive, must have a base case
template <typename T>
T sum(T t) { return(t); }

//Test it
int main()
{
output();
output('5');
output('5', 2);
output('5', 2, "cows");
output('5', 2, "cows", -1);
output('5', 2, "cows", -1, 0.5f);
output('5', 2, "cows", -1, 0.5f, 16.3);

cout << endl;

cout << sum(1) << '\n'
<< sum(1, 2) << '\n'
<< sum(1, 2, 3) << '\n'
<< sum(1, 2, 3, 4) << '\n'
<< sum(1, 2, 3, 4, 5) << '\n';

cout << endl;

cout << sum(0.1) << '\n'
<< sum(0.1, 0.2) << '\n'
<< sum(0.1, 0.2, 0.3) << '\n';

cout << endl;

return(0);
}


Here's how to compile and what the output looks like:

/tmp> g++-4.4 -Wall -o test test.cpp -std=gnu++0x
/tmp> ./test

5
5 2
5 2 cows
5 2 cows -1
5 2 cows -1 0.5
5 2 cows -1 0.5 16.3

1
3
6
10
15

0.1
0.3
0.6

/tmp>


As can be seen I achieved my desired goals of being able to pass various types, not know the types to be passed in advance, and not need a type list passed anywhere. All the other examples I've seen seem to be forgetting these important points.

As for how this works "typename ...P" defines a template type of P which will be many parameters packed into a single variable. "P ...p" receives all these parameters in a variable named "p".

However the only things I can really do is see how many parameters are in "p" using "sizeof...(p)", or split "p" up into all its parameters, to then pass to a function by using "p...". It would be nice to be able to simply iterate through the values of "p", but that's not possible at the moment.

Using standard recursion techniques, you can pass the split up parameters from "p" to a function which has enough templated parameters as "p" is currently holding, or a lesser amount, and the last parameter can be a variadic parameter to catch all the others which don't fit into the first few. Then using those first parameters, you can access the values you need, and so on ad infinitum.

Read up on recursion if you're confused.

Now if you're still wondering what this may be useful for, just consider functions which handle data serialization or something similar.

One thing to note, at least with GCC at the moment, you always need to have a separate function as a base case, even if it's never used.

Take "output('5', 2, "cows", -1, 0.5f, 16.3);", on the last leg of iteration, when the double is passed to the first parameter to be received by "t", and "p" is empty, "if (sizeof...(p)) { output(p...); }" which is the line responsible for the recursion seeing an empty "p" won't ask for "output()", but it seems the compiler still wants it to exist:


test.cpp: In function ‘void output(T, P ...) [with T = double, P = ]’:
test.cpp:8: instantiated from ‘void output(T, P ...) [with T = float, P = double]’
test.cpp:8: instantiated from ‘void output(T, P ...) [with T = int, P = float, double]’
test.cpp:8: instantiated from ‘void output(T, P ...) [with T = const char*, P = int, float, double]’
test.cpp:8: instantiated from ‘void output(T, P ...) [with T = int, P = const char*, int, float, double]’
test.cpp:8: instantiated from ‘void output(T, P ...) [with T = char, P = int, const char*, int, float, double]’
test.cpp:33: instantiated from here
test.cpp:8: error: no matching function for call to ‘output()’


I'm not sure though if this is required by the standard, or a bug in GCC.

As for C++-201x support in general, GCC seems to be the furthest ahead, even surpassing Comeau C++.

See a comparison of the various compilers' support of C++-201x.

7 comments:

Erik said...

I'd guess the base case has to be there.

Another take on output:

template
void output(T t)
{
cout << t << endl;
}

template
void output(T t, P ...p)
{
cout << t << ' ';
output(p...);
}

sizeof... or if-statements begone.

insane coder said...

Hi Erik.

Yeah, it's easy to remove the usage of sizeof...() and the if, but I wanted to demonstrate how to pack it all into a single function and not really need the base case.

Dan said...

Insane Coder, unfortunately, the base case IS always necessary. Though the code path can never actually get inside the if statement, the function output(p...) when p is empty translates into just output(), which isn't handled by your variadic template. So the compiler takes "if(sizeof...(p)) { output(p...); }" and turns it into "if(0) { output(); }" and while a good optimizer can and should optimize that out, if statements are still technically considered to be runtime operations, hence, the function being called has to exist in order to be linked properly in case it isn't optimized out. In response to Erik, a better way to do it might be:

template
void output(T t, P ...p)
{
cout << t << ' ';
output(p...);
}

void output() { cout << '\n'; }

which is the same code minus the if-else with no code repition (whereas you have "cout << t" twice).

Lachlan Stuart said...

Is it possible to access template args by index? For instance, if I was allocating memory for all of the parameters, could I do this?

size_t size = 0;
for(int i = 0; i < sizeof...(P); i++)
size += sizeof(P[i]);
data = malloc(size);

insane coder said...

Hi Dan.

Even if I do something like this:
if (sizeof...(p) >= 5)
{
t += sum(p...);
}

So it'll only add up the parameters after the fifth one, GCC still tries to recursively generate a bunch of dead code and complain there is no code for a base case.

Hi Lachlan Stuart.

As I explained in the article, iteration is not possible. Templates in fact always require recursion.

Take this code for example:
template <typename ...P>
void output(P ...p)
{
auto mytuple = make_tuple(p...);
for (size_t i = 0; i < sizeof...(p); ++i)
{
cout << mytuple.get<i>() << ' ';
}
cout << '\n';
}

That code actually places all the parameters neatly into a tuple, and attempts to iterate over it. But alas, it won't compile:
test.cpp: In function ‘void output(P ...)’:
test.cpp:12: error: ‘i’ cannot appear in a constant-expression

Templates always need recursion, and no trick I can think of at the moment would allow for iteration.

Unknown said...

at runtime, noarg output() will not be called

at compile time, noarg output() is used and therefore correctly required

don't mix these two perspectives

also, behind the scenes, there is no recursion: different functions will be called for each params combinations (not the same); it's just that the compiler is smart and does the work for us

precisely because of such things, I fear compilers will get smarter and programmers will be more dumb

Karel-Lodewijk verdonck said...

Iterating over a variadic template pack is not so hard if it's types are homogenous or at least convertible to one type.

std::array list = {pack...};

or

std::vector list = {pack...};

Will actually work.