Mozilla is the new Microsoft

So who here loves Mozilla and their browsers such as Firefox?

I do too, but am I the only one finding Mozilla turning more into Microsoft every day?

To start off, lets point fingers at the memory issue. Notice how Windows' memory requirements have gone through the roof? Yet comparable desktop environments and windowing systems from other sources seem to be just as advanced as Microsoft's latest offering and don't use a fraction of the RAM. I'm told KDE 4 runs comfortably with only 512MB of RAM, and you can even get away with 256MB, while a friend of mine running Vista started with 1.5GB but said he had to add another 512MB so the system didn't get all slugish on him. Does Vista offer anything special that KDE 4 doesn't?

Compare that to Firefox these days. I open up Firefox with two tabs, one in a pretty ordinary text page with a single image on it, and another with some tables and a form, and it uses over 400MB of RAM. When mentioning it to Mozilla, they of course tell you it's a feature not a bug (sound familiar?). What feature exactly needs all that RAM? I'm told some nonsense about how it caches previous pages one went to, so they load quickly when you press back. Considering I just opened the browser, which pages is it caching exactly?

To further compound my bafflement of the issue, I even disabled that feature in about:config, and yet Firefox still eats a ton of RAM when I open it. I used to run Netscape Navigator on a machine with 16MB of RAM, and two whole browser windows opened, and it didn't eat a fraction of the RAM Firefox does now (and this was before Windows had virtual memory). Heck, even other web browsers such as Konqueror and Opera don't use that much.

Well, now you'll tell me, but at least Firefox started the whole standards revolution right? At least it's better than Internet Explorer, right?

While it's true that it's better than IE, I wouldn't even consider IE a browser, as it can't even get basic standards right. If you had a program which edited text files, and you have it open a file, and it comes up with paragraphs out of order, and some not even appearing, would you call it a text editor? Yet I'm starting to see Gecko (the engine powering Firefox and Seamonkey) go the same route. Try setting a background for a radio button with CSS in Firefox, it completely ignores it. Yet even IE, the non-webbrowser, can do CSS backgrounds for radio buttons (albeit it only changes the background color outside the radio button itself within its rendering square).

This gets worse when trying to couple some JavaScript with your pages in Firefox. The onchange event for select boxes only fires in Firefox when changing them with the keyboard when one presses enter. All other browsers fire the event as soon as you can see a change within the box, such as when one presses up or down. Now perhaps you'll tell me that's undefined behavior so Firefox can implement it differently than everyone else (sound like Microsoft any?), but now apply this to how it handles dynamic text. Say you want to take our example above and convert your onchange in a select box to an onkeydown, so it should fire for every up/down request in the box without the user needing to press enter. Firefox again fails completely here, since it won't update text on a page modified by JavaScript till the screen is clicked, or a button is pressed. Meaning if you have your onkeydown event for a select box call a function which changes some text on the page, to say perhaps the contents of that select box. You won't actually see those changes till you press a button, perhaps up or down again, showing you the previous value on your page from the select box, instead of the current one. Sounds like a real stellar browser, doesn't it?

Then there's the issue about how Mozilla is trying to make their browser the only platform out there. Why do most people today still use Windows, even when there's a better OS for the average person? It all boils down to the applications. Anyone who is proposed to switch asks, can I play my favorite games on this other OS? Does my digital camera's photo sorting software work on it? What about all the other applications that I use on a regular basis? As Microsoft's chief primate keeps yelling, it's all about the developers. People will be stuck using whatever runs their favorite apps.

Now of course we know that the internet in many ways is becoming the new platform out there, hence why Microsoft tried to crush it or control it all these years. So as long as your browser is standards compliant, if people develop for it, we should be okay, no?

Unfortunately it's not like that. GMail doesn't work on Konqueror unless Konqueror changes its user agent to report as Gecko. Meaning many sites lock out browsers just because they didn't test with them, not because it doesn't work for any particular reason. A similar problem exists how IIS servers gibberish to browsers reporting themself as Opera. Yet Microsoft only wants to keep it that way.

Recently Microsoft reported that the upcoming IE 8 will only go into standard compliance mode when websites include an IE8 tag in their code. Now most people who discussed this feature have totally missed the main problem. It's not about having to add an extra tag. It's that the default mode for any whacko out there writing a webpage is to render in IE 6 or IE 7 super-bug-filled-non-standards-compliant-mode, so webpages will continue to not become standardized. Of course Microsoft realizes this, they'll do everything they can to make sure the large majority of stuff coming out by the clueless masses only works on their platform.

Now I'm not talking about Mozilla making the web buggy like Microsoft is trying to (but I bet they'd like that if they could get away with it), but it's that they're trying to get people dependent on their web browser. How so? It's all about the extensions!
Mozilla invented their own language and format and ranted and raved about how everyone should be writing extensions for their browser, and how their browser is the de-facto standard of the free world.

How many of you using Firefox feel they need all kinds of extras that people write for it? How often do you sit down at someone else's Firefox and feel naked about a lack of certain features you tailored your browser back home or at work to?

Now while I could go on and on about all kinds of extensions, everyone has their own tastes, so it'd be pointless to, but I'll point out a couple of examples.

Some of the extensions I use regularly for Firefox, such as auto translations of pages in languages I don't read/speak, or user agent spoofing are built right into Konqueror, many others aren't available anywhere else, or only for certain browsers written specifically for them. I'm talking about the GMail/Yahoo/And whatever else e-mail notifiers that can tell you when you received new e-mail, or ForcastFox which can tell me mostly accurate weather info for the week for just about anywhere in the world.

Now these extensions I just mentioned are perhaps not anything to get addicted to, but what about Ad blocking? Now of course other browsers also have a Mozilla compatible ad block framework installed, how many of them allow you to install something like the "Filterset.G Updater" so you can have your blacklists automatically updated by others to hide all the new types of ads on the internet that come out, such as the "Download Firefox Ad" that appears on the right of this page?

Not only that, we can hardly forget the wide plethora of developer related things one can get for Firefox, such as JavaScript Console, DOM Inspector, and all the extensions such as the Web Developer Toolbar and who knows what else.

Am I the only one who now feels locked into Firefox and can't switch because their special browser software doesn't run elsewhere? Even when we realize what a horrible wreck Firefox is, and terrible on machines with little RAM available?

Furthermore, you know how Microsoft always forces their desires upon you? Take this picture of Bill Gates for example:


Notice how his own personal desktop is the default Windows XP background? Surely a man as technical as Bill Gates can figure out how to change it to something he enjoys, just like everyone else has.
Or perhaps Bill Gates never changed it, since the default Windows XP background was Bill Gates' personal background at the time? Of course leave it to a guy like him to force his personal desires on everyone else.

Now lets see how Mozilla does the exact same thing. Ever wondered where Firefox's logo came from? Who would think that an image of a fox conquering the world is a good logo for a browser?

Check out a picture of Mozilla's former CEO, Mitchell Baker:


Now what do you think came first, the Firefox logo? Or her ridiculous hair cut?
Still don't get what I mean? See this:


If you think you can point out how Mozilla isn't the Microsoft of the internet platform, please let me know, I'd love to hear how you came to your false conclusions.

Say goodbye to the former Qt and hello to MiniQt?

So everyone knows that Nokia is looking to buy Trolltech, the company behind Qt right? If not, read the press release.
Trolltech also has up a list of what this means for everyone, as well as a letter to the open source community (rename it to .pdf).

Now after hearing all that, you're probably running in one of two directions. Either thinking that "oh no, my Qt has been taken over by an evil organization and it'll be destroyed!", or that "KDE and Trolltech have an agreement over Qt so we're safe!". The interesting point is that both are true.

What? Both are true?!?! How can that be?

First, a little background. Qt which is the foundation of KDE, and what all the pillars of KDE stand on, is pretty much irreplaceable. Not wanting to ever get bitten by developing such a massive project on such a foundation controlled by a company, KDE and Trolltech signed an agreement, and formed a new group called the "KDE Free Qt Foundation".

To put it in their own words:

To fulfill the purpose of the Foundation, an agreement between Trolltech and the Foundation was made. This gives the Foundation the right to release Qt under a BSD-style license in case Trolltech doesn't continue the development of the Qt Free Edition for any reason including, but not limited to, a buy-out of Trolltech, a merger or bankruptcy.

As for the latest agreement, you can find links to it on their site, but the primary part to worry about is this. Let me summarize. If 12 months have passed without seeing an open source release of Qt, or the open source version falls behind the closed source one in that period of time, or the board as a whole (which half is composed of by Trolltech) agrees to release Qt as BSD, then it shall take the latest open source version of Qt, and open source it under the BSD license as they will become legally able to.

Now one must wonder why is that important? The current release is under GPL2/3 anyway, why do we need to have it stuck under BSD? We can always continue using the latest release as GPL2/3.

The great thing about Qt is that they have a corporation backing it, making sure the toolkit is excellent. They pay employees to do all the dirty and annoying work in the library too. They also ensure that it remains portable on a large variety of hardware. This kind of heavy work is not something the open source community is usually able to coordinate too well. There are very few large open source projects that are fully portable, run on greatly varied setups, and cover areas that most people don't want to program for. For example, Ubuntu made a release a while back where a significant portion of printer support was broken, since it's not fun to test or fix, there's nothing really challenging about printer support (except maybe getting those annoying devices to work in the first place, and not paper jamming), and it's not cost effective to make sure it continues to work, and the "save the universe, time, and pluto" groups wouldn't like you using all that paper either.

Also, Qt is currently able to be gotten either via a proprietary license or under the GPL (with exceptions too!), so companies producing closed source software, and open source nutjobs can play on equal footing. That means when a company hires a developer to develop with Qt, it's the same Qt, so s/he can later go home and use his or her knowledge to write a cool new open source application involving giraffes e-mailing pictures of people with dyed hair around, or whatever s/he might fancy. It also means that open source developers who want to create their own startup company, can use the knowledge they already gathered to write terrific applications, since they can just go and buy some licenses, and write their new apps.

The "Poison Pill" that they signed on would put Qt under BSD, which would continue to allow the companies, and the communities to continue to work with the same software, in the event that Trolltech became evil (or by extension of being bought). It could also mean that a company could throw developers at Qt, and use stuff privately in their own products too, and not be forced to not be allowed to do anything with their own changes, since it's virally all GPL.


This all falls apart however when one considers precisely what is in the agreement. IFF ('If and only if', for those of you who missed school that day), Trolltech or the new company fails to deliver an open source version or equal to the version offered proprietarily for an extended period of time, they can BSD what they have. This is great as long as the evil company doesn't continue to offer something which is worthless to the community but not so to the companies.

To quote Nokia:

We will continue to actively develop Qt and Qtopia. We also want to underline that we will continue to support the open source community by continuing to release these technologies under the GPL

Lets step back and think for a moment what Nokia is interested in. Cell Phones, right? Meaning only Qtopia, Qt in itself is worthless to them. Also consider what Trolltech's primary business is. They advertise these applications as the coolest using Qt. How many of those applications are done by those who buy a license? Now once you figured that out, how many of those have anything other than a Windows build? Many companies use Qt, not for the portability, but because of how easy it is. Here, read what Adobe, a company who has never produced Linux software, has to say. Since companies use it for the ease of use, and speed of development time, and don't really care if it works on Linux or not, since we all know that those Linux guys don't buy anything anyway, why should Nokia bother continuing to develop the X11 port of Qt? It's just a waste of time for any normal business model.

So Nokia could continue to develop Qt for their cellphones, and for customers such as Adobe, and even release Windows only, or perhaps even Windows and Mac OS X builds of Qt both proprietary and under the GPL, but none for X11. And what can the "KDE Free Qt Foundation" do about it anyway? Their agreement is worthless as there can still be an open source version which won't be useful to most of their developers. And who in the world are they going to complain to about 'intent' of the agreement? KDE runs on Windows and Mac OS X now, right? Their petty foundation can continue making their Kool Desktop Environment as long as it's done on Windows right? No one is stopping them.

At this point, the companies and the communities can be fractured, and even worse if the library is very different one platform to the next, it will become completely worthless for the sake of portability. Having an unfunded fork off of an older Qt won't do as well either, just look at what an awful mess the crippled broken GTK is, even with it being LGPL so companies can use it - if they were drugged up enough to.

On the other hand, Qtopia does run on Linux, but isn't protected by the KDE foundation either. If Nokia for whatever reason continued releasing Qtopia under open source, and the X11 port died, but cool new features of equal footing came to the Windows/OS X/Qtopia ports, perhaps the community might get some movement in the direction of moving away from the horrible X11. Bleh, who am I kidding, the nightmare that is X11 is here to stay, right?

KDE 4 Review - Insane Style

So KDE 4 is out, along with major architectural changes, a new face, and new software, as well as tweaks to what KDE 3 had.

Now if you wanted to know about all the new libraries, and new platforms supported, or kinds of other flashy changes they made, you can look elsewhere on the internet. Instead I'm going to just describe the changes to the standard file open dialog, and the ramifications of having Dolphin as the new file manager, keeping in tradition of the previous file dialog articles.

So without further babbling, here's the new file open dialog:


Now ignoring all styling issues for the time being, let us compare that to what KDE 3.5 did:


First big difference is that the KDE 4 file dialog now has a crumb view on top. The crumb view is nice, in that if you want to jump back up a few directories, you can do so in a single click, instead of pressing up multiple times.

In fact you don't even need an up button anymore, but don't worry, the design experts at KDE gave you one anyway, since space on that line isn't important. Because if it was important they might consider moving the encoding drop down elsewhere - oh wait they did do that.

You've got to love mixed messages when it comes to GUI design, they needed more room for the crumb browser so they moved the encoding elsewhere to take up a full line down below, yet leave in the up button. Also interesting to note that KDE 4 is now using Qt 4, which quite nicely offers a file system watcher for paths to tell when they get updated to automatically refresh a file view, it seems despite that, the KDE team thinks you also need a nice refresh button on top.

Now if they really wanted to manage space properly, perhaps they'd put the crumb navigation on a line by itself up top, instead of giving all that room to encoding drop down which doesn't need a fraction of the room it's getting. Maybe something along the lines of this mockup I just made in KolourPaint:


Now there's plenty of room for large paths for the crumb browser, and no ridiculously huge encoding selector.

But these obvious issues aside, we haven't even gone past the surface of how utterly useless the dialog has become.

Remember how I spoke about how Windows added annoying virtual locations? Seems KDE copied them down the last annoying detail. Now "/home/insane" appears as "Home" in the crumb viewer (and no absolute path anywhere). If I want to go up to "/home", and then "/home/spouse", it's impossible. "Home" is treated as a root path, even to the extent that the unneeded up button doesn't do anything at this point either. One has to click on the left on "Root", then select "/home". This gets worse if you're on a machine where home directories are stored several levels deep. Of course power users, can type in the full path in the location bar on the bottom, but you'd think a group as intelligent as KDE can avoid glaring Windows design flaws. Don't even get me started on what happens if you delete the Root button from the quick locations on the left. Poor users who want to find someone else's home directory now have to navigate through the root path filled with stuff like "usr", "opt", "bin", "srv", and other unintelligent gibberish to do something that once was a single button click.

And speaking of annoying locations. Why the heck is Trash a location that appears by default in the quick locations? How often do you decide that the document you need to open is in the trash? I'm glad they removed Desktop, but replacing it with Trash? What were they thinking? Give me something useful like a Media quick link, for optical drives and all kinds of USB/Firewire external memory devices. They also completely removed support for right clicking on the quick locations panel, and just adding a new quick location. Now one is forced to drag a directory's folder icon from the right to the left. I wonder how that will work if one is in some kind of virtual directory that has no parent, making it impossible to drag.

Anyways regarding other features, remember how KDE 3 gave the user a lot of power and allowed them to have something similar to the old Windows 3 split view?


For some reason this useful method to navigate your files is completely gone.
And now with the old location input bar merged into the one for files, I can't even easily edit the current location by hand, since it doesn't display the current path at all. Furthermore, entering a new path seems to make it have useless data left over in it like the last path directory component after enter is pressed.

Old KDE file browser: 10
New KDE file browser: 6

I guess making the perfect file browser, especially when you were almost there is next to impossible.

Speaking of file browsers, lets look at how the new edition of Dolphin fares, touted as being better than Konqueror for simple management of files.

Now I remember I complained to some KDE devs a while back regarding the KDE 4 betas and how Dolphin seemed to be a bit brain damaged at file management, but they assured me it would change for the final release of KDE 4.0, and change it did! For the worse.

Back in the older betas, one could click a button to bring up a full path editor instead of the crumb view, now that button is completely gone. To bring it up, one has to go browse through one of the menus on top to switch back and forth between the quick browsing methods.

Here too, they also kept the brain damaged virtual Home path. There is no way to go up. Nice to see they didn't put in a useless up button, but here again, if I want to see everyone's home directory, I have to navigate there from Root.

Also, if one switches to the classic path editor, if one wants to go up, they're forced to backspace the old path, or switch back to the crumb view, since in the KDE Dolphin developers' great vision, they couldn't stick both views in at the same time, or make an up button appear when in the classic path editor.

There's also no nice directory and file split view here like good ol' File Manager. But one can bring up a directory tree browser:

And in keeping with the new KDE 4 design goal of copying stupidity from Windows (Vista), when viewing the tree view, the file viewing pane is also filled with now redundant directory listings.

Isn't it wonderful to see how people who used to be way ahead of the curve and tried so hard to offer a really advanced product have completely fell between the cracks when it came to normal sane usability of files?

For those of you wondering if perhaps GTK got any better since the last time I reviewed it, you can rest assured that it has not. In fact, I now have a new refined hatred for how lousy GTK is. In the course of uploading some images from FireFox in an article I was writing, the GTK file menu had the file type filter locked to (*.png/*.bmp/*.gif/*.jpg/*.jpeg). Since the pictures I had were all *.JPG, none showed up, as GTK in their infinite wisdom made the extension filter case sensitive.

Seems UNIX file management is just as lousy as ever and getting worse. I'm afraid to even look at Konqueror in KDE 4, fearing they ruined it too.

Feel free to start a flame war in the comments, I got my flame retardant suit on, besides nothing you can write can make me feel worse than what a downgrade KDE went through.

Directory safety when working with paths

As we discussed in our previous two articles, we run into issues of thread safety when we try changing the path during a running program. Therefor, in addition to bypassing PATH_MAX problems, working around buffer issues thanks to implementing with C++, we would also love to implement our functions without ever calling chdir() and changing the working directory. We'll now look into how we can make changes to our implementation to avoid a chdir() call, along with the pros and cons of techniques UNIX operating systems provide us, as well as what they're lacking.

If one looks back at how we implemented getcwd(), they'll notice that we call chdir("..") for each step in our loop. We need this to sanitize the location for these three calls:

opendir(".")
lstat(entry->d_name, &sb)
stat(".", &sb)

To remove the need of changing directories, we can place the ".." paths directly into our calls to opendir(), lstat(), and stat(). Thanks to C++ Strings being dynamic, it makes it easy for us to manipulate paths for these situations properly. We'll also see that being clever, we can even avoid string manipulation for some of these cases. And if our OSs supplied enough functionality, without any string manipulation at all!

Here's how we can implement getcwd() based off our previous implementation, but without calls to chdir(), with key details highlighted:

bool getcwd(std::string& path)
{
  typedef std::pair<dev_t, ino_t> file_id;

  bool success = false;
  struct stat sb;
  if (!stat(".", &sb))
  {
    file_id current_id(sb.st_dev, sb.st_ino);
    if (!stat("/", &sb)) //Get info for root directory, so we can determine when we hit it
    {
      std::vector<std::string> path_components;
      file_id root_id(sb.st_dev, sb.st_ino);
      std::string up_path("..");

      while (current_id != root_id) //If they're equal, we've obtained enough info to build the path
      {
        bool pushed = false;
        DIR *dir = opendir(up_path.c_str());
        if (dir)
        {
          dirent *entry;
          up_path += "/";
          std::string::size_type after_slash = up_path.size();
          while ((entry = readdir(dir))) //We loop through each entry trying to find where we came from
          {
            if (strcmp(entry->d_name, ".") && strcmp(entry->d_name, ".."))
            {
              up_path.replace(after_slash, std::string::npos, entry->d_name);
              if (!lstat(up_path.c_str(), &sb))
              {
                file_id child_id(sb.st_dev, sb.st_ino);
                if (child_id == current_id) //We found where we came from, add its name to the list
                {
                  path_components.push_back(entry->d_name);
                  pushed = true;
                  break;
                }
              }
            }
          }

          if (pushed && !fstat(dirfd(dir), &sb)) //If we have a reason to contiue, we update the current dir id
          {
            current_id = file_id(sb.st_dev, sb.st_ino);
            up_path.replace(after_slash, std::string::npos, ".."); //Keep recursing towards root each iteration
          }

          closedir(dir);
        }
        if (!pushed) { break; } //If we didn't obtain any info this pass, no reason to continue
      }

      if (current_id == root_id) //Unless they're equal, we failed above
      {
        //Built the path, will always end with a slash
        path = "/";
        for (std::vector<std::string>::reverse_iterator i = path_components.rbegin(); i != path_components.rend(); ++i)
        {
          path += *i+"/";
        }
        success = true;
      }
    }
  }

  return(success);
}

First of all, we removed all the code related to saving the current directory when we started, since we don't need that information anymore. Our first major change is that we created a new C++ String called up_path, we'll use this variable to keep track of the path, initializing it to "..". then for each step through the loop, make it "../..", "../../.." and so on, till we reach the root. We use this to replace our calls to opendir() to "." as we were doing before.
At this point, we'll add a slash to the path, and keep track of the spot with the variable after_slash. Now in our read directory loop, we can replace whatever is after the slash with the filename in the directory to pass to lstat(), again bypassing the need to be in the same directory as the file when making the function call.
Now for the stat() call on the directory itself, we got a little interesting. Instead of doing a path manipulation trick again, we call fstat() on the file descriptor returned from dirfd() on the already open directly handle. Notice how the call to close directory has been moved to after the block of code, so the directory is still open. And of course it's all wrapped up nicely with appending ".." after the slash.

Noticing how we eliminated path manipulation for the last part, it would be nice to eliminate more of it, especially if anyone wants to port this C++ code to C. The good news is that on Solaris and Linux we can, and as soon as the "at" functions get standardized, we can use them on the other UNIX OSs too. You can read more about them in one of our previous articles, File Descriptors and why we can't use them.

Here's how we can use the at functions to eliminate the path manipulation needed for calling lstat(), with the differences highlighted:

bool getcwd(std::string& path)
{
  typedef std::pair<dev_t, ino_t> file_id;

  bool success = false;
  struct stat sb;
  if (!stat(".", &sb))
  {
    file_id current_id(sb.st_dev, sb.st_ino);
    if (!stat("/", &sb)) //Get info for root directory, so we can determine when we hit it
    {
      std::vector<std::string> path_components;
      file_id root_id(sb.st_dev, sb.st_ino);
      std::string up_path("..");

      while (current_id != root_id) //If they're equal, we've obtained enough info to build the path
      {
        bool pushed = false;
        DIR *dir = opendir(up_path.c_str());
        if (dir)
        {
          int dir_fd = dirfd(dir);
          dirent *entry;
          while ((entry = readdir(dir))) //We loop through each entry trying to find where we came from
          {
            if (strcmp(entry->d_name, ".") && strcmp(entry->d_name, "..") && !fstatat(dir_fd, entry->d_name, &sb, AT_SYMLINK_NOFOLLOW))
            {
              file_id child_id(sb.st_dev, sb.st_ino);
              if (child_id == current_id) //We found where we came from, add its name to the list
              {
                path_components.push_back(entry->d_name);
                pushed = true;
                break;
              }
            }
          }

          if (pushed && !fstat(dir_fd, &sb)) //If we have a reason to contiue, we update the current dir id
          {
            current_id = file_id(sb.st_dev, sb.st_ino);
            up_path += "/.."; //Keep recursing towards root each iteration
          }

          closedir(dir);
        }
        if (!pushed) { break; } //If we didn't obtain any info this pass, no reason to continue
      }

      if (current_id == root_id) //Unless they're equal, we failed above
      {
        //Built the path, will always end with a slash
        path = "/";
        for (std::vector<std::string>::reverse_iterator i = path_components.rbegin(); i != path_components.rend(); ++i)
        {
          path += *i+"/";
        }
        success = true;
      }
    }
  }

  return(success);
}

As should be easily discernible, the string manipulation got easier, and mostly vanished. All the keeping track of a slash, and replaces has been replaced with only needing to append "/.." at the end of each loop.
Instead of manipulating a path string for our call to lstat(), we save the directory's file descriptor above (and subsequently use that instead of recalculating it lower down for fstat()), then use it to get the lstat() for each file in the directory, but now via fstatat().
fstatat() is the same as stat()/lstat(), but takes a directory descriptor as the first parameter to offset where the filename is relative to. The last parameter can be 0 or AT_SYMLINK_NOFOLLOW, which makes it act like stat() or lstat() respectively. fstatat() instead of a file descriptor can also take the special value AT_FDCWD to have it automatically work in the current directory.

This implementation should be much more elegant, but a bit less portable. If one would like to implement fstatat() themselves, it's not hard, here's how you can do it:

int fstatat(int dirfd, const char *pathname, struct stat *buf, int flags)
{
  int success = -1;

  if ((!flags || (flags == AT_SYMLINK_NOFOLLOW)))
  {
    int cwdfd = -1;
    if ((dirfd == AT_FDCWD) || (pathname && (*pathname == '/')) || (((cwdfd=open(".", O_RDONLY)) != -1) && !fchdir(dirfd)))
    {
      success = (!flags) ? stat(pathname, buf) : lstat(pathname, buf);
    }

    if (cwdfd != -1)
    {
      fchdir(cwdfd);
      close(cwdfd);
    }
  }
  else
  {
    errno = EINVAL;
  }

  return(success);
}

You'll also need the defines for AT_FDCWD and AT_SYMLINK_NOFOLLOW. You have to make sure that the value you choose for AT_FDCWD can't be a valid file descriptor or the normal failure return value. Therefor I chose -2 in my implementations, but choosing any negative value should be okay. AT_SYMLINK_NOFOLLOW shouldn't matter what it is, and a value of 1 or for all I care, 42, should be fine.
If your OS supports the at functions (currently Linux 2.6.16+ and recent versions of Solaris), it's better not to use a custom implementation, as this isn't thread safe, since it calls fchdir() internally, and for our example, runs counter to the whole point of using fstatat(). It would also be faster to use the original getcwd() implementation from a few days ago than using an emulated fstatat(), since there's less overhead from repeated fchdir() calls.
It's also interesting to note that glibc on Linux now, even for older than 2.6.16 implements fstatat() and the whole slew of at functions even when not supported in the Kernel. It's similar to ours, can be thread unsafe due to changing the working directory, and for some inexplicable reason, segfaulted in certain circumstances when I ran a large battery of tests on it and my implementation against the Kernel's to make sure that mine was working properly.

Anyways, with that out of the way, one can't hope but wonder if it would be possible to also eliminate the need of any path string manipulation for the call to opendir(). Mysteriously, neither Solaris nor Linux have an opendirat() call. If there was such, we could easily keep the previous directory open, till we obtained a new handle for its parent directory.
Baring having an opendirat() function, it'd be nice to implement such a thing. Some UNIX OSs I understand have a function perhaps named fopendir() or fdopendir() to promote a file descriptor to a directory handle. With such a function, we can simply write an opendirat() which calls openat(), then promotes it, but Linux doesn't have a method of promotion, although Solaris does (fdopendir()). Lets hope the people who are standardizing the new at functions for POSIX seriously consider what they're doing, otherwise we won't be able to use them.

Now that we've gotten getcwd() to be safe, we still need realpath() to be. In our realpath() implementation, the only unsafe parts was our function chdir_getcwd() which called chdir() internally as well as getcwd(). getcwd() is now taken care of, but we still need to rewrite the rest of chdir_getcwd() to not actually chdir().
To do such should now be easy, we simply do getcwd(), but with a new start path. We'd make a new function called getcwd_internal() which would take two parameters, one for the current directory, and another to initialize the up one, which we can wrap everything else around. Basically copy your favorite getcwd(), but make the modifications to the beginning like so:

bool getcwd_internal(std::string& path, const std::string& start_dir, std::string& up_path)
{
  typedef std::pair<dev_t, ino_t> file_id;

  bool success = false;
  struct stat sb;
  if (!stat(start_dir.c_str(), &sb))
  {
    file_id current_id(sb.st_dev, sb.st_ino);
    if (!stat("/", &sb)) //Get info for root directory, so we can determine when we hit it
    {
      std::vector<std::string> path_components;
      file_id root_id(sb.st_dev, sb.st_ino);

      while (current_id != root_id) //If they're equal, we've obtained enough info to build the path
--SNIP--

Now we'd turn getcwd() and chdir_getcwd() into wrappers like so:

bool getcwd(std::string& path)
{
  std::string up_path("..");
  return(getcwd_internal(path, ".", up_path));
}

bool chdir_getcwd(const std::string& dir, std::string& path)
{
  std::string up_path(dir+"/..");
  return(getcwd_internal(path, dir, up_path));
}

Note that the name of chdir_getcwd() is now misleading that it doesn't actually change directories anymore. For this reason, it's a good idea to not put any implementation details into a function name, as a user shouldn't have to know about them, and over time, it can become inaccurate.

And there you have it folks, getcwd() and realpath() implemented safely. Hopefully in the process all you loyal readers learned a couple other things too.

Other improvements over all we discussed, might be to run a smart pass over concatenated path names at key locations to remove extra slashes together "///", unneeded current directory "/./", and extra parent directories "path1/path2/.." -> "path1/", which is useful if certain functions like opendir() or the stat() family of calls have internal buffer issues. But doing such is a discussion for another time.

This wraps up our long discussion. All comments and suggestions are welcome.

Implementing realpath() in C++

Before reading this post, please read the background in the previous post PATH_MAX simply isn't, as it covers important background, and has some code we'll be building on over here.

Here's what realpath() is supposed to do:

realpath() expands all symbolic links and resolves references to ’/./’, ’/../’ and extra ’/’ characters in the null terminated string named by path and stores the canonicalized absolute pathname in the buffer of size PATH_MAX named by resolved_path. The resulting path will have no symbolic link, ’/./’ or ’/../’ components.

Instead of attacking implementing such a thing head on, we'll take a bottom up approach. One should consider that being able to break down a path between the path components prior to the last slash and after the last slash is essential to figuring out which directory the file actually lies.

The built in functions for this are called dirname() and basename(), but they modify their arguments, and they do some funny handling in special cases which we don't want.

So to implement our own for our purposes, we can write a function as follows:

void relative_dir_base_split(const std::string& path, std::string& dir, std::string& base)
{
  std::string::size_type slash_pos = path.rfind("/"); //Find the last slash
  if (slash_pos != std::string::npos) //If there is a slash
  {
    slash_pos++;
    dir = path.substr(0, slash_pos); //Directory is before slash
    base = path.substr(slash_pos); //And obviously, the file is after
  }
  else //Otherwise, there is no directory present
  {
    dir.clear(); 
    base = path;
  }
}

The function is simple enough, pass it a path, and two C++ strings for the return values, and have it split them up.

The next function which we'll use is one to get the realpath() of a directory. We can easily get such a thing by chdir()'ing to the directory, running our getcwd(), and then chdir() back.
This simple function is as follows:

bool chdir_getcwd(const std::string& dir, std::string& path)
{
  bool success = false;
  int start_fd = open(".", O_RDONLY); //Open current directory so we can save a handle to it
  if (start_fd != -1)
  {
    if (!chdir(dir.c_str())) //Change to directory
    {
      success = getcwd(path); //And get its path
      fchdir(start_fd); //And change back of course
    }
    close(start_fd);
  }
  return(success);
}

This simple function to resolve where a directory is located uses should be obvious, however, it's also a core function in figuring our where a file is, since a file is located at the realpath() of its parent directory, plus the base name for the file itself.

Now to get the realpath() of a file, we have this simple wrapper over the above functions:

static inline bool realpath_file(const std::string& path, std::string& resolved_path)
{
  bool success = false;
  std::string dir;
  std::string base;
  relative_dir_base_split(path, dir, base);

  //If there is a directory, get the realpath() for it, otherwise the current directory
  if (dir.size() ? chdir_getcwd(dir, resolved_path) : getcwd(resolved_path))
  {
    resolved_path += base;
    success = true;
  }
  return(success);
}

Now we have chdir_getcwd() which is basically a realpath() for a directory, and realpath_file() for files. Note however, that if you call realpath_file() directly, it won't actually ensure that the file exists, so probably a bad idea to call directly, unless you don't care if it exists or not. It also won't handle symlinks. To handle symlinks we need quite a bit more code.

First, let us write a wrapper in C++ for reading links:

bool readlink_internal(const std::string& path, std::string& buffer, ssize_t length)
{
  bool success = false;
  if (length > 0)
  {
    char *buf = new(nothrow) char[length+1]; //Room for Null
    if (buf)
    {
      ssize_t amount = ::readlink(path.c_str(), buf, length+1); //Give room for failure
      if ((amount > 0) && (amount <= length)) //If > length, it was modified mid check
      {
        buf[amount] = 0;
        buffer = buf;
        success = true;
      }
      delete[] buf;
    }
  }
  return(success);
}

This function simply wraps up the built in one, and uses C++ strings for dynamic allocation, so we don't have to worry about it directly in our function to resolve symlinks.

Before diving into the actual details in resolving a symlink, let me present one more helper function:

void build_path_base_swap(std::string &path, const std::string& newbase)
{
  string dir;
  string base;
  relative_dir_base_split(path, dir, base);

  if (dir.size())
  {
    path = dir + newbase;
  }
  else
  {
    path = newbase;
  }
}

This above function will take a path, and replace the base part of it, meaning if you pass it "path/cows.txt" and "feet.txt", path will become "path/feet.txt".

Now onto the symlink discussion.

Resolving symlinks in itself is hard though, since one symlink could lead to another, meaning we'll need a loop. We also need to defend ourselves against an infinite loop if one symlink links to itself, or a symlink earlier on in our resolve loop.

It is not safe enough to call a regular stat() to check for the final link existing (and not a loop), since during our resolve, an attacker can modify a link, so we have to detect that.
Unfortunately, most OSs/libraries have a define called MAXSYMLINKS which I've seen set anywhere from 8 to 30. They stop their loop once they've iterated MAXSYMLINKS times and return an errno of ELOOP. Unfortunately, doing such has two flaws, which I hope should be apparent. First of all, if a link links to itself immediately, it would take MAXSYMLINKS passes until the function finally realizes it (I'm looking at you OpenBSD). The other issue is that it's just wrong.
Hopefully any of us who has taken a data structures course would know that there are algorithms for detecting a loop in a linked list, and not have to resort to any dumb method which doesn't actually detect a loop. Also a number such as 8 is way too low which I've seen some do. I look at one of my Linux boxes, and I see some binaries are actually symlinked a dozen times. Like "upx" is symlinked to "upx-3.00", which is symlinked to "upx-3.00-nonfree", which is symlinked to, etc...

The optimal way to detect a loop in a linked list is explained here, see the last paragraph for the optimal O(N) solution. Unfortunately though, the optimal solution means we need to create a second iterator to check on symlinks, resolving links ahead of the main loop, two at a time. While great in memory, it can be a pain on a hard drive, where we have to deal with slow access, and the links can change during the check. Therefor I settled on a variation of the first solution, where we'll keep track of the file IDs (described in the previous post) of each symlink, and see if we ever hit a match. We'll store this data in an std::set, which is usually implemented as a Red & Black Tree which has O(log(N)) time for insertion and searching. Making our algorithm work at O(log(N))*2 (since we insert and find each time) * N, which reduces to O(N log(N)).

Here's the code:

bool symlink_resolve(const std::string& start, std::string& end)
{
  typedef std::pair<dev_t, ino_t> file_id;

  bool success = false;
  if (start.size())
  {
    std::string path = start; //Need a modifyable copy
    struct stat sb;
    std::set<file_id> seen_links;

    bool resolved_link;
    do //The symlink resolve loop
    {
      resolved_link = false;
      if (!lstat(path.c_str(), &sb))
      {
        file_id current_id(sb.st_dev, sb.st_ino);
        if (seen_links.find(current_id) == seen_links.end()) //Not a link we've seen
        {
          seen_links.insert(current_id); //Add to our set

          if (S_ISLNK(sb.st_mode)) //Another link
          {
            std::string newpath;
            if (readlink_internal(path, newpath, sb.st_size))
            {
              if (newpath[0] == '/') //Absolute
              {
                path = newpath;
              }
              else //We need to calculate the relative path in relation to the current
              {
                build_path_base_swap(path, newpath);
              }
              resolved_link = true;
            } //Else, Link can't be read, time to quit
          }
          else //Yay, it's not a link! got to the last part finally!
          {
            success = realpath_file(path, end);
          }
        } //Else, Nice try, someone linked a link back into a previous link during the scan to try to trick us into an infinite loop
      } //Else, Dangling link, can't resolve
    } while (resolved_link);
  }
  return(success);
}

And now with all those helper functions out of the way, we can finally write the actual realpath() function itself, and it's a heck of a lot simpler than other implementations you'll find out there.

Here it is:

bool realpath(const std::string& path, std::string& resolved_path, bool resolve_link = true)
{
  bool success = false;
  if (path.size())
  {
    struct stat sb;
    if (!stat(path.c_str(), &sb))
    {
      bool (*rp)(const std::string&, std::string&) = resolve_link ? symlink_resolve : realpath_file;
      success = S_ISDIR(sb.st_mode) ? chdir_getcwd(path, resolved_path) : rp(path, resolved_path);
    }
  }
  return(success);
}

It simply stat()s then runs chdir_getcwd() on directories and symlink_resolve() on files. I also offer an added feature. If you don't want the file name itself resolved, which is quite often what people want when a program saves a new file based off a name of another files passed to it, you can pass false as the third parameter, and it'll call realpath_file() instead of symlink_resolve(). With this wrapper, realpath_file() also has a stat() call before it, so the file is checked for existence like the built in realpath().

This implementation protects against the issues of PATH_MAX as described in our previous posts. It's also written in C++ to avoid buffer issues. Unlike built in ones, we generally have faster realpath() on directory, since it uses a getcwd() technique, while most other ones out there use a complicated set of steps on every component in the path until the last one is reached, which is especially slow if there are symlinks in the path. We also beat the built in ones regarding speed in cases of link to self, and have support for a deeper set of links. Some implementations also have some fixed sized buffer which can easily get full, and return failure, which is annoying, but our implementation won't run into. Lastly, we offer not resolving the links at all on the last component.

Drawbacks to our implementation is that it won't work on Windows as described in our last post. But you can #ifdef for that, and easily enough wrap around fullpath(). It also isn't thread safe due to the use of chdir() (which many built in implementations also do), see the previous post on how to work around that. Next time, we'll discuss methods to do all this without a chdir() at all, thus ensuring thread safety without any headaches.