Take the following example:
FILE *fp = fopen("myfile.txt", "r+b")
if (fp)
{
if (verify_file_is_my_type(fp))
{
truncate("myfile.txt", 0); //Erase contents of file, change its size to 0
}
fclose(fp);
}
What if after I opened the file, while it was doing its lengthy check, the file was renamed, and a different file was renamed in place? I would be truncating the wrong file.
To solve problems like these, we can truncate the file directly via the file descriptor:
FILE *fp = fopen("myfile.txt", "r+b")
if (fp)
{
if (verify_file_is_my_type(fp))
{
ftruncate(fileno(fp), 0); //Erase contents of file, change its size to 0
}
fclose(fp);
}
With this method there is no TOCTOU race condition. Having a whole slew of file descriptor based functions such as fstat() and fchmod() for dealing with checking and setting permissions greatly alleviate security issues.
In certain cases, file descriptors can also seem to make certain operations easier. Say you're working with zlib and want to work with a gzip file but need to know how big the uncompressed file will be. Unfortunately, zlib doesn't provide a way to get this data, even though the data is quite simply the last 4 bytes in the file. One could open the file, read the last 4 bytes, close it, then open it with gzopen(), or one could take advantage of gzdopen() like so:
uint32_t gzfile_size(const char *filename)
{
uint32_t size = 0;
FILE *fp = fopen(filename, "rb");
if (fp)
{
uint32_t fsize, gzsize;
gzFile gfp;
fseek(fp, -4, SEEK_END); //Seek to the uncompressed size info
gzsize = fread4(fp); //The gzfile is read in using fread4()
//a custom function to read 4 bytes in little endian
fsize = ftell(fp); //At this point we can also get the external file size
rewind(fp); //Reset so zlib will start reading from the right place
//Open GZip file for decompression, use existing file handle
if ((gfp = gzdopen(fileno(fp), "rb"))) //Notice how we made use of gzdopen() and an fd
{
size = gzdirect(gzp) ? fsize : gzsize; //Since zlib can open non
//gzip'd files transparently, check for it
gzclose(gzp);
}
fclose(fp);
}
return size;
}
The above function can easily be modified to make it be more useful and actually read in the file to a buffer with gzread(), but this demonstrates how the file doesn't have to be closed and opened to get it's uncompressed size regardless if its a gzip file or not.
Another related area is what if directories changed while working with a particular path. Say for example the path to a set of settings files to my program was given as:
"/etc/program-a/" and in there I'd be dealing with files such as "net.conf", "sound.conf", and "ui.conf".
I could have my program strpcy()+strcat() or sprintf() the 3 needed files into a buffer large enough to hold any of those strings and use that buffer as needed to access any of those files, or alternatively, I could chdir() to "/etc/program-a/" and access the files directly as they are now part of the current directory.
The first method would be a problem if after I recieved "/etc/program-a/" it was renamed to something else, thus pointing to the wrong file, and it would also slow down the program a bit if for every file I wanted to access in a path, it would have to do many string operations.
The second method wouldn't be a problem if a path component was renamed, as UNIX based systems today can handle maintaining the representation of the CWD properly, nor would it need any relatively slow string operations to be performed. However having many instances of chdir() in a program could lead to maintenance hell, and would also be problematic or downright impossible to stay correct and secure when threads are involved.
To solve this series of problems, Solaris has invented the openat() and a whole family of *at() functions. Linux 2.6.16+ also now contains this nice family, and these functions are proposed for inclusion in a future revision of the POSIX standard.
In the above case, we'd do as follows:
int dirfd = open("/etc/program-a/", O_RDONLY);
Then to access any file:
int fd = openat(dirfd, "net.conf", O_RDWR);
int fd2 = openat(dirfd, "sound.conf", O_RDWR);
Then you can read()/write() on fd and fd2, or promote to a
FILE *fp = fdopen(fd, "r+");
And use fread()/fwrite().
Once we have dirfd from opening the directory in question, we never have to come up with any strings or change directories to interact with a file. If we wanted to stat a file, we could do:
struct stat sb;
fstatat(dirfd, "ui.conf", &sb, 0);
Or:
fstat(fd3, &sb); //fd3 acquired from openat() on dirfd and "ui.conf"
One can also get a directory file descriptor off of a
Once looking at *at(), I was able to rewrite some code I had which dealt with files all over the place and make it more secure, plus significantly faster as I was able to drop many string manipulating function calls.
Reading all this, one must be thinking to themselves one of two things. Either, wow this sounds great, I should look more into file descriptors, I hope all my supposedly secure apps use file descriptors and don't have any TOCTOU race conditions. Or, okay great I know all this already, what's your point?
However file descriptors have a general flaw, that being that THE FILE MUST BE OPENABLE. Say for example you have a file with the permissions of 000, you can run stat() or chmod() on it, but you can't alter the permissions with fchmod()! Now this might not seem so bad, but say you wanted to make the file writable then write something to it? Or worse, you want one single code base to do some operations in order not to commit the sin of code repetition, but you're faced with either using the safer file descriptor based function which doesn't always work, or the more dangerous file name based function which will work even if you can't open the file.
The problem gets even worse when one considers the new *at() functions Solaris and Linux added. In my case above, say my directory had permissions of 111 (--x--x--x), I can chdir() to it, or access file via the full path. But I can't call open() on "/etc/program-a/" as open() for directories only works if the directory has read permissions. If my program allowed one to pass a path to it to tell it where the config files were located, I would need to have two separate code paths, one the fast secure method using openat() and friends, and another calling open() and friends on the full path or chdir() to there and then open() on the file names directly.
Once this fatal flaw is realized, I see two possible solutions, one involving 3 new functions, or another involving the modification of 2 existing functions.
The first method would introduce the following:
int pathfd(const char *pathname);
int pathfdat(int dirfd, const char *pathname);
int openfd(int fd, int flags);
int openfd(int fd, int flags, mode_t mode);
pathfd() would allow one to get a file descriptor to a file as long as the directories leading up to it where all marked execute, and allow one to get a directory file descriptor if it was all marked execute including the directory itself. This would allow me to access any files for information functions or get a directory file descriptor to use with *at().
pathfdat() would allow one to get a file descriptor based of off another file descriptor in cases where using openat() would fail because there wasn't sufficient permission.
openfd() would allow one to promote an info descriptor into one I can read/write from. After I pathfd() and fchmod() a file to be writable, I can then openfd() promote it so I can write to it, all without having to worry about TOCTOU, with a chmod() and then open().
An alternative method would be to enhance the existing open() and fcntl() calls. open() and openat() can get a new flag perhaps labled O_INFO, or O_NONE, which would only need enough permissions as mentioned above for pathfd() and pathfdat().
For promotion, fcntl() and cmd F_DUPFD should be allowed to specify the args of O_RDONLY, O_WRONLY, or O_RDWR, so one could promote or demote a descriptor as needed.
Thoughts? Feel free to tell me if you agree or disagree, or why we have to be so limited.
If you agree with my ideas, any chance we can get the ball rolling with getting an OS or C/POSIX library group to implement some of these?
Posts
8 comments:
Windows, being somewhat younger than Unix, has a more graceful solution here. When you're opening an object, whether a file or not, you tell Windows what sort of access you want to the handle. This solves the access issue, by allowing the operation to succeed as long as you have enough access to do exactly what you need.
However, what's needed in general to solve these problems is the concept of locking. Locking allows you to acheive atomicity (also called transactional integrity). Oftentimes, file descriptors are used as a simple locking device, as they give you certain guarantees. (For example, an open file descriptor to a directory generally prevents the directory from being removed.) A fine-grained locking mechanism would be enough to ensure atomicity, and thus eliminate TOCTOU.
Locking has always been one of Unix's weak points, partly because the underlying filesystems are diverse. Networked file systems in particular are difficult to use with locking; the performance penalty of locking is often severe.
Another place locking and file descriptor operations fail is where we can't open something because the OS doesn't provide the proper primative. For example, Win32 has no atomic file replace operation like POSIX's rename(). Synthesizing one yields a race condition between removing the destination and changing the file name. The file can't be locked, because the rename operation itself destroys the file. What's really needed is a way to lock a particular file path, but not the file itself.
In practice, these aren't huge issues for end-programmers, as its usually easy to detect weird conditions, and abort to a consistent state if anything weird happens. It's a bigger issue for library writers, who don't have the convenience of knowing how exactly a particular function is being used.
Yes, you're touching the points I wanted to make.
When we open() a directory, we essentially lock it under UNIX, and using the *at() functions, we can go about our buisiness safely. This is a problem though when we can't call open() on a directory because it lacks read permission, when really the main permission for accessing directory contents is execute permission, not read.
Windows, being somewhat misdesigned, missed the point completely. Locking
is the wrong solution in a multiuser system, because it prevents other processes from progressing and requires coordination.
The POSIX solution is superior: once I opened a file for reading, I can slurp through it to the very end. Processes wishing to alter the file write to a new file in the same directory and rename it once done (actual code is harder than that). Readers know nothing about writers and writers know nothing about readers, but the file never in its existence is in an half written state.
However, the original poster's problem with openat() seems real and the problem to me seems to lie with opendir(), which assumes that you are opening the directory in order to sift through file names and should probably grow a flag saying "open for the benefit of the openat() family of functions".
Coming to think of it, with openat() it becomes obvious that opendir() and dirfd() are getting somewhat long in the tooth.
Thanks for provide great informatic and looking beautiful blog, really nice required information & the things i never imagined and i would request, wright more blog and blog post like that for us. Thanks you once agian
Birth certificate in delhi
Birth certificate in ghaziabad
Birth certificate in gurgaon
Birth certificate in noida
How to get birth certificate in ghaziabad
how to get birth certificate in delhi
birth certificate agent in delhi
how to download birth certificate
birth certificate in greater noida
birth certificate agent in delhi
Birth certificate delhi
Hehe, and only 4 years after your post you got O_PATH.
Hi.....
A file descriptor is a number that uniquely identifies an open file in a computer's operating system. It describes a data resource, and how that resource may be accessed. When a program asks to open a file — or another data resource, like a network socket — the kernel: Grants access.
You are also read more 100 Home Loan
Post a Comment