Opened 5 months ago
Closed 5 days ago
#897 closed defect (fixed)
Deleting directory tree fails on tmpfs
| Reported by: | Jiri Svoboda | Owned by: | Jiri Svoboda |
|---|---|---|---|
| Priority: | major | Milestone: | 0.16.1 |
| Component: | helenos/unspecified | Version: | mainline |
| Keywords: | Cc: | ||
| Blocker for: | Depends on: | ||
| See also: |
Description
Deleting a directory tree works with an Ext4 volume:
/ # cp -r /include /vol/test / # rm -rf /vol/test/include / #
but fails with tmpfs:
/ # cp -r /include /tmp / # rm -rf /tmp/include Can not remove /tmp/include/libc/elf (Method Not Supported) Can not remove /tmp/include/libc/io (Method Not Supported) Can not remove /tmp/include/libc/ipc (Method Not Supported) Can not remove /tmp/include/libc/loader (Method Not Supported) Can not remove /tmp/include/libc/rtld (Method Not Supported) Can not remove /tmp/include/libc/types (Method Not Supported) Can not remove /tmp/include/libc/vfs (Method Not Supported) Can not remove /tmp/include/libc/device (Method Not Supported) Can not remove /tmp/include/libc/adt (Method Not Supported) Can not remove /tmp/include/libc/libarch (Method Not Supported) Can not remove /tmp/include/libc/abi/ddi (Method Not Supported) Can not remove /tmp/include/libc/abi/proc (Method Not Supported) Can not remove /tmp/include/libc/abi (Method Not Supported) Can not remove /tmp/include/libc/_bits (Method Not Supported) Can not remove /tmp/include/libc (Method Not Supported) Can not remove /tmp/include/libinput/io (Method Not Supported) Can not remove /tmp/include/libinput (Method Not Supported) Can not remove /tmp/include/liboutput/io (Method Not Supported) Can not remove /tmp/include/liboutput (Method Not Supported) Can not remove /tmp/include/libconsole (Method Not Supported) Can not remove /tmp/include/libpcm/pcm (Method Not Supported) Can not remove /tmp/include/libpcm (Method Not Supported) Can not remove /tmp/include/libposix/sys (Method Not Supported) Can not remove /tmp/include/libposix (Method Not Supported) Can not remove /tmp/include/common/elf (Method Not Supported) Can not remove /tmp/include/common/io (Method Not Supported) Can not remove /tmp/include/common/ipc (Method Not Supported) Can not remove /tmp/include/common/loader (Method Not Supported) Can not remove /tmp/include/common/rtld (Method Not Supported) Can not remove /tmp/include/common/types (Method Not Supported) Can not remove /tmp/include/common/vfs (Method Not Supported) Can not remove /tmp/include/common/device (Method Not Supported) Can not remove /tmp/include/common/adt (Method Not Supported) Can not remove /tmp/include/common/libarch (Method Not Supported) Can not remove /tmp/include/common/abi/ddi (Method Not Supported) Can not remove /tmp/include/common/abi/proc (Method Not Supported) Can not remove /tmp/include/common/abi (Method Not Supported) Can not remove /tmp/include/common/_bits (Method Not Supported) Can not remove /tmp/include/common (Method Not Supported) Can not remove /tmp/include/libdisplay/display (Method Not Supported) Can not remove /tmp/include/libdisplay/types/display (Method Not Supported) Can not remove /tmp/include/libdisplay/types (Method Not Supported) Can not remove /tmp/include/libdisplay (Method Not Supported) Can not remove /tmp/include/libui/types/ui (Method Not Supported) Can not remove /tmp/include/libui/types (Method Not Supported) Can not remove /tmp/include/libui/ui (Method Not Supported) Can not remove /tmp/include/libui (Method Not Supported) Can not remove /tmp/include (Method Not Supported) / #
With Navigator or delete we also fail.
Deleting an individual file or empty directory on tmpfs succeeds.
Change History (7)
comment:1 by , 5 months ago
comment:2 by , 5 months ago
My intuition is that active opendir() should not "block" the directory for modification. Just as opening a file for reading does not block writes to that same file. Ideally, the filesystem would cache an atomic snapshot of the directory state at the time of opendir() and then list that snapshot regardless of what other changes to the filesystem are co-occuring.
comment:3 by , 5 months ago
That said, I think there also should be a "remove directory" macro method on the filesystem end. All that IPC back and forth when the only goal is to delete everything sounds like a performance nightmare.
comment:4 by , 2 months ago
In theory the file system can implement vfs_unlink() on a non-empty directory and rm will first try to do that, only falling back to deleting individual entries if that does not work. (libfmgt currently does not do that), but our file systems do not implement that.
I know what the problem is. Position in the directory is tracked by the client. 0 = beginning/first entry, 1 = second entry, etc. In tmpfs if we do vfs_read() at position n, it will return the nth entry in the linked list of directory entries.
What happens is, we have a directory containing A(0),B(1),C(2),D(3). We open it, position = 0, read A, pos = 1. We delete A, which means now we are at position 1, the directory contains B(0),C(1),D(2). We read C and delete it, moving to position 2. We now have B(0),D(1) and that's it. We only deleted half of the files. Then we try to unlink the directory, but that fails because it's not empty.
Not sure why this works on other file system types.
comment:5 by , 2 months ago
| Owner: | set to |
|---|---|
| Status: | new → accepted |
comment:6 by , 2 months ago
I don't think there is a perfect solution for this given how the file system framework is constructed. If open directory positions were tracked in the file system servers, we could provide a stable directory position pointer and it would just work.
In the current situations, some of these solutions, or possibly a combination of, could help:
- read a list of all files (or some, to avoid excessive memory usage) and delete them in batch
- seek back to directory start after deleting a file (batch of files)
- repeat operation multiple times until directory is empty
comment:7 by , 5 days ago
| Milestone: | → 0.16.1 |
|---|---|
| Resolution: | → fixed |
| Status: | accepted → closed |
I decided to go with a different approach. I modified tmpfs so that it provides stable directory positions. During the unlink operation we do not completely remove a directory entry, we just invalidate it (by setting the first character of the name to a null character). We only delete the entries when the parent directory is deleted, we delete all the invalid entries. This is similar to how e.g. FAT handles directories.
https://github.com/HelenOS/helenos/commit/576bc96d282280a3e002ead835ade32cd6207138
https://github.com/HelenOS/helenos/commit/34bc1839d62ae326aee68e246ddb97452c9d76fd
I think this might be because we have a directory open, read one entry after another and delete them. In other words, the directory is being modified while we read it and it seems this does not go well with tmpfs. If so, is it a bug/design issue in tmpfs, VFS or the file system utilities?