1 2Miscellaneous Device control operations for the autofs kernel module 3==================================================================== 4 5The problem 6=========== 7 8There is a problem with active restarts in autofs (that is to say 9restarting autofs when there are busy mounts). 10 11During normal operation autofs uses a file descriptor opened on the 12directory that is being managed in order to be able to issue control 13operations. Using a file descriptor gives ioctl operations access to 14autofs specific information stored in the super block. The operations 15are things such as setting an autofs mount catatonic, setting the 16expire timeout and requesting expire checks. As is explained below, 17certain types of autofs triggered mounts can end up covering an autofs 18mount itself which prevents us being able to use open(2) to obtain a 19file descriptor for these operations if we don't already have one open. 20 21Currently autofs uses "umount -l" (lazy umount) to clear active mounts 22at restart. While using lazy umount works for most cases, anything that 23needs to walk back up the mount tree to construct a path, such as 24getcwd(2) and the proc file system /proc/<pid>/cwd, no longer works 25because the point from which the path is constructed has been detached 26from the mount tree. 27 28The actual problem with autofs is that it can't reconnect to existing 29mounts. Immediately one thinks of just adding the ability to remount 30autofs file systems would solve it, but alas, that can't work. This is 31because autofs direct mounts and the implementation of "on demand mount 32and expire" of nested mount trees have the file system mounted directly 33on top of the mount trigger directory dentry. 34 35For example, there are two types of automount maps, direct (in the kernel 36module source you will see a third type called an offset, which is just 37a direct mount in disguise) and indirect. 38 39Here is a master map with direct and indirect map entries: 40 41/- /etc/auto.direct 42/test /etc/auto.indirect 43 44and the corresponding map files: 45 46/etc/auto.direct: 47 48/automount/dparse/g6 budgie:/autofs/export1 49/automount/dparse/g1 shark:/autofs/export1 50and so on. 51 52/etc/auto.indirect: 53 54g1 shark:/autofs/export1 55g6 budgie:/autofs/export1 56and so on. 57 58For the above indirect map an autofs file system is mounted on /test and 59mounts are triggered for each sub-directory key by the inode lookup 60operation. So we see a mount of shark:/autofs/export1 on /test/g1, for 61example. 62 63The way that direct mounts are handled is by making an autofs mount on 64each full path, such as /automount/dparse/g1, and using it as a mount 65trigger. So when we walk on the path we mount shark:/autofs/export1 "on 66top of this mount point". Since these are always directories we can 67use the follow_link inode operation to trigger the mount. 68 69But, each entry in direct and indirect maps can have offsets (making 70them multi-mount map entries). 71 72For example, an indirect mount map entry could also be: 73 74g1 \ 75 / shark:/autofs/export5/testing/test \ 76 /s1 shark:/autofs/export/testing/test/s1 \ 77 /s2 shark:/autofs/export5/testing/test/s2 \ 78 /s1/ss1 shark:/autofs/export1 \ 79 /s2/ss2 shark:/autofs/export2 80 81and a similarly a direct mount map entry could also be: 82 83/automount/dparse/g1 \ 84 / shark:/autofs/export5/testing/test \ 85 /s1 shark:/autofs/export/testing/test/s1 \ 86 /s2 shark:/autofs/export5/testing/test/s2 \ 87 /s1/ss1 shark:/autofs/export2 \ 88 /s2/ss2 shark:/autofs/export2 89 90One of the issues with version 4 of autofs was that, when mounting an 91entry with a large number of offsets, possibly with nesting, we needed 92to mount and umount all of the offsets as a single unit. Not really a 93problem, except for people with a large number of offsets in map entries. 94This mechanism is used for the well known "hosts" map and we have seen 95cases (in 2.4) where the available number of mounts are exhausted or 96where the number of privileged ports available is exhausted. 97 98In version 5 we mount only as we go down the tree of offsets and 99similarly for expiring them which resolves the above problem. There is 100somewhat more detail to the implementation but it isn't needed for the 101sake of the problem explanation. The one important detail is that these 102offsets are implemented using the same mechanism as the direct mounts 103above and so the mount points can be covered by a mount. 104 105The current autofs implementation uses an ioctl file descriptor opened 106on the mount point for control operations. The references held by the 107descriptor are accounted for in checks made to determine if a mount is 108in use and is also used to access autofs file system information held 109in the mount super block. So the use of a file handle needs to be 110retained. 111 112 113The Solution 114============ 115 116To be able to restart autofs leaving existing direct, indirect and 117offset mounts in place we need to be able to obtain a file handle 118for these potentially covered autofs mount points. Rather than just 119implement an isolated operation it was decided to re-implement the 120existing ioctl interface and add new operations to provide this 121functionality. 122 123In addition, to be able to reconstruct a mount tree that has busy mounts, 124the uid and gid of the last user that triggered the mount needs to be 125available because these can be used as macro substitution variables in 126autofs maps. They are recorded at mount request time and an operation 127has been added to retrieve them. 128 129Since we're re-implementing the control interface, a couple of other 130problems with the existing interface have been addressed. First, when 131a mount or expire operation completes a status is returned to the 132kernel by either a "send ready" or a "send fail" operation. The 133"send fail" operation of the ioctl interface could only ever send 134ENOENT so the re-implementation allows user space to send an actual 135status. Another expensive operation in user space, for those using 136very large maps, is discovering if a mount is present. Usually this 137involves scanning /proc/mounts and since it needs to be done quite 138often it can introduce significant overhead when there are many entries 139in the mount table. An operation to lookup the mount status of a mount 140point dentry (covered or not) has also been added. 141 142Current kernel development policy recommends avoiding the use of the 143ioctl mechanism in favor of systems such as Netlink. An implementation 144using this system was attempted to evaluate its suitability and it was 145found to be inadequate, in this case. The Generic Netlink system was 146used for this as raw Netlink would lead to a significant increase in 147complexity. There's no question that the Generic Netlink system is an 148elegant solution for common case ioctl functions but it's not a complete 149replacement probably because its primary purpose in life is to be a 150message bus implementation rather than specifically an ioctl replacement. 151While it would be possible to work around this there is one concern 152that lead to the decision to not use it. This is that the autofs 153expire in the daemon has become far to complex because umount 154candidates are enumerated, almost for no other reason than to "count" 155the number of times to call the expire ioctl. This involves scanning 156the mount table which has proved to be a big overhead for users with 157large maps. The best way to improve this is try and get back to the 158way the expire was done long ago. That is, when an expire request is 159issued for a mount (file handle) we should continually call back to 160the daemon until we can't umount any more mounts, then return the 161appropriate status to the daemon. At the moment we just expire one 162mount at a time. A Generic Netlink implementation would exclude this 163possibility for future development due to the requirements of the 164message bus architecture. 165 166 167autofs Miscellaneous Device mount control interface 168==================================================== 169 170The control interface is opening a device node, typically /dev/autofs. 171 172All the ioctls use a common structure to pass the needed parameter 173information and return operation results: 174 175struct autofs_dev_ioctl { 176 __u32 ver_major; 177 __u32 ver_minor; 178 __u32 size; /* total size of data passed in 179 * including this struct */ 180 __s32 ioctlfd; /* automount command fd */ 181 182 /* Command parameters */ 183 union { 184 struct args_protover protover; 185 struct args_protosubver protosubver; 186 struct args_openmount openmount; 187 struct args_ready ready; 188 struct args_fail fail; 189 struct args_setpipefd setpipefd; 190 struct args_timeout timeout; 191 struct args_requester requester; 192 struct args_expire expire; 193 struct args_askumount askumount; 194 struct args_ismountpoint ismountpoint; 195 }; 196 197 char path[0]; 198}; 199 200The ioctlfd field is a mount point file descriptor of an autofs mount 201point. It is returned by the open call and is used by all calls except 202the check for whether a given path is a mount point, where it may 203optionally be used to check a specific mount corresponding to a given 204mount point file descriptor, and when requesting the uid and gid of the 205last successful mount on a directory within the autofs file system. 206 207The union is used to communicate parameters and results of calls made 208as described below. 209 210The path field is used to pass a path where it is needed and the size field 211is used account for the increased structure length when translating the 212structure sent from user space. 213 214This structure can be initialized before setting specific fields by using 215the void function call init_autofs_dev_ioctl(struct autofs_dev_ioctl *). 216 217All of the ioctls perform a copy of this structure from user space to 218kernel space and return -EINVAL if the size parameter is smaller than 219the structure size itself, -ENOMEM if the kernel memory allocation fails 220or -EFAULT if the copy itself fails. Other checks include a version check 221of the compiled in user space version against the module version and a 222mismatch results in a -EINVAL return. If the size field is greater than 223the structure size then a path is assumed to be present and is checked to 224ensure it begins with a "/" and is NULL terminated, otherwise -EINVAL is 225returned. Following these checks, for all ioctl commands except 226AUTOFS_DEV_IOCTL_VERSION_CMD, AUTOFS_DEV_IOCTL_OPENMOUNT_CMD and 227AUTOFS_DEV_IOCTL_CLOSEMOUNT_CMD the ioctlfd is validated and if it is 228not a valid descriptor or doesn't correspond to an autofs mount point 229an error of -EBADF, -ENOTTY or -EINVAL (not an autofs descriptor) is 230returned. 231 232 233The ioctls 234========== 235 236An example of an implementation which uses this interface can be seen 237in autofs version 5.0.4 and later in file lib/dev-ioctl-lib.c of the 238distribution tar available for download from kernel.org in directory 239/pub/linux/daemons/autofs/v5. 240 241The device node ioctl operations implemented by this interface are: 242 243 244AUTOFS_DEV_IOCTL_VERSION 245------------------------ 246 247Get the major and minor version of the autofs device ioctl kernel module 248implementation. It requires an initialized struct autofs_dev_ioctl as an 249input parameter and sets the version information in the passed in structure. 250It returns 0 on success or the error -EINVAL if a version mismatch is 251detected. 252 253 254AUTOFS_DEV_IOCTL_PROTOVER_CMD and AUTOFS_DEV_IOCTL_PROTOSUBVER_CMD 255------------------------------------------------------------------ 256 257Get the major and minor version of the autofs protocol version understood 258by loaded module. This call requires an initialized struct autofs_dev_ioctl 259with the ioctlfd field set to a valid autofs mount point descriptor 260and sets the requested version number in version field of struct args_protover 261or sub_version field of struct args_protosubver. These commands return 2620 on success or one of the negative error codes if validation fails. 263 264 265AUTOFS_DEV_IOCTL_OPENMOUNT and AUTOFS_DEV_IOCTL_CLOSEMOUNT 266---------------------------------------------------------- 267 268Obtain and release a file descriptor for an autofs managed mount point 269path. The open call requires an initialized struct autofs_dev_ioctl with 270the path field set and the size field adjusted appropriately as well 271as the devid field of struct args_openmount set to the device number of 272the autofs mount. The device number can be obtained from the mount options 273shown in /proc/mounts. The close call requires an initialized struct 274autofs_dev_ioct with the ioctlfd field set to the descriptor obtained 275from the open call. The release of the file descriptor can also be done 276with close(2) so any open descriptors will also be closed at process exit. 277The close call is included in the implemented operations largely for 278completeness and to provide for a consistent user space implementation. 279 280 281AUTOFS_DEV_IOCTL_READY_CMD and AUTOFS_DEV_IOCTL_FAIL_CMD 282-------------------------------------------------------- 283 284Return mount and expire result status from user space to the kernel. 285Both of these calls require an initialized struct autofs_dev_ioctl 286with the ioctlfd field set to the descriptor obtained from the open 287call and the token field of struct args_ready or struct args_fail set 288to the wait queue token number, received by user space in the foregoing 289mount or expire request. The status field of struct args_fail is set to 290the errno of the operation. It is set to 0 on success. 291 292 293AUTOFS_DEV_IOCTL_SETPIPEFD_CMD 294------------------------------ 295 296Set the pipe file descriptor used for kernel communication to the daemon. 297Normally this is set at mount time using an option but when reconnecting 298to a existing mount we need to use this to tell the autofs mount about 299the new kernel pipe descriptor. In order to protect mounts against 300incorrectly setting the pipe descriptor we also require that the autofs 301mount be catatonic (see next call). 302 303The call requires an initialized struct autofs_dev_ioctl with the 304ioctlfd field set to the descriptor obtained from the open call and 305the pipefd field of struct args_setpipefd set to descriptor of the pipe. 306On success the call also sets the process group id used to identify the 307controlling process (eg. the owning automount(8) daemon) to the process 308group of the caller. 309 310 311AUTOFS_DEV_IOCTL_CATATONIC_CMD 312------------------------------ 313 314Make the autofs mount point catatonic. The autofs mount will no longer 315issue mount requests, the kernel communication pipe descriptor is released 316and any remaining waits in the queue released. 317 318The call requires an initialized struct autofs_dev_ioctl with the 319ioctlfd field set to the descriptor obtained from the open call. 320 321 322AUTOFS_DEV_IOCTL_TIMEOUT_CMD 323---------------------------- 324 325Set the expire timeout for mounts within an autofs mount point. 326 327The call requires an initialized struct autofs_dev_ioctl with the 328ioctlfd field set to the descriptor obtained from the open call. 329 330 331AUTOFS_DEV_IOCTL_REQUESTER_CMD 332------------------------------ 333 334Return the uid and gid of the last process to successfully trigger a the 335mount on the given path dentry. 336 337The call requires an initialized struct autofs_dev_ioctl with the path 338field set to the mount point in question and the size field adjusted 339appropriately. Upon return the uid field of struct args_requester contains 340the uid and gid field the gid. 341 342When reconstructing an autofs mount tree with active mounts we need to 343re-connect to mounts that may have used the original process uid and 344gid (or string variations of them) for mount lookups within the map entry. 345This call provides the ability to obtain this uid and gid so they may be 346used by user space for the mount map lookups. 347 348 349AUTOFS_DEV_IOCTL_EXPIRE_CMD 350--------------------------- 351 352Issue an expire request to the kernel for an autofs mount. Typically 353this ioctl is called until no further expire candidates are found. 354 355The call requires an initialized struct autofs_dev_ioctl with the 356ioctlfd field set to the descriptor obtained from the open call. In 357addition an immediate expire that's independent of the mount timeout, 358and a forced expire that's independent of whether the mount is busy, 359can be requested by setting the how field of struct args_expire to 360AUTOFS_EXP_IMMEDIATE or AUTOFS_EXP_FORCED, respectively . If no 361expire candidates can be found the ioctl returns -1 with errno set to 362EAGAIN. 363 364This call causes the kernel module to check the mount corresponding 365to the given ioctlfd for mounts that can be expired, issues an expire 366request back to the daemon and waits for completion. 367 368AUTOFS_DEV_IOCTL_ASKUMOUNT_CMD 369------------------------------ 370 371Checks if an autofs mount point is in use. 372 373The call requires an initialized struct autofs_dev_ioctl with the 374ioctlfd field set to the descriptor obtained from the open call and 375it returns the result in the may_umount field of struct args_askumount, 3761 for busy and 0 otherwise. 377 378 379AUTOFS_DEV_IOCTL_ISMOUNTPOINT_CMD 380--------------------------------- 381 382Check if the given path is a mountpoint. 383 384The call requires an initialized struct autofs_dev_ioctl. There are two 385possible variations. Both use the path field set to the path of the mount 386point to check and the size field adjusted appropriately. One uses the 387ioctlfd field to identify a specific mount point to check while the other 388variation uses the path and optionally in.type field of struct args_ismountpoint 389set to an autofs mount type. The call returns 1 if this is a mount point 390and sets out.devid field to the device number of the mount and out.magic 391field to the relevant super block magic number (described below) or 0 if 392it isn't a mountpoint. In both cases the the device number (as returned 393by new_encode_dev()) is returned in out.devid field. 394 395If supplied with a file descriptor we're looking for a specific mount, 396not necessarily at the top of the mounted stack. In this case the path 397the descriptor corresponds to is considered a mountpoint if it is itself 398a mountpoint or contains a mount, such as a multi-mount without a root 399mount. In this case we return 1 if the descriptor corresponds to a mount 400point and and also returns the super magic of the covering mount if there 401is one or 0 if it isn't a mountpoint. 402 403If a path is supplied (and the ioctlfd field is set to -1) then the path 404is looked up and is checked to see if it is the root of a mount. If a 405type is also given we are looking for a particular autofs mount and if 406a match isn't found a fail is returned. If the the located path is the 407root of a mount 1 is returned along with the super magic of the mount 408or 0 otherwise. 409