Lines Matching +full:in +full:- +full:band +full:- +full:status
1 .. SPDX-License-Identifier: GPL-2.0
7 OrangeFS is an LGPL userspace scale-out parallel storage system. It is ideal
11 Orangefs, originally called PVFS, was first developed in 1993 by
47 OrangeFS is usually run in large installations with multiple servers and
51 On Fedora, install orangefs and orangefs-server::
53 dnf -y install orangefs orangefs-server
55 There is an example server configuration file in
61 There is an example client configuration file in /etc/pvfs2tab. It is a
64 pvfs2-client-core.
68 pvfs2-server -f /etc/orangefs/orangefs.conf
72 systemctl start orangefs-server
76 pvfs2-ping -m /pvfsmnt
78 Start the client. The module must be compiled in or loaded before this
81 systemctl start orangefs-client
85 mount -t pvfs2 tcp://localhost:3334/orangefs /pvfsmnt
102 You can omit --prefix if you don't care that things are sprinkled around
103 in /usr/local. As of version 2.9.6, OrangeFS uses Berkeley DB by
108 ./configure --prefix=/opt/ofs --with-db-backend=lmdb --disable-usrint
114 Create an orangefs config file by running pvfs2-genconfig and
115 specifying a target config file. Pvfs2-genconfig will prompt you
120 /opt/ofs/bin/pvfs2-genconfig /etc/pvfs2.conf
127 Create the mount point you specified in the tab file if needed::
133 /opt/ofs/sbin/pvfs2-server -f /etc/pvfs2.conf
137 /opt/ofs/sbin/pvfs2-server /etc/pvfs2.conf
139 Now the server should be running. Pvfs2-ls is a simple
142 /opt/ofs/bin/pvfs2-ls /pvfsmnt
147 /opt/ofs/sbin/pvfs2-client -p /opt/ofs/sbin/pvfs2-client-core
151 mount -t pvfs2 tcp://`hostname`:3334/orangefs /pvfsmnt
160 Make a second copy of the FileSystem section in the server configuration
171 pvfs2-server -f /etc/orangefs/orangefs.conf
182 ./check -pvfs2
195 filesystem can be interruptible, such as changes in debug levels
201 locking kicks in if the filesystem is mounted with -o local_lock.
208 If you want the debug (GOSSIP) statements in a particular
211 echo inode > /sys/kernel/debug/orangefs/kernel-debug
215 echo none > /sys/kernel/debug/orangefs/kernel-debug
219 echo inode,dir > /sys/kernel/debug/orangefs/kernel-debug
223 echo all > /sys/kernel/debug/orangefs/kernel-debug
227 cat /sys/kernel/debug/orangefs/debug-help
237 many of the important structures. Function and variable names in
247 ----------
249 At startup userspace allocates two page-size-aligned (posix_memalign)
258 mapping routine in the kernel module with an ioctl. The structure is
264 - a reference counter
265 * desc_size - PVFS2_BUFMAP_DEFAULT_DESC_SIZE (4194304) - the IO buffer's
267 is used for s_blocksize in super blocks.
268 * desc_count - PVFS2_BUFMAP_DEFAULT_DESC_COUNT (10) - the number of
269 partitions in the IO buffer.
270 * desc_shift - log2(desc_size), used for s_blocksize_bits in super blocks.
271 * total_size - the total size of the IO buffer.
272 * page_count - the number of 4096 byte pages in the IO buffer.
273 * page_array - a pointer to ``page_count * (sizeof(struct page*))`` bytes
275 to each of the pages in the IO buffer through a call to get_user_pages.
276 * desc_array - a pointer to ``desc_count * (sizeof(struct orangefs_bufmap_desc))``
280 structure. user_desc->ptr points to the IO buffer.
284 pages_per_desc = bufmap->desc_size / PAGE_SIZE
287 bufmap->desc_array[0].page_array = &bufmap->page_array[offset]
288 bufmap->desc_array[0].array_count = pages_per_desc = 1024
289 bufmap->desc_array[0].uaddr = (user_desc->ptr) + (0 * 1024 * 4096)
294 bufmap->desc_array[9].page_array = &bufmap->page_array[offset]
295 bufmap->desc_array[9].array_count = pages_per_desc = 1024
296 bufmap->desc_array[9].uaddr = (user_desc->ptr) +
300 * buffer_index_array - a desc_count sized array of ints, used to
302 * buffer_index_lock - a spinlock to protect buffer_index_array during update.
303 * readdir_index_array - a five (ORANGEFS_READDIR_DEFAULT_DESC_COUNT) element
306 * readdir_index_lock - a spinlock to protect readdir_index_array during
310 ----------
321 in flight at any given time.
326 - op was just initialized
328 - op is on request_list (upward bound)
330 - op is in progress (waiting for downcall)
332 - op has matching downcall; ok
334 - op has to start a timer since client-core
337 - submitter has given up waiting for it
347 function through a wait queue. Userspace is polling the pseudo-device
351 request list is searched for an op that seems ready-to-process.
353 the filled-out upcall struct are copy_to_user'ed back to userspace.
357 the request list. Otherwise, the op's state is changed to "in progress",
358 and the op is hashed on its tag and put onto the end of a list in the
363 the pseudo device in a series of io_vecs. This triggers the Orangefs
373 The "client-core" is the bridge between the kernel module and
374 userspace. The client-core is a daemon. The client-core has an
375 associated watchdog daemon. If the client-core is ever signaled
376 to die, the watchdog daemon restarts the client-core. Even though
377 the client-core is restarted "right away", there is a period of
378 time during such an event that the client-core is dead. A dead client-core
382 if the client-core stays dead too long, the arbitrary userspace processes
385 have their states set to "given up". In-progress ops that can't
391 - readdir ops use the smaller of the two pre-allocated pre-partitioned
397 - io (read and write) ops use the larger of the two pre-allocated
398 pre-partitioned memory buffers. The IO buffer is accessible from
406 Responses to kernel requests are all packaged in pvfs2_downcall_t
414 - type of operation.
415 ``int32_t status``
416 - return code for the operation.
418 - 0 unless readdir operation.
420 - initialized to NULL, used during readdir operations.
438 fill in a PVFS_sys_attr_s (tons of stuff the kernel doesn't need)
439 fill in a string with the link target when the object is a symlink.
446 us to know, in a timely fashion, these statistics about our
451 except its members are in a different order and "__pad1" is replaced
471 the readdir buffer descriptor specified in the upcall.
473 Userspace uses writev() on /dev/pvfs2-req to pass responses to the requests
478 - a pointer to the prepared response to the request from the
480 - and also, in the case of a readdir request, a pointer to a
481 buffer containing descriptors for the objects in the target
514 Orangefs exploits the dcache in order to avoid sending redundant
515 requests to userspace. We keep object inode attributes up-to-date with
518 Orangefs keeps private data in an object's inode that includes a short
526 Creation of a new object (file, dir, sym-link) includes the evaluation of
527 its pathname, resulting in a negative directory entry for the object.
535 dentry. If there is no corresponding dentry, one is created for it in
537 short timeout value in the dentry's d_time, and the dentry will be trusted
539 can potentially change out-of-band with any particular Orangefs kernel module
541 dentries is to always obtain the needed information from userspace - at
542 least a trip to the client-core, maybe to the servers. Obtaining information
547 code is designed to avoid the jiffy-wrap problem::
549 "In general, if the clock may have wrapped around more than once, there
552 difference in a way that takes into account the possibility that the