diff --git a/INSTALL b/INSTALL index 1884a83..01a5bcf 100644 --- a/INSTALL +++ b/INSTALL @@ -1,3 +1,5 @@ +Building Cpuset from Source +=========================== This is a distutils-based package and as such, it contains a number of differnet installation methods. By far the most convenient is to create an rpm. This rpm can then be later @@ -12,7 +14,7 @@ For basic installation: python setup.py install By default, the above command installs cpuset in the -$HOME/{bin,lib,share} directories. For a different location, +/usr/local/{bin,lib,share} directories. For a different location, use the --prefix option: python setup.py install --prefix=/usr @@ -20,3 +22,10 @@ use the --prefix option: For more information: http://docs.python.org/inst/inst.html + +Prebuild Binary Packages +======================== +Alternatively, you may download pre-built binary packages for multiple +distributions from the OpenSUSE Build Service here: + + http://download.opensuse.org/repositories/home:/tsariounov:/cpuset diff --git a/MANIFEST b/MANIFEST index b7f1162..4ed5f75 100644 --- a/MANIFEST +++ b/MANIFEST @@ -4,15 +4,15 @@ INSTALL MANIFEST MANIFEST.in Makefile +NEWS README -TODO cset setup.cfg setup.py cpuset/__init__.py +cpuset/config.py cpuset/cset.py cpuset/main.py -cpuset/task.py cpuset/util.py cpuset/version.py cpuset/commands/__init__.py @@ -37,5 +37,6 @@ doc/cset.1 doc/cset.html doc/cset.txt doc/tutorial.conf +doc/tutorial.html doc/tutorial.txt t/README diff --git a/MANIFEST.in b/MANIFEST.in index b937c86..c6c3730 100644 --- a/MANIFEST.in +++ b/MANIFEST.in @@ -1,3 +1,3 @@ -include README Makefile MANIFEST MANIFEST.in AUTHORS COPYING INSTALL ChangeLog TODO +include README Makefile MANIFEST MANIFEST.in AUTHORS COPYING INSTALL NEWS ChangeLog TODO include t/README include doc/*.txt doc/Makefile doc/*.conf doc/callouts.xsl doc/*.1 doc/*.html diff --git a/NEWS b/NEWS new file mode 100644 index 0000000..a34bfe8 --- /dev/null +++ b/NEWS @@ -0,0 +1,79 @@ +============================================================ + +Cpuset 1.5.0 (February 7, 2009) +https://forgesvn1.novell.com/svn/cpuset/tags/rel_1.5.0 +http://download.opensuse.org/repositories/home:/tsariounov:/cpuset/ + +Major upgrade since 1.04. This release includes a number of bug fixes and +enhancements as outlined below. The release number also has transitioned to a +three-digit number. This release has many usability enhancements from the +past months experience. It also includes a lengthy tutorial to introduce +users to the tool. + +The bug numbers refer to cpuset's bugzilla instance which can be found +here: http://devzilla.novell.com/cpuset + +Fixed Bugs: +* Fix bug #1: Origination cpuset not accepted for process move... +* Fix bug #2: Proc move from set to set does not recognize... +* Fix bug #3: Interspersed arguments do not work +* Fix bug #4: Add policy and priority to proc output list +* Fix bug #5: Add ability to move all threads in a process container wi. +* Fix bug #6: The -a option for set listing should be the default +* Fix bug #7: Add destination cpuset for the proc move command +* Fix bug #8: Let leading slash for nested cpuset spec be optional +* Fix bug #9: Rearrange info output some, include machine readable output +* Fix bug #10: Add a recursive cpuset delete with option +* Fix bug #11: Add a configuration file +* Fix bug #12: Add hex mask output for cpuset definition spec (cpuspec) +* Fix bug #13: Add cpuset rename option to set command +* Fix bug #21: Create shield shortcuts + +Other fixes: +* Changed behavior of shield command, now it behaves more like the proc move + command: -s or -u needs a -p to shield or unshield, however, the shortcut of + shield pidspec still works. +* Fixed exception when using cset with pipes +* Fixed set list behavior if --set was specified but not --list +* Added error message if proc list command does not specify a set +* Fixed exception if proc is used with -l and -s +* Location for cpuset filesystem mount can be specified in config file +* Changed proc option -a/--all to --force only +* Updated copyrights, URLs and documentation +* Added spec file to svn which is used in build service +* Cleaned up status output for clarity and consistency +* Additional minor corner case bug fixes included + +============================================================ + +Cpuset 1.04 (September 2, 2008) +https://forgesvn1.novell.com/svn/cpuset/tags/rel_1.04 +http://download.opensuse.org/repositories/home:/tsariounov:/cpuset/ + +Minor bug fixes and inclusion in SLERT10SP2 and slotted for SLES11. + +============================================================ + +Cpuset 1.03 (July 18, 2008) +https://forgesvn1.novell.com/svn/cpuset/tags/rel_1.03 +http://download.opensuse.org/repositories/home:/tsariounov:/cpuset/ + +Fixed class variables that made import of cset.as python library possible. + +============================================================ + +Cpuset 1.02 (June 10, 2008) +https://forgesvn1.novell.com/svn/cpuset/tags/rel_1.02 +http://download.opensuse.org/repositories/home:/tsariounov:/cpuset/ + +Minor fixes. + +============================================================ + +Cpuset 1.0 (May 30, 2008) +https://forgesvn1.novell.com/svn/cpuset/tags/rel_1.0 +http://download.opensuse.org/repositories/home:/tsariounov:/cpuset/ + +First full-featured public release complete with man pages and html docs. + +============================================================ diff --git a/README b/README index f4e3519..372cb7f 100644 --- a/README +++ b/README @@ -2,17 +2,23 @@ Cpuset is a Python application that forms a wrapper around the standard Linux filesystem calls to make using the cpusets facilities in the Linux kernel easier. -For the latest version see http://opensuse.org/build_service/cpuset -Bugs or feature requests should be sent to http://bugs.opensuse.org +For the latest version see: + http://developer.novell.com/wiki/index.php/cpuset + +For packages, see the OpenSUSE build service: + http://download.opensuse.org/repositories/home:/tsariounov:/Cpuset + +Bugs or feature requests should be sent to: + http://devzilla.novell.com/cpuset/ ----- -Copyright (C) 2008 Novell Inc. +Copyright (C) 2008, 2009 Novell Inc. Author: Alex Tsariounov This program is free software; you can redistribute it and/or modify -it under the terms of the GNU General Public License version 2 only as +it under the terms of the GNU General Public License version 2 as published by the Free Software Foundation. This program is distributed in the hope that it will be useful, diff --git a/cpuset.spec b/cpuset.spec new file mode 100644 index 0000000..e083285 --- /dev/null +++ b/cpuset.spec @@ -0,0 +1,82 @@ +# +# spec file for package cpuset (Version 1.04) +# +# Copyright (c) 2008 Novell, Inc. Waltham, MA, USA +# This file and all modifications and additions to the pristine +# package are under the same license as the package itself. +# +# Please submit bugfixes or comments via http://devzilla.novell.com/cpuset +# + +# norootforbuild + +Name: cpuset +Version: 1.5.0 +Release: 1 +License: GPL v2 only +BuildRoot: %{_tmppath}/%{name}-%{version}-build +Url: http://developer.novell.com/wiki/index.php/Cpuset +Group: System/Management +Summary: Allows manipulation of cpusets on system and provides higher level functions +Source: %{name}-%{version}.tar.gz +BuildRequires: python-devel + +%if 0%{?suse_version} > 0 +%py_requires +%endif + +%{!?python_sitelib: %define python_sitelib %(python -c "from distutils.sysconfig import get_python_lib; print get_python_lib()")} + +%description + +Cpuset is a Python application to make using the cpusets facilities in +the Linux kernel easier. The actual included command is called cset +and it allows manipulation of cpusets on the system and provides higher +level functions such as implementation and control of a basic cpu +shielding setup. + + +Authors: +-------- + Alex Tsariounov + +%prep +%setup + + +%build +CFLAGS="%{optflags}" \ +%{__python} setup.py build +#make doc ->not yet, asciidoc is missing... + + +%install +# Install binaries, but do not install docs via setup.py +%{__python} setup.py install --root=%{buildroot} --prefix=%{_prefix} --install-data=/eraseme +%{__rm} -rf %{buildroot}/eraseme + +# Install documentation +%{__mkdir_p} %{buildroot}/%{_defaultdocdir}/cpuset +%{__cp} NEWS README INSTALL AUTHORS COPYING %{buildroot}/%{_defaultdocdir}/cpuset/ +%{__mkdir_p} %{buildroot}/%{_mandir}/man1 +cd doc +%{__gzip} *.1 +%{__cp} *.1.gz %{buildroot}/%{_mandir}/man1 +%{__cp} *.txt %{buildroot}/%{_defaultdocdir}/cpuset/ +%{__mkdir} %{buildroot}/%{_defaultdocdir}/cpuset/html +%{__cp} *.html %{buildroot}/%{_defaultdocdir}/cpuset/html/ + + +%clean +%{__rm} -rf %{buildroot} + + +%files +%defattr(-,root,root) +%{_bindir}/cset +%{python_sitelib}/* +%{_mandir}/man1/* +%{_defaultdocdir}/* + + +%changelog diff --git a/cpuset/__init__.py b/cpuset/__init__.py index 87f611b..07a880a 100644 --- a/cpuset/__init__.py +++ b/cpuset/__init__.py @@ -1,5 +1,5 @@ __copyright__ = """ -Copyright (C) 2008 Novell Inc. +Copyright (C) 2008, 2009 Novell Inc. Author: Alex Tsariounov This program is free software; you can redistribute it and/or modify diff --git a/cpuset/commands/__init__.py b/cpuset/commands/__init__.py index 87f611b..07a880a 100644 --- a/cpuset/commands/__init__.py +++ b/cpuset/commands/__init__.py @@ -1,5 +1,5 @@ __copyright__ = """ -Copyright (C) 2008 Novell Inc. +Copyright (C) 2008, 2009 Novell Inc. Author: Alex Tsariounov This program is free software; you can redistribute it and/or modify diff --git a/cpuset/commands/common.py b/cpuset/commands/common.py index 41ac835..a829aab 100644 --- a/cpuset/commands/common.py +++ b/cpuset/commands/common.py @@ -2,7 +2,7 @@ """ __copyright__ = """ -Copyright (C) 2008 Novell Inc. +Copyright (C) 2008, 2009 Novell Inc. Author: Alex Tsariounov This program is free software; you can redistribute it and/or modify diff --git a/cpuset/commands/mem.py b/cpuset/commands/mem.py index af49f71..45c4417 100644 --- a/cpuset/commands/mem.py +++ b/cpuset/commands/mem.py @@ -2,7 +2,7 @@ """ __copyright__ = """ -Copyright (C) 2008 Novell Inc. +Copyright (C) 2008, 2009 Novell Inc. Author: Alex Tsariounov This program is free software; you can redistribute it and/or modify diff --git a/cpuset/commands/proc.py b/cpuset/commands/proc.py index 960abe0..8f785dc 100644 --- a/cpuset/commands/proc.py +++ b/cpuset/commands/proc.py @@ -2,7 +2,7 @@ """ __copyright__ = """ -Copyright (C) 2008 Novell Inc. +Copyright (C) 2008, 2009 Novell Inc. Author: Alex Tsariounov This program is free software; you can redistribute it and/or modify @@ -22,11 +22,15 @@ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA import sys, os, re, logging, pwd, grp from optparse import OptionParser, make_option +from cpuset import config from cpuset import cset from cpuset.util import * from cpuset.commands.common import * try: from cpuset.commands import set -except: pass +except SyntaxError: + raise +except: + pass global log log = logging.getLogger('proc') @@ -44,9 +48,9 @@ To list which tasks are running in a particular cpuset, use the --list command. For example: - # cset proc --list --set priset + # cset proc --list --set myset This command will list all the tasks running in the - cpuset called "priset". + cpuset called "myset". Processes are created by specifying the path to the executable and specifying the cpuset that the process is to be created in. @@ -56,6 +60,15 @@ For example: This command will execute the /usr/bin/fast_code program on the "blazing_cpuset" cpuset. +Note that if your command takes options, then use the traditional +"--" marker to separate cset's options from your command's +options. + +For example: + # cset proc --set myset --exec -- ls -l + This command will execute "ls -l" on the cpuset called + "myset". + The PIDSPEC argument taken for the move command is a comma separated list of PIDs or TIDs. The list can also include brackets of PIDs or TIDs (i.e. tasks) that are inclusive of the @@ -83,7 +96,7 @@ For example, the following commands are equivalent: running task between 3000 and 3200 inclusive of the ends to the cpuset called "reserved_set". -Specifying the --fromset is not necesary since the tasks will be +Specifying the --fromset is not necessary since the tasks will be moved to the destination cpuset no matter which cpuset they are currently running on. @@ -93,6 +106,12 @@ are running in the cpuset specified by --fromset will be moved. I.e., if there is a task running on the system but not in --fromset that is in PIDSPEC, it will not be moved. +If the --threads switch is used, then the proc command will +gather any threads of belonging to any processes or threads that +are specified in the PIDSPEC and move them. This provides an easy +way to move all related threads: just pick one TID from the set +and use the --threads option. + To move all userspace tasks from one cpuset to another, you need to specify the source and destination cpuset by name. @@ -105,11 +124,11 @@ Note that the move command will not move kernel threads unless the -k/--kthread switch is specified. If it is, then all unbound kernel threads will be added to the move. Unbound kernel threads are those that can run on any CPU. If you also specify the --a/--all switch, then all tasks, kernel or not, bound or not, +--force switch, then all tasks, kernel or not, bound or not, will be moved. -CAUTION: Please be cautious with the --all switch, since moving a -kernel thread that is bound to a specific CPU to a cpuset that +CAUTION: Please be cautious with the --force switch, since moving +a kernel thread that is bound to a specific CPU to a cpuset that does not include that CPU can cause a system hang. You must specify unique cpuset names for the both exec and move @@ -138,7 +157,9 @@ options = [make_option('-l', '--list', help = 'list processes in the specified cpuset', action = 'store_true'), make_option('-e', '--exec', - help = 'execute arguments in the specified cpuset', + help = 'execute arguments in the specified cpuset; ' + 'use the "--" option separator to separate ' + 'cset options from your exec\'ed command options', dest = 'exc', action = 'store_true'), make_option('-u', '--user', @@ -148,11 +169,17 @@ options = [make_option('-l', '--list', make_option('-m', '--move', help = 'move specified tasks to specified cpuset; ' 'to move a PIDSPEC to a cpuset, use -m PIDSPEC cpuset; ' - 'to move all tasks specify --fromset and --toset', + 'to move all tasks only specify --fromset and --toset', action = 'store_true'), make_option('-p', '--pid', metavar = 'PIDSPEC', - help = 'specify pid or tid specification'), + help = 'specify pid or tid specification for move'), + make_option("--threads", + help = 'if specified, any processes found in the PIDSPEC to have ' + 'multiple threads will automatically have all their threads ' + 'added to the PIDSPEC; use to move all related threads to a ' + 'cpuset', + action = 'store_true'), make_option('-s', '--set', metavar = 'CPUSET', help = 'specify name of immediate cpuset'), @@ -163,7 +190,7 @@ options = [make_option('-l', '--list', make_option('-k', '--kthread', help = 'move, or include moving, unbound kernel threads', action = 'store_true'), - make_option('-a', '--all', + make_option('--force', help = 'force all processes and threads to be moved', action = 'store_true'), make_option('-v', '--verbose', @@ -186,9 +213,11 @@ def func(parser, options, args): elif options.toset: tset = cset.unique_set(options.toset) elif len(args) > 0: - tset = cset.unique_set(args[0]) - if options.exc: del args[0] - else: tset = args + if options.exc: + tset = cset.unique_set(args[0]) + del args[0] + else: + tset = args else: raise CpusetException("cpuset not specified") try: @@ -203,57 +232,84 @@ def func(parser, options, args): return if options.move or options.kthread: - # first, we need to know the destination + fset = None tset = None + # first, we need to know the destination if options.toset: tset = cset.unique_set(options.toset) - elif options.set: + elif options.set and options.pid: + tset = cset.unique_set(options.set) + elif options.set and options.fromset: tset = cset.unique_set(options.set) elif len(args) > 0: if len(args) > 1 and options.pid == None: options.pid = args[0] - tset = cset.unique_set(args[1]) + if len(args) < 3: + tset = cset.unique_set(args[1]) + else: + # "-m 123 set1 set2" shortcut + fset = cset.unique_set(args[1]) + tset = cset.unique_set(args[2]) + # take care of set1->set2 shortcut + pids = pidspec_to_list(options.pid, threads=options.threads) + if len(pids) == 1: + try: + fset = cset.unique_set(pids[0]) + options.pid = None + except: + pass # must be real pidspec else: - tset = cset.unique_set(args[0]) + if len(args) < 2: + tset = cset.unique_set(args[0]) + else: + fset = cset.unique_set(args[0]) + tset = cset.unique_set(args[1]) else: raise CpusetException("destination cpuset not specified") set.active(tset) # next, if there is a pidspec, move just that if options.pid: - fset = None - if options.fromset and not options.all: + if options.fromset and not options.force: fset = cset.unique_set(options.fromset) - pids = pidspec_to_list(options.pid, fset) + elif options.toset and options.set: + fset = cset.unique_set(options.set) + pids = pidspec_to_list(options.pid, fset, options.threads) if len(pids): - l = [] - l.append('--> moving following pidspec: %s' % options.pid) - l.extend(task_detail_header(' ')) - l.extend(task_detail_table(pids, ' ', 76)) - log.info("\n".join(l)) - selective_move(None, tset, pids, options.kthread, options.all) + log.info('moving following pidspec: %s' % ','.join(pids)) + selective_move(None, tset, pids, options.kthread, options.force) + else: + log.info('**> no tasks moved') log.info('done') else: + fset = None # here we assume move everything from fromset to toset - if options.fromset == None: + if options.fromset: + fset = cset.unique_set(options.fromset) + elif options.set: + fset = cset.unique_set(options.set) + elif len(args) > 0: + # this must be the fromset, then... + fset = cset.unique_set(args[0]) + if fset == None: raise CpusetException("origination cpuset not specified") - fset = cset.unique_set(options.fromset) nt = len(fset.tasks) if nt == 0: raise CpusetException('no tasks to move from cpuset "%s"' % fset.path) if options.move: - log.info('--> moving all tasks from "%s" to "%s"...', - fset.path, tset.path) - selective_move(fset, tset, None, options.kthread, options.all) + log.info('moving all tasks from %s to %s', + fset.name, tset.path) + selective_move(fset, tset, None, options.kthread, options.force, + options.threads) else: - log.info('--> moving all kernel threads from "%s" to "%s"...', + log.info('moving all kernel threads from %s to %s', fset.path, tset.path) # this is a -k "move", so only move kernel threads pids = [] for task in fset.tasks: try: os.readlink('/proc/'+task+'/exe') except: pids.append(task) - selective_move(fset, tset, pids, options.kthread, options.all) + selective_move(fset, tset, pids, options.kthread, options.force) log.info('done') return @@ -261,22 +317,40 @@ def func(parser, options, args): list_sets(args) def list_sets(args): + log.debug("entering list_sets, args=%s", args) l = [] if isinstance(args, list): - for s in args: l.extend(cset.find_sets(s)) + for s in args: + if isinstance(s, str): + l.extend(cset.find_sets(s)) + elif not isinstance(s, cset.CpuSet): + raise CpusetException( + 'list_sets() args=%s, of which "%s" not a string or CpuSet' + % (args, s)) + else: + l.append(s) else: - l.extend(cset.find_sets(args)) + if isinstance(args, str): + l.extend(cset.find_sets(args)) + elif not isinstance(args, cset.CpuSet): + raise CpusetException( + "list_sets() passed args=%s, which is not a string or CpuSet" % args) + else: + l.append(args) + if len(l) == 0: + raise CpusetException("cpuset(s) to list not specified"); for s in l: if len(s.tasks) > 0: if verbose: - log_detailed_task_table(s, ' ') + log_detailed_task_table(s, ' ') else: - log_detailed_task_table(s, ' ', 76) + log_detailed_task_table(s, ' ', 78) else: log.info(cset.summary(s)) -def move(fromset, toset, plist=None): - log.debug('entering move, fromset=%s toset=%s list=%s', fromset, toset, plist) +def move(fromset, toset, plist=None, verb=None, force=None): + log.debug('entering move, fromset=%s toset=%s list=%s force=%s verb=%s', + fromset, toset, plist, force, verb) if isinstance(fromset, str): fset = cset.unique_set(fromset) elif not isinstance(fromset, cset.CpuSet) and plist == None: @@ -293,17 +367,39 @@ def move(fromset, toset, plist=None): tset = toset if plist == None: log.debug('moving default of all processes') - tset.tasks = fset.tasks - else: - tset.tasks = plist + if tset != fset and not force: + plist = fset.tasks + else: + raise CpusetException( + "cannot move tasks into their origination cpuset") + output = 0 + if verb: + output = verb + elif verbose: + output = verbose + if output: + l = [] + if config.mread: + l.append('move_tasks_start') + l.extend(task_detail_table(plist)) + l.append('move_tasks_stop') + else: + l.append(' ') + l.extend(task_detail_header(' ')) + if output > 1: + l.extend(task_detail_table(plist, ' ')) + else: + l.extend(task_detail_table(plist, ' ', 76)) + log.info("\n".join(l)) + # do the move... + tset.tasks = plist -def selective_move(fset, tset, plist=None, kthread=None, force=None): +def selective_move(fset, tset, plist=None, kthread=None, force=None, threads=None): log.debug('entering selective_move, fset=%s tset=%s plist=%s kthread=%s force=%s', fset, tset, plist, kthread, force) - target = cset.unique_set(tset) + task_check = [] tasks = [] task_heap = [] - task_check = [] utsk = 0 ktsk = 0 autsk = 0 @@ -311,12 +407,19 @@ def selective_move(fset, tset, plist=None, kthread=None, force=None): utsknr = 0 ktsknr = 0 ktskb = 0 + sstsk = 0 + target = cset.unique_set(tset) if fset: - task_check = cset.unique_set(fset).tasks + fset = cset.unique_set(fset) + if fset == target and not force: + raise CpusetException( + "error, same source/destination cpuset, use --force if ok") + task_check = fset.tasks if plist: task_heap = plist else: task_heap = cset.unique_set(fset).tasks + log.debug('processing task heap') for task in task_heap: try: # kernel threads do not have an excutable image @@ -326,15 +429,30 @@ def selective_move(fset, tset, plist=None, kthread=None, force=None): try: task_check.index(task) tasks.append(task) + log.debug(' added task %s', task) utsk += 1 - except: + if threads: + log.debug(' thread matching, looking for threads for task %s', task) + dirs = os.listdir('/proc/'+task+'/task') + if len(dirs) > 1: + for thread in dirs: + if thread != task: + log.debug(' adding thread %s', thread) + tasks.append(thread) + utsk += 1 + except ValueError: log.debug(' task %s not running in %s, skipped', task, fset.name) utsknr += 1 else: - tasks.append(task) - utsk += 1 - except: + if not force and cset.lookup_task_from_cpusets(task) == target.path: + log.debug(' task %s moving to orgination set %s, skipped', + task, target.path) + sstsk += 1 + else: + tasks.append(task) + utsk += 1 + except OSError: aktsk += 1 try: # this is in try because the task may not exist by the @@ -347,6 +465,10 @@ def selective_move(fset, tset, plist=None, kthread=None, force=None): if is_unbound(task): tasks.append(task) ktsk += 1 + elif cset.lookup_task_from_cpusets(task) == target.path: + log.debug(' task %s moving to orgination set %s, skipped', + task, target.path) + sstsk += 1 else: log.debug(' kernel thread %s is bound, not adding', task) @@ -356,31 +478,44 @@ def selective_move(fset, tset, plist=None, kthread=None, force=None): task) ktsknr += 1 # ok, move 'em - log.debug('moving %d tasks to "%s"...', len(tasks), tset.name) + log.debug('moving %d tasks to %s ...', len(tasks), tset.name) if len(tasks) == 0: - log.info('** no task matched move criteria') - if autsk > 0: + log.info('**> no task matched move criteria') + if sstsk > 0: + raise CpusetException('same source/destination cpuset, use --force if ok') + elif len(task_heap) > 0 and not kthread: + raise CpusetException('if you want to move kernel threads, use -k') + elif ktskb > 0: + raise CpusetException('kernel tasks are bound, use --force if ok') + return + if utsk > 0: l = [] l.append('moving') l.append(str(utsk)) - l.append('userspace tasks') - if utsknr > 0: - l.append('- not moving') - l.append(str(utsknr)) - l.append('tasks (not in fromset)') + l.append('userspace tasks to') + l.append(tset.path) + log.info(' '.join(l)) + if utsknr > 0: + l = [] + l.append('--> not moving') + l.append(str(utsknr)) + l.append('tasks (not in fromset, use --force)') log.info(' '.join(l)) if ktsk > 0 or kthread: l = [] l.append('moving') l.append(str(ktsk)) - l.append('kernel threads') - if ktskb > 0: - l.append('- not moving') - l.append(str(ktskb)) - l.append('threads (not unbound)') + l.append('kernel threads to:') + l.append(tset.path) + log.info(' '.join(l)) + if ktskb > 0: + l = [] + l.append('--> not moving') + l.append(str(ktskb)) + l.append('threads (not unbound, use --force)') log.info(' '.join(l)) if aktsk > 0 and force and not kthread and autsk == 0: - log.info('** not moving kernel threads since both --all and --kthread needed') + log.info('*** not moving kernel threads, need both --force and --kthread') if ktsknr > 0: l = [] l.append('--> not moving') @@ -399,8 +534,8 @@ def run(tset, args, usr_par=None, grp_par=None): log.debug('entering run, set=%s args=%s ', s.path, args) set.active(s) # check user - if usr_par: - try: + if usr_par: + try: user = pwd.getpwnam(usr_par)[2] except KeyError: try: @@ -434,8 +569,10 @@ def run(tset, args, usr_par=None, grp_par=None): os.execvp(args[0], args) def is_unbound(proc): - # FIXME: popen is slow... need to use sched_getaffinity() directly, - # but python doesn't have it... maybe use pyrex to wrap.... + # FIXME: popen is slow... + # --> use /proc//status -> Cpus_allowed + # int(line.replace(',',''), 16) + # note: delete leading zeros to compare to allcpumask line = os.popen('/usr/bin/taskset -p ' + str(proc), 'r').readline() aff = line.split()[-1] log.debug('is_unbound, proc=%s aff=%s allcpumask=%s', @@ -443,8 +580,10 @@ def is_unbound(proc): if aff == cset.allcpumask: return True return False -def pidspec_to_list(pidspec, fset=None): - log.debug('entering pidspecToList, pidspec=%s', pidspec) +def pidspec_to_list(pidspec, fset=None, threads=False): + """create a list of process ids out of a pidspec""" + log.debug('entering pidspecToList, pidspec=%s fset=%s threads=%s', + pidspec, fset, threads) if fset: if isinstance(fset, str): fset = cset.unique_set(fset) elif not isinstance(fset, cset.CpuSet): @@ -454,6 +593,7 @@ def pidspec_to_list(pidspec, fset=None): raise CpusetException('pidspec=%s is not a string' % pidspec) groups = pidspec.split(',') plist = [] + nifs = 0 if fset: chktsk = fset.tasks log.debug('parsing groups: %s', groups) for sub in groups: @@ -470,6 +610,7 @@ def pidspec_to_list(pidspec, fset=None): log.debug(' added single pid: %s', items[0]) except: log.debug(' task %s not running in %s, skipped', items[0], fset.name) + nifs += 1 else: plist.append(items[0]) log.debug(' added single pid: %s', items[0]) @@ -485,30 +626,76 @@ def pidspec_to_list(pidspec, fset=None): log.debug(' added task from range: %s', tsk) except: log.debug(' task %s not running in %s, skipped', tsk, fset.name) + nifs += 1 else: plist.extend(rng) log.debug(' added range of pids from %s-%s: %s', items[0], items[1], rng) else: raise CpusetException('pidspec=%s has bad group=%s' % (pidspec, items)) - log.debug('returning parsed pid list: %s', plist) - log.info('%s tasks match criteria', len(plist)) + log.debug('raw parsed pid list of %s tasks: %s', len(plist), plist) + if nifs > 0: + if nifs > 1: nmsg = "tasks" + else: nmsg = "task" + log.info('**> skipped %s %s, not in origination set "%s"', nifs, nmsg, fset.name) + log.debug('checking for duplicates...') + pdict = {} + dups = 0 + for task in plist: + if task in pdict: + dups += 1 + continue + pdict[task] = True + log.debug('found %s duplicates', dups) + if threads: + log.debug('thread matching activated, looking for threads...') + dups = 0 + hits = 0 + for task in pdict.keys(): + dirs = os.listdir('/proc/'+str(task)+'/task') + if len(dirs) > 1: + hits += 1 + for thread in dirs: + if thread in pdict: + dups += 1 + continue + pdict[thread] = True + log.debug('found %s multithreaded containers and %s duplicates', hits, dups) + plist = pdict.keys() + log.debug('returning parsed pid list of %s tasks: %s', len(plist), plist) return plist -def move_pidspec(pidspec, toset, fset=None): - log.debug('entering move_pidspec, pidspec=%s toset=%s', pidspec, toset) +def move_pidspec(pidspec, toset, fset=None, threads=False): + log.debug('entering move_pidspec, pidspec=%s toset=%s threads=%s', pidspec, toset, + threads) if not fset: - pids = pidspec_to_list(pidspec) + pids = pidspec_to_list(pidspec, None, threads) else: # if fromset is specified, only move tasks that are in pidspec # and are running in fromset log.debug('specified fset=%s', fset) - pids = pidspec_to_list(pidspec, fset) + pids = pidspec_to_list(pidspec, fset, threads) if len(pids) == 0: raise CpusetException('tasks do not match all criteria, none moved') move(None, toset, pids) -def task_detail(pid, width=65): - # get task details from /proc +def task_detail(pid, width=70): + # scheduler policy definitions + policy = ['o', 'f', 'r', 'b'] + # stat location definitions + statdef = { + 'pid': 0, + 'name': 1, + 'state': 2, + 'ppid': 3, + 'pgid': 4, + 'sid': 5, + 'priority': 17, + 'nice': 18, + 'numthreads': 19, + 'rtpriority': 39, + 'rtpolicy': 40, + } + # get task details from /proc, stat has rtprio/policy but not uid... pid = str(pid) if not os.access('/proc/'+pid, os.F_OK): raise CpusetException('task "%s" does not exist' % pid) @@ -519,28 +706,49 @@ def task_detail(pid, width=65): stdict[line.split()[0][:-1]] = line.split(':')[1].strip() except: pass # sometimes, we get an extra \n out of this file... + stat = file('/proc/'+pid+'/stat', 'r').readline() + stat = stat.split() cmdline = file('/proc/'+pid+'/cmdline').readline() # assume that a zero delimits the cmdline (it does now...) cmdline = cmdline.replace('\0', ' ') - used = 0 - out = pwd.getpwuid(int(stdict['Uid'].split()[0]))[0][:8].ljust(8) - used += 8 - out += stdict['Pid'].rjust(6) - used += 6 - out += stdict['PPid'].rjust(6) - used += 6 - out += stdict['State'].split()[0].center(3) - used += 3 + + out = [] + out.append(pwd.getpwuid(int(stdict['Uid'].split()[0]))[0][:8].ljust(8)) + out.append(stdict['Pid'].rjust(5)) + out.append(stdict['PPid'].rjust(5)) + + out2 = [] + out2.append(stdict['State'].split()[0]) + out2.append(policy[int(stat[statdef['rtpolicy']])]) + if stat[statdef['rtpolicy']] == '0': + out2.append('th') + elif stat[statdef['rtpolicy']] == '3': + out2.append('at') + else: + if int(stat[statdef['rtpriority']]) < 10: + out2.append('_') + out2.append(stat[statdef['rtpriority']]) + else: + out2.append(stat[statdef['rtpriority']].rjust(2)) + out.append(''.join(out2)) + try: os.readlink('/proc/'+pid+'/exe') #prog = stdict['Name'] + ' '.join(cmdline.split()[1:]) prog = cmdline except: prog = '['+stdict['Name']+']' - if width == 0: - out += prog - else: - out += prog[:(width-used)] + out.append(prog) + + if config.mread: + l2 = [] + for line in out: + l2.append(line.strip()) + return ';'.join(l2) + + out = ' '.join(out) + if width != 0 and len(out) >= width: + out = out[:width-3] + "..." return out @@ -548,8 +756,8 @@ def task_detail_header(indent=None): if indent == None: istr = "" else: istr = indent l = [] - l.append(istr + 'USER PID PPID S TASK NAME') - l.append(istr + '-------- ----- ----- - ---------') + l.append(istr + 'USER PID PPID SPPr TASK NAME') + l.append(istr + '-------- ----- ----- ---- ---------') return l def task_detail_table(pids, indent=None, width=None): @@ -565,8 +773,13 @@ def log_detailed_task_table(set, indent=None, width=None): log.debug("entering print_detailed_task_table, set=%s indent=%s width=%s", set.path, indent, width) l = [] - l.append(cset.summary(set)) - l.extend(task_detail_header(indent)) - l.extend(task_detail_table(set.tasks, indent, width)) + if not config.mread: + l.append(cset.summary(set)) + l.extend(task_detail_header(indent)) + l.extend(task_detail_table(set.tasks, indent, width)) + else: + l.append('proc_list_start-' + set.name) + l.extend(task_detail_table(set.tasks)) + l.append('proc_list_stop-' + set.name) log.info("\n".join(l)) diff --git a/cpuset/commands/set.py b/cpuset/commands/set.py index c2076df..377c0b5 100644 --- a/cpuset/commands/set.py +++ b/cpuset/commands/set.py @@ -2,7 +2,7 @@ """ __copyright__ = """ -Copyright (C) 2008 Novell Inc. +Copyright (C) 2008, 2009 Novell Inc. Author: Alex Tsariounov This program is free software; you can redistribute it and/or modify @@ -22,11 +22,15 @@ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA import sys, os, logging from optparse import OptionParser, make_option +from cpuset import config from cpuset import cset from cpuset.util import * from cpuset.commands.common import * try: from cpuset.commands import proc -except: pass +except SyntaxError: + raise +except: + pass global log log = logging.getLogger('set') @@ -43,7 +47,7 @@ A cpuset is an organizational unit that defines a group of CPUs and a group of memory nodes where a process or thread (i.e. task) is allowed to run on. For non-NUMA machines, the memory node is always 0 (zero) and cannot be set to anything else. For NUMA -machines, the memory node can be set to a similar specifcation +machines, the memory node can be set to a similar specification as the CPU definition and will tie those memory nodes to that cpuset. You will usually want the memory nodes that belong to the CPUs defined to be in the same cpuset. @@ -113,7 +117,7 @@ Create a cpuset that specifies both CPUs and memory nodes: Note that this command uses the full path method to specify the name of the new cpuset "/rad/set_one". It - also names the new cpuset implicitily (i.e. no --set + also names the new cpuset implicitly (i.e. no --set option, although you can use that if you want to). If the "set_one" name is unique, you can subsequently refer to is just by that. Memory node 3 is assigned to this @@ -124,8 +128,7 @@ already exist, they will modify them to the new specifications.""" verbose = 0 options = [make_option('-l', '--list', - help = 'list the named cpuset(s); recursive list if also -r; ' - 'members if also -a', + help = 'list the named cpuset(s); recursive list if also -r', action = 'store_true'), make_option('-c', '--cpu', help = 'create or modify cpuset in the specified ' @@ -133,19 +136,26 @@ options = [make_option('-l', '--list', metavar = 'CPUSPEC'), make_option('-m', '--mem', help = 'specify which memory nodes to assign ' - 'to the created or modified cpuset', + 'to the created or modified cpuset (optional)', metavar = 'MEMSPEC'), + make_option('-n', '--newname', + help = 'rename cpuset specified with --set to NEWNAME'), make_option('-d', '--destroy', help = 'destroy specified cpuset', action = 'store_true'), make_option('-s', '--set', metavar = 'CPUSET', help = 'specify cpuset'), - make_option('-a', '--all', - help = 'also do listing of member cpusets', - action = 'store_true'), make_option('-r', '--recurse', - help = 'do recursive listing, for use with --list', + help = 'do things recursively, use with --list and --destroy', + action = 'store_true'), + make_option('--force', + help = 'force recursive deletion even if processes are running ' + 'in those cpusets (they will be moved to parent cpusets)', + action = 'store_true'), + make_option('-x', '--usehex', + help = 'use hexadecimal value for CPUSPEC and MEMSPEC when ' + 'listing cpusets', action = 'store_true'), make_option('-v', '--verbose', help = 'prints more detailed output, additive', @@ -167,10 +177,10 @@ def func(parser, options, args): if options.list: if options.set: - list_sets(options.set, options.recurse, options.all) + list_sets(options.set, options.recurse, options.usehex) return - if len(args): list_sets(args, options.recurse, options.all) - else: list_sets('root', options.recurse, options.all) + if len(args): list_sets(args, options.recurse, options.usehex) + else: list_sets('root', options.recurse, options.usehex) return if options.cpu or options.mem: @@ -178,21 +188,27 @@ def func(parser, options, args): create_from_options(options, args) return + if options.newname: + rename_set(options, args) + return + if options.destroy: - if options.set: destroy_sets(options.set) - else: destroy_sets(args) + if options.set: destroy_sets(options.set, options.recurse, options.force) + else: destroy_sets(args, options.recurse, options.force) return if options.cpu_exclusive or options.mem_exclusive: - # modification of existing cpusets for exclusivity + # FIXME: modification of existing cpusets for exclusivity return # default behavior if no options specified is list log.debug('no options set, default is listing cpusets') - if len(args): list_sets(args, options.recurse, options.all) - else: list_sets('root', options.recurse, options.all) + if options.set: list_sets(options.set, options.recurse, options.usehex) + elif len(args): list_sets(args, options.recurse, options.usehex) + else: list_sets('root', options.recurse, options.usehex) -def list_sets(tset, recurse=None, members=None): +def list_sets(tset, recurse=None, usehex=False): + """list cpusets specified in tset as cpuset or list of cpusets, recurse if true""" log.debug('entering list_sets, tset=%s recurse=%s', tset, recurse) sl = [] if isinstance(tset, list): @@ -200,51 +216,74 @@ def list_sets(tset, recurse=None, members=None): else: sl.extend(cset.find_sets(tset)) log.debug('total unique sets in passed tset: %d', len(sl)) - if recurse: members = True - if members: - sl2 = [] - for s in sl: - sl2.append(s) - if len(s.subsets) > 0: - sl2.extend(s.subsets) - if recurse: - for node in s.subsets: - for nd in cset.walk_set(node): - sl2.append(nd) - sl = sl2 - pl = [''] - pl.extend(set_header(' ')) + sl2 = [] + for s in sl: + sl2.append(s) + if len(s.subsets) > 0: + sl2.extend(s.subsets) + if recurse: + for node in s.subsets: + for nd in cset.walk_set(node): + sl2.append(nd) + sl = sl2 + if config.mread: + pl = ['cpuset_list_start'] + else: + pl = [''] + pl.extend(set_header(' ')) + for s in sl: if verbose: - pl.append(set_details(s,' ', 0)) + pl.append(set_details(s,' ', None, usehex)) else: - pl.append(set_details(s,' ')) + pl.append(set_details(s,' ', 78, usehex)) + + if config.mread: + pl.append('cpuset_list_end') log.info("\n".join(pl)) -def destroy_sets(sets): - log.debug('enter destroy_sets, sets=%s', sets) +def destroy_sets(sets, recurse=False, force=False): + """destroy cpusets in list of sets, recurse if true, if force destroy if tasks running""" + log.debug('enter destroy_sets, sets=%s, force=%s', sets, force) nl = [] try: nl.extend(sets) except: nl.append(sets) # check that sets passed are ok, will raise if one is bad + sl2 = [] for s in nl: st = cset.unique_set(s) + sl2.append(st) if len(st.subsets) > 0: - raise CpusetException('cpuset "%s" has subsets, delete them first' - % st.path) + if not recurse: + raise CpusetException( + 'cpuset "%s" has subsets, delete them first, or use --recurse' + % st.path) + elif not force: + raise CpusetException( + 'cpuset "%s" has subsets, use --force to destroy' + % st.path) + sl2.extend(st.subsets) + for node in st.subsets: + for nd in cset.walk_set(node): + sl2.append(nd) + # ok, good to go - for s in nl: + if recurse: sl2.reverse() + for s in sl2: s = cset.unique_set(s) + # skip the root set!!! or you'll have problems... + if s.path == '/': continue log.info('--> processing cpuset "%s", moving %s tasks to parent "%s"...', s.name, len(s.tasks), s.parent.path) proc.move(s, s.parent) - log.info('deleting cpuset "%s"', s.path) + log.info('--> deleting cpuset "%s"', s.path) destroy(s) log.info('done') def destroy(name): + """destroy one cpuset by name as cset or string""" log.debug('entering destroy, name=%s', name) if isinstance(name, str): set = cset.unique_set(name) @@ -261,7 +300,40 @@ def destroy(name): # fixme: perhaps reparsing the all the sets is not so efficient... cset.rescan() +def rename_set(options, args): + """rename cpuset as specified in options and args lists""" + log.debug('entering rename_set, options=%s args=%s', options, args) + # figure out target cpuset name, if --set not used, use first arg + name = options.newname + if options.set: + tset = cset.unique_set(options.set) + elif len(args) > 0: + tset = cset.unique_set(args[0]) + else: + raise CpusetException('desired cpuset not specified') + path = tset.path[0:tset.path.rfind('/')+1] + log.debug('target set="%s", path="%s", name="%s"', tset.path, path, name) + try: + if name.find('/') == -1: + chk = cset.unique_set(path+name) + else: + if name[0:name.rfind('/')+1] != path: + raise CpusetException('desired name cannot have different path') + chk = cset.unique_set(name) + raise CpusetException('cpuset "'+chk.path+'" already exists') + except CpusetNotFound: + pass + except: + raise + + if name.rfind('/') != -1: + name = name[name.rfind('/')+1:] + log.info('--> renaming "%s" to "%s"', cset.CpuSet.basepath+tset.path, name) + os.rename(cset.CpuSet.basepath+tset.path, cset.CpuSet.basepath+path+name) + cset.rescan() + def create_from_options(options, args): + """create cpuset as specified by options and args lists""" log.debug('entering create_from_options, options=%s args=%s', options, args) # figure out target cpuset name, if --set not used, use first arg if options.set: @@ -292,6 +364,7 @@ def create_from_options(options, args): active(tset) def create(name, cpuspec, memspec, cx, mx): + """create one cpuset by name, cpuspec, memspec, cpu and mem exclusive flags""" log.debug('entering create, name=%s cpuspec=%s memspec=%s cx=%s mx=%s', name, cpuspec, memspec, cx, mx) try: @@ -310,6 +383,7 @@ def create(name, cpuspec, memspec, cx, mx): modify(name, cpuspec, memspec, cx, mx) def modify(name, cpuspec=None, memspec=None, cx=None, mx=None): + """modify one cpuset by name, cpuspec, memspec, cpu and mem exclusive flags""" log.debug('entering modify, name=%s cpuspec=%s memspec=%s cx=%s mx=%s', name, cpuspec, memspec, cx, mx) if isinstance(name, str): @@ -326,6 +400,7 @@ def modify(name, cpuspec=None, memspec=None, cx=None, mx=None): if mx: nset.mem_exclusive = mx def active(name): + """check that cpuset by name or cset is ready to be used""" log.debug("entering active, name=%s", name) if isinstance(name, str): set = cset.unique_set(name) @@ -339,55 +414,66 @@ def active(name): raise CpusetException('"%s" cpuset not active, no mems defined' % set.path) def set_header(indent=None): + """return list of cpuset output header""" if indent: istr = indent else: istr = '' l = [] - # '1234567890-1234567890-1234567890-1234567890-1234567890' + # '123456789-123456789-123456789-123456789-123456789-123456789-' l.append(istr + ' Name CPUs-X MEMs-X Tasks Subs Path') l.append(istr + '------------ ---------- - ------- - ----- ---- ----------') return l -def set_details(name, indent=None, width=75): +def set_details(name, indent=None, width=None, usehex=False): + """return string of cpuset details""" + if width == None: width = 0 if isinstance(name, str): set = cset.unique_set(name) elif not isinstance(name, cset.CpuSet): raise CpusetException("passing bogus set=%s" % name) else: set = name - if indent: istr = indent - else: istr = '' + l = [] - used = 0 - l.append(istr) - used += len(istr) l.append(set.name.rjust(12)) - used += 12 cs = set.cpus if cs == '': cs = '*****' - l.append(cs.rjust(11)) - used += 11 + elif usehex: cs = cset.cpuspec_to_hex(cs) + l.append(cs.rjust(10)) if set.cpu_exclusive: - l.append(' y') + l.append('y') else: - l.append(' n') - used += 2 + l.append('n') cs = set.mems if cs == '': cs = '*****' - l.append(cs.rjust(8)) - used += 8 + elif usehex: cs = cset.cpuspec_to_hex(cs) + l.append(cs.rjust(7)) if set.mem_exclusive: - l.append(' y') + l.append('y') else: - l.append(' n') - used += 2 - l.append(str(len(set.tasks)).rjust(6)) - used += 6 - l.append(str(len(set.subsets)).rjust(5)) - used += 5 - l.append(' ') - used += 1 - if width == 0: + l.append('n') + l.append(str(len(set.tasks)).rjust(5)) + l.append(str(len(set.subsets)).rjust(4)) + + if config.mread: l.append(set.path) + l2 = [] + for line in l: + l2.append(line.strip()) + return ';'.join(l2) + + out = ' '.join(l) + ' ' + tst = out + set.path + + if width != 0 and len(tst) > width: + target = width - len(out) + patha = set.path[:len(set.path)/2-3] + pathb = set.path[len(set.path)/2:] + patha = patha[:target/2-3] + pathb = pathb[-target/2:] + out += patha + '...' + pathb else: - l.append(set.path[:(width-used)]) - return ''.join(l) + out = tst + + if indent: istr = indent + else: istr = '' + return istr + out diff --git a/cpuset/commands/shield.py b/cpuset/commands/shield.py index f151232..7f2cedc 100644 --- a/cpuset/commands/shield.py +++ b/cpuset/commands/shield.py @@ -2,7 +2,7 @@ """ __copyright__ = """ -Copyright (C) 2008 Novell Inc. +Copyright (C) 2008, 2009 Novell Inc. Author: Alex Tsariounov This program is free software; you can redistribute it and/or modify @@ -27,6 +27,7 @@ from cpuset.commands import proc from cpuset.commands import set from cpuset import cset from cpuset.util import * +from cpuset import config global log log = logging.getLogger('shield') @@ -103,16 +104,16 @@ shielded cpuset with the --exec subcommand, or move processes or threads already running to the shielded cpuset with the --shield subcommand. -The PIDSPEC argument taken for the --shield (or -s) subcommand is -a comma separated list of PIDs or TIDs. The list can also -include brackets of PIDs or TIDs that are inclusive of the -endpoints. +The PIDSPEC argument taken for the --pid (or -p) option (used in +conjunction with a --shield or --unshield command) is a comma +separated list of PIDs or TIDs. The list can also include +brackets of PIDs or TIDs that are inclusive of the endpoints. For example: 1,2,5 Means processes 1, 2 and 5 1,2,600-700 Means processes 1, 2 and from 600 to 700 - # cset shield --shield=50-65 + # cset shield --shield --pid=50-65 This command moves all processes and threads with PID or TID in the range 50-65 inclusive, from any cpuset they may be running in into the shielded user cpuset. @@ -122,24 +123,18 @@ position populated. In other words, for the example above, if there is only one process, say PID 57, in the range of 50-65, then only that process will be moved. -DANGER: Please note that there is no checking of processes you -request to move into the shield with the --shield command. This -means that the tool will happily move, for example, kernel -threads that are bound to specific CPUs with this command. You -can hang your system by indiscriminately specifying arbitrary -PIDs to the --shield command so please be careful. - The --unshield (or -u) subcommand will remove the specified processes or threads from the shielded cpuset and move them into -the unshielded (or system) cpuset. This option also takes a -PIDSPEC argument, the same as for the --shield subcommand. +the unshielded (or system) cpuset. This option is use with a +--pid and a PIDSPEC argument, the same as for the --shield +subcommand. Both the --shield and the --unshield commands will also finally output the number of tasks running in the shield and out of the -shield. If you do not specify a PIDSPEC to these commands, then -just that status is output. By specifying also a --verbose in -addition, then you will get a listing of every task that is -running in either the shield or out of the shield. +shield if you do not specify a PIDSPEC with -p. By specifying +also a --verbose in addition, then you will get a listing of +every task that is running in either the shield or out of the +shield. Using no subcommand, ie. only "cset shield", will output the status of both shield and non-shield. Tasks will be listed if @@ -179,25 +174,6 @@ USR_SET = '/user' SYS_SET = '/system' verbose = 0 -# callback for --shield/--unshield, allows optional argument -# if no arg, then status of either shield or unshield is displayed -def shield_cb(option, opt_str, value, parser): - if value == None: - try: - arg = parser.rargs[0] - if arg[0] != '-': - # assign and consume argument - value = arg - del parser.rargs[0] - else: - value = True - except IndexError, err: - # no further arguments - value = True - except: - raise - setattr(parser.values, option.dest, value) - options = [make_option('-c', '--cpu', metavar = 'CPUSPEC', help = 'modifies or initializes the shield cpusets'), @@ -208,22 +184,24 @@ options = [make_option('-c', '--cpu', help = 'executes args in the shield', dest = 'exc', action = 'store_true'), - make_option('--user', + make_option('--user', help = 'use this USER for --exec (id or name)'), - make_option('--group', + make_option('--group', help = 'use this GROUP for --exec (id or name)'), make_option('-s', '--shield', - action = 'callback', - callback=shield_cb, - dest = 'shield', - metavar = 'PIDSPEC', - help = 'shield specified PIDSPEC of processes or threads'), + help = 'shield specified PIDSPEC of processes or threads', + action = 'store_true'), make_option('-u', '--unshield', - action = 'callback', - callback=shield_cb, - dest = 'unshield', + help = 'remove specified PIDSPEC of processes or threads from shield', + action = 'store_true'), + make_option('-p', '--pid', metavar = 'PIDSPEC', - help = 'remove specified PIDSPEC of processes or threads from shield'), + help = 'specify pid or tid specification for shield/unshield'), + make_option("--threads", + help = 'if specified, any processes found in the PIDSPEC to have ' + 'multiple threads will automatically have all their threads ' + 'added to the PIDSPEC; use to affect all related threads', + action = 'store_true'), make_option('-k', '--kthread', metavar = 'on|off', choices = ['on', 'off'], @@ -256,61 +234,76 @@ def func(parser, options, args): if (not options.cpu and not options.reset and not options.exc and not options.shield and not options.unshield and not options.kthread): shield_exists() + doshield = False if len(args) == 0: log.info("--> shielding system active with") print_all_stats() else: - exec_args(args, options.user, options.group) - return + # shortcut: first assume that arg is a pidspec, if not, then exec it + try: + plist = proc.pidspec_to_list(args[0]) + for pid in plist: int(pid) + doshield = True + # ok, if we're here, then it's probably a pidspec, shield it + except: + exec_args(args, options.user, options.group) + if doshield: + # drop through to shield section below + options.pid = args[0] + options.shield = True + else: + return if options.reset: reset_shield() return - # note that the following options fall through to others - # to allow for multiple options/commands on one cmdline - if options.cpu: make_shield(options.cpu, options.kthread) + if options.cpu: + make_shield(options.cpu, options.kthread) + return - elif options.kthread: make_kthread(options.kthread) + if options.kthread: + make_kthread(options.kthread) + return - if options.exc: exec_args(args, options.user, options.group) + if options.exc: + exec_args(args, options.user, options.group) + # exec_args does not return... - if options.shield: + if options.shield or options.unshield: shield_exists() - try: - log.info('--> shielding following pidspec: %s', options.shield) - if options.force: - proc.move_pidspec(options.shield, USR_SET) - else: - proc.move_pidspec(options.shield, USR_SET, SYS_SET) - log.info('done') - except Exception, err: - if str(err).find('Permission denied') != -1: - raise - if options.shield != True: - log.info(err) - log.info('--> bad pidspec: %s, shield state is:', options.shield) + if options.shield: + smsg = 'shielding' + to_set = USR_SET + from_set = SYS_SET + print_stats = print_usr_stats + else: + smsg = 'unshielding' + to_set = SYS_SET + from_set = USR_SET + print_stats = print_sys_stats + if options.pid == None: + if len(args) > 0: + # shortcut, assumes arg[0] is a pidspec + options.pid = args[0] else: - log.info('--> shielded state is:') - print_usr_stats() - - if options.unshield: - shield_exists() - try: - log.info('--> unshielding following pidspec: %s', options.unshield) + # no pidspec so output shield state + print_stats() + if options.pid: + if options.threads: tmsg = '(with threads)' + else: tmsg = '' + log.info('--> %s following pidspec: %s %s', smsg, options.pid, tmsg) if options.force: - proc.move_pidspec(options.unshield, SYS_SET) + proc.move_pidspec(options.pid, to_set, None, options.threads) else: - proc.move_pidspec(options.unshield, SYS_SET, USR_SET) - log.info('done') - except Exception, err: - if str(err).find('Permission denied') != -1: - raise - if options.unshield != True: - log.info('--> bad pidspec: %s, unshielded state is:', options.unshield) - else: - log.info('--> not shielded state is:') - print_sys_stats() + try: + proc.move_pidspec(options.pid, to_set, from_set, options.threads) + except CpusetException, err: + if str(err).find('do not match all criteria') != -1: + log.info("--> hint: perhaps use --force if sure of command") + raise + log.info('done') + return def print_all_stats(): print_sys_stats() @@ -323,7 +316,12 @@ def print_sys_stats(): else: proc.log_detailed_task_table(cset.unique_set(SYS_SET), ' ') else: - log.info(cset.summary(cset.unique_set(SYS_SET))) + if config.mread: + str = SYS_SET + if str[0] == '/': str = str[1:] + log.info('proc_list_no_tasks-' + str) + else: + log.info(cset.summary(cset.unique_set(SYS_SET))) def print_usr_stats(): if verbose and len(cset.unique_set(USR_SET).tasks) > 0: @@ -332,7 +330,12 @@ def print_usr_stats(): else: proc.log_detailed_task_table(cset.unique_set(USR_SET), ' ') else: - log.info(cset.summary(cset.unique_set(USR_SET))) + if config.mread: + str = USR_SET + if str[0] == '/': str = str[1:] + log.info('proc_list_no_tasks-' + str) + else: + log.info(cset.summary(cset.unique_set(USR_SET))) def shield_exists(): try: @@ -340,8 +343,8 @@ def shield_exists(): cset.unique_set(SYS_SET) return True except CpusetNotFound: - log.info('can\'t find "%s" and "%s" cpusets on system...', SYS_SET, USR_SET) - raise CpusetException('--> shielding not active on system') + log.debug('can\'t find "%s" and "%s" cpusets on system...', SYS_SET, USR_SET) + raise CpusetException('shielding not active on system') def reset_shield(): log.info("--> deactivating/reseting shielding") @@ -349,11 +352,11 @@ def reset_shield(): tasks = cset.unique_set(USR_SET).tasks log.info('moving %s tasks from "%s" user set to root set...', len(tasks), USR_SET) - proc.move(USR_SET, 'root') + proc.move(USR_SET, 'root', None, verbose) tasks = cset.unique_set(SYS_SET).tasks log.info('moving %s tasks from "%s" system set to root set...', len(tasks), SYS_SET) - proc.move(SYS_SET, 'root') + proc.move(SYS_SET, 'root', None, verbose) log.info('deleting "%s" and "%s" sets', USR_SET, SYS_SET) set.destroy(USR_SET) set.destroy(SYS_SET) @@ -407,7 +410,7 @@ def make_shield(cpuspec, kthread): pass if len(tasks) != 0: log.info("moving %s tasks from root into system cpuset...", len(tasks)) - proc.move('root', SYS_SET, tasks) + proc.move('root', SYS_SET, tasks, verbose) # move kernel theads into system set if asked for if kthread == 'on': root_tasks = cset.unique_set('/').tasks @@ -420,7 +423,7 @@ def make_shield(cpuspec, kthread): if len(tasks) != 0: log.info("kthread shield activated, moving %s tasks into system cpuset...", len(tasks)) - proc.move('root', SYS_SET, tasks) + proc.move('root', SYS_SET, tasks, verbose) # print out stats print_all_stats() @@ -442,11 +445,7 @@ def make_kthread(state): log.debug("total root tasks %s", len(root_tasks)) log.info("kthread shield activated, moving %s tasks into system cpuset...", len(tasks)) - proc.move('root', SYS_SET, tasks) - if verbose == 1: - proc.log_detailed_task_table(tasks, ' ', 76) - elif verbose > 1: - proc.log_detailed_task_table(tasks, ' ') + proc.move('root', SYS_SET, tasks, verbose) else: log.info('--> deactivating kthread shielding') usr_tasks = cset.unique_set(SYS_SET).tasks @@ -458,11 +457,7 @@ def make_kthread(state): tasks.append(task) if len(tasks) != 0: log.info("moving %s tasks into root cpuset...", len(tasks)) - proc.move(SYS_SET, '/', tasks) - if verbose == 1: - proc.log_detailed_task_table(tasks, ' ', 76) - elif verbose > 1: - proc.log_detailed_task_table(tasks, ' ') + proc.move(SYS_SET, '/', tasks, verbose) log.info('done') def exec_args(args, upar, gpar): diff --git a/cpuset/config.py b/cpuset/config.py new file mode 100644 index 0000000..7c67d20 --- /dev/null +++ b/cpuset/config.py @@ -0,0 +1,76 @@ +""" Cpuset Configuration Module + +The config module maintains global (class) variables of the various +configuration parameters for the cpuset application. These are filled in from +applicable configuration file passed as a path to the ReadConfigFile() method, +if desired. The class variables are given default values in the module source. +Anything found in the configuration files in the list of paths will override +these defaults. +""" + +__copyright__ = """ +Copyright (C) 2009 Novell Inc. +Author: Alex Tsariounov + +This program is free software; you can redistribute it and/or modify +it under the terms of the GNU General Public License version 2 as +published by the Free Software Foundation. + +This program is distributed in the hope that it will be useful, +but WITHOUT ANY WARRANTY; without even the implied warranty of +MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU +General Public License for more details. + +You should have received a copy of the GNU General Public License +along with this program; if not, write to the Free Software +Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA +""" + +import sys +import types +import ConfigParser + +############################################################################ +# Default configuration variable values +############################################################################ +defloc = '/etc/cset.conf' # default config file location +mread = False # machine readable output, usually set + # via option -m/--machine +mountpoint = '/cpusets' # cpuset filessytem mount point +############################################################################ + +def ReadConfigFiles(path=None): + if path == None: path = defloc + cf = ConfigParser.ConfigParser() + try: + fr = cf.read(path) + if len(fr) == 0: return + # can't use logging, too early... + if len(cf.sections()) != 1: + print "cset: warning, more than one section found in config file:", cf.sections() + if 'default' not in cf.sections(): + print 'cset: [default] section not found in config file "%s"' % path + sys.exit(3) + + except ConfigParser.MissingSectionHeaderError: + f = open(path) + cstr = f.read() + f.close() + import StringIO + cf.readfp(StringIO.StringIO('[default]\n' + cstr)) + + # override our globals... + for opt in cf.options('default'): + typ = type(globals()[opt]) + if typ == types.BooleanType: + globals()[opt] = cf.getboolean('default', opt) + elif typ == types.IntType: + globals()[opt] = cf.getint('default', opt) + else: + globals()[opt] = cf.get('default', opt) + +# Importing module autoinitializes it +def __init(): + ReadConfigFiles() + +__init() diff --git a/cpuset/cset.py b/cpuset/cset.py index 8dcd0d2..69d13e4 100644 --- a/cpuset/cset.py +++ b/cpuset/cset.py @@ -2,7 +2,7 @@ """ __copyright__ = """ -Copyright (C) 2008 Novell Inc. +Copyright (C) 2008, 2009 Novell Inc. Author: Alex Tsariounov This program is free software; you can redistribute it and/or modify @@ -26,8 +26,8 @@ if __name__ == '__main__': logging.basicConfig() from cpuset.util import * -global log log = logging.getLogger('cset') +RootSet = None class CpuSet(object): # sets is a class variable dict that keeps track of all @@ -135,14 +135,14 @@ class CpuSet(object): if not path: # mounted cpusets not found, so mount them - # FIXME: provide default mount directory from config file - if not os.access("/cpusets", os.F_OK): - os.mkdir("/cpusets") - ret = os.system("mount -t cpuset none /cpusets") + + if not os.access(config.mountpoint, os.F_OK): + os.mkdir(config.mountpoint) + ret = os.system("mount -t cpuset none " + config.mountpoint) if ret: raise CpusetException( 'mount of cpuset filesystem failed, do you have permission?') - path = "/cpusets" + path = config.mountpoint log.debug("cpusets mounted at: " + path) return path @@ -231,9 +231,10 @@ class CpuSet(object): f.write(task) f.close() except Exception, err: - if str(err).find('Permission denied') != -1: + if str(err).find('No such process') != -1: + log.info('**> task %s not found, not moved', task) + else: raise - # if here, means process is already gone, racy stuff... if prog: tick += 1 pb(tick) @@ -245,9 +246,41 @@ class CpuSet(object): # Helper functions # +def lookup_task_from_proc(pid): + """lookup the cpuset of the specified pid from proc filesystem""" + log.debug("entering lookup_task_from_proc, pid = %s", str(pid)) + path = "/proc/"+str(pid)+"/cpuset" + if os.access(path, os.F_OK): + set = file(path).readline()[:-1] + log.debug('lookup_task_from_proc: found task %s cpuset: %s', str(pid), set) + return set + # FIXME: add search for threads here... + raise CpusetException("task ID %s not found, i.e. not running" % str(pid)) + +def lookup_task_from_cpusets(pid): + """lookup the cpuset of the specified pid from cpuset filesystem""" + log.debug("entering lookup_task_from_cpusets, pid = %s", str(pid)) + global RootSet + if RootSet == None: rescan() + gotit = None + if pid in RootSet.tasks: + gotit = RootSet + else: + for node in walk_set(RootSet): + if pid in node.tasks: + gotit = node + break + if gotit: + log.debug('lookup_task_from_cpusets: found task %s cpuset: %s', str(pid), + gotit.path) + return gotit.path + raise CpusetException("task ID %s not found, i.e. not running" % str(pid)) + def unique_set(name): """find a unique cpuset by name or path, raise if multiple sets found""" log.debug("entering unique_set, name=%s", name) + if name == None: + raise CpusetException('unique_set() passed None as arg') if isinstance(name, CpuSet): return name nl = find_sets(name) if len(nl) > 1: @@ -256,7 +289,7 @@ def unique_set(name): return nl[0] def find_sets(name): - """find cpusets by name or path, return None if not found""" + """find cpusets by name or path, raise CpusetNotFound if not found""" log = logging.getLogger("cset.find_sets") log.debug('finding "%s" in cpusets', name) nodelist = [] @@ -267,13 +300,15 @@ def find_sets(name): log.debug("returning root set") nodelist.append(RootSet) else: - log.debug("walking from: %s", RootSet.name) + log.debug("walking from: %s", RootSet.path) for node in walk_set(RootSet): if node.name == name: log.debug('... found node "%s"', name) nodelist.append(node) else: log.debug("find by path") + # make sure that leading slash is used if searching by path + if name[0] != '/': name = '/' + name if name in CpuSet.sets: log.debug('... found node "%s"', CpuSet.sets[name].name) nodelist.append(CpuSet.sets[name]) @@ -294,30 +329,65 @@ def walk_set(set): yield result def rescan(): - """ re-read the cpuset directory to sync system with data structs """ + """re-read the cpuset directory to sync system with data structs""" + log.debug("entering rescan") global RootSet, maxcpu, allcpumask RootSet = CpuSet() # figure out system properties maxcpu = int(RootSet.cpus[-1]) allcpumask = calc_cpumask(maxcpu) -def cpuspec_check(cpuspec): - """ check format of cpuspec for validity """ - log.debug("cpuspec_check(%s), maxcpu=%s", cpuspec, maxcpu) - groups = cpuspec.split(',') - if int(groups[-1].split('-')[-1]) > int(maxcpu): - str = 'CPUSPEC "%s" specifies higher max(%s) than available(%s)' % \ - (cpuspec, groups[-1].split('-')[-1], maxcpu) - log.debug(str) - raise CpusetException(str) +def cpuspec_check(cpuspec, usemax=True): + """check format of cpuspec for validity""" + log.debug("cpuspec_check(%s)", cpuspec) mo = re.search("[^0-9,\-]", cpuspec) if mo: str = 'CPUSPEC "%s" contains invalid charaters: %s' % (cpuspec, mo.group()) log.debug(str) raise CpusetException(str) + groups = cpuspec.split(',') + if usemax and int(groups[-1].split('-')[-1]) > int(maxcpu): + str = 'CPUSPEC "%s" specifies higher max(%s) than available(%s)' % \ + (cpuspec, groups[-1].split('-')[-1], maxcpu) + log.debug(str) + raise CpusetException(str) + for sub in groups: + it = sub.split('-') + if len(it) == 2: + if len(it[0]) == 0 or len(it[1]) == 0: + # catches negative numbers + raise CpusetException('CPUSPEC "%s" has bad group "%s"' % (cpuspec, sub)) + if len(it) > 2: + raise CpusetException('CPUSPEC "%s" has bad group "%s"' % (cpuspec, sub)) + +def cpuspec_to_hex(cpuspec): + """convert a cpuspec to the hexadecimal string representation""" + log.debug('cpuspec_to_string(%s)', cpuspec) + cpuspec_check(cpuspec, usemax=False) + groups = cpuspec.split(',') + number = 0 + for sub in groups: + items = sub.split('-') + if len(items) == 1: + if not len(items[0]): + # two consecutive commas in cpuspec + continue + # one cpu in this group + log.debug(" adding cpu %s to result", items[0]) + number |= 1 << int(items[0]) + elif len(items) == 2: + il = [int(ii) for ii in items] + if il[1] >= il[0]: rng = range(il[0], il[1]+1) + else: rng = range(il[1], il[0]+1) + log.debug(' group=%s has cpu range of %s', sub, rng) + for num in rng: number |= 1 << num + else: + raise CpusetException('CPUSPEC "%s" has bad group "%s"' % (cpuspec, sub)) + log.debug(' final int number=%s in hex=%x', number, number) + return '%x' % number def memspec_check(memspec): - """check format of memspec for validity """ + """check format of memspec for validity""" # FIXME: look under /sys/devices/system/node for numa memory node # information and check the memspec that way, currently we only do # a basic check @@ -329,7 +399,7 @@ def memspec_check(memspec): raise CpusetException(str) def cpuspec_inverse(cpuspec): - """ calculate inverse of cpu specification """ + """calculate inverse of cpu specification""" cpus = [0 for x in range(maxcpu+1)] groups = cpuspec.split(',') log.debug("cpuspec_inverse(%s) maxcpu=%d groups=%d", @@ -382,8 +452,10 @@ def cpuspec_inverse(cpuspec): def summary(set): """return summary of cpuset with number of tasks running""" log.debug("entering summary, set=%s", set.path) - return ('"%s" cpuset of: %+10s cpu, with: %+5s tasks running' % - (set.name, set.cpus, len(set.tasks)) ) + if len(set.tasks) == 1: msg = 'task' + else: msg = 'tasks' + return ('"%s" cpuset of CPUSPEC(%s) with %s %s running' % + (set.name, set.cpus, len(set.tasks), msg) ) def calc_cpumask(max): all = 1 diff --git a/cpuset/main.py b/cpuset/main.py index 68d4f77..c90cec1 100644 --- a/cpuset/main.py +++ b/cpuset/main.py @@ -2,7 +2,7 @@ """ __copyright__ = """ -Copyright (C) 2008 Novell Inc. +Copyright (C) 2008, 2009 Novell Inc. Author: Alex Tsariounov This program is free software; you can redistribute it and/or modify @@ -21,6 +21,7 @@ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA import sys, os from optparse import OptionParser +from cpuset import config import cpuset.commands from cpuset.commands.common import CmdException from cpuset.util import CpusetException @@ -73,7 +74,12 @@ def _print_helpstring(cmd): print ' ' + cmd + ' ' * (12 - len(cmd)) + commands[cmd].help def print_help(): - print 'Usage: %s [options]' % os.path.basename(sys.argv[0]) + print 'Usage: %s [global options] [command options]' % os.path.basename(sys.argv[0]) + print + print 'Global options:' + print ' -l/--log output debugging log in fname' + print ' -m/--machine print machine readable output' + print ' -x/--tohex convert a CPUSPEC to hex' print print 'Generic commands:' print ' help print the detailed command usage' @@ -94,6 +100,10 @@ def print_help(): def main(): + # handle pipes better + import signal + signal.signal(signal.SIGPIPE, signal.SIG_DFL) + global prog prog = os.path.basename(sys.argv[0]) @@ -117,67 +127,6 @@ def main(): log = logging.getLogger('') log.setLevel(logging.DEBUG) - cmd = sys.argv[1] - if cmd in ['-l', '--log']: - # FIXME: very fragile - logfile = sys.argv[2] - #trace = logging.FileHandler('/var/log/cset.log', 'w') - trace = logging.FileHandler(logfile, 'a') - trace.setLevel(logging.DEBUG) - formatter = logging.Formatter('%(asctime)s %(name)-6s %(levelname)-8s %(message)s', - '%y%m%d-%H:%M:%S') - trace.setFormatter(formatter) - logging.getLogger('').addHandler(trace) - log.debug("---------- STARTING ----------") - from cpuset.version import version - log.debug('Cpuset (cset) %s' % version) - del(sys.argv[2]) - del(sys.argv[1]) - if len(sys.argv) > 1: - cmd = sys.argv[1] - else: - log.debug("no arguments to process, exit") - print >> sys.stderr, 'usage: %s ' % prog - print >> sys.stderr, \ - ' Try "%s --help" for a list of supported commands' % prog - sys.exit(1) - if cmd in ['-h', '--help']: - if len(sys.argv) >= 3: - cmd = commands.canonical_cmd(sys.argv[2]) - sys.argv[2] = '--help' - else: - print_help() - sys.exit(0) - if cmd == 'help': - if len(sys.argv) == 3 and not sys.argv[2] in ['-h', '--help']: - cmd = commands.canonical_cmd(sys.argv[2]) - if not cmd in commands: - log.error('help: "%s" command unknown' % cmd) - sys.exit(1) - - sys.argv[0] += ' %s' % cmd - command = commands[cmd] - parser = OptionParser(usage = command.usage, - option_list = command.options) - from pydoc import pager - pager(parser.format_help()) - else: - print_help() - sys.exit(0) - if cmd in ['-v', '--version', 'version']: - from cpuset.version import version - log.info('Cpuset (cset) %s' % version) - sys.exit(0) - if cmd in ['copyright']: - log.info(__copyright__) - sys.exit(0) - - # re-build the command line arguments - cmd = commands.canonical_cmd(cmd) - sys.argv[0] += ' %s' % cmd - del(sys.argv[1]) - log.debug('cmdline: ' + ' '.join(sys.argv)) - try: debug_level = int(os.environ['CSET_DEBUG_LEVEL']) except KeyError: @@ -186,6 +135,88 @@ def main(): log.error('Invalid CSET_DEBUG_LEVEL environment variable') sys.exit(1) + while True: + if len(sys.argv) == 1: + log.error('no arguments, nothing to do!') + sys.exit(2) + cmd = sys.argv[1] + if cmd in ['-l', '--log']: + if len(sys.argv) < 3: + log.critical('not enough arguments') + sys.exit(1) + # FIXME: very fragile + logfile = sys.argv[2] + #trace = logging.FileHandler('/var/log/cset.log', 'w') + trace = logging.FileHandler(logfile, 'a') + trace.setLevel(logging.DEBUG) + formatter = logging.Formatter('%(asctime)s %(name)-6s %(levelname)-8s %(message)s', + '%y%m%d-%H:%M:%S') + trace.setFormatter(formatter) + logging.getLogger('').addHandler(trace) + log.debug("---------- STARTING ----------") + from cpuset.version import version + log.debug('Cpuset (cset) %s' % version) + del(sys.argv[2]) + del(sys.argv[1]) + continue + if cmd in ['-h', '--help']: + if len(sys.argv) >= 3: + cmd = commands.canonical_cmd(sys.argv[2]) + sys.argv[2] = '--help' + else: + print_help() + sys.exit(0) + if cmd == 'help': + if len(sys.argv) == 3 and not sys.argv[2] in ['-h', '--help']: + cmd = commands.canonical_cmd(sys.argv[2]) + if not cmd in commands: + log.error('help: "%s" command unknown' % cmd) + sys.exit(1) + + sys.argv[0] += ' %s' % cmd + command = commands[cmd] + parser = OptionParser(usage = command.usage, + option_list = command.options) + from pydoc import pager + pager(parser.format_help()) + else: + print_help() + sys.exit(0) + if cmd in ['-v', '--version', 'version']: + from cpuset.version import version + log.info('Cpuset (cset) %s' % version) + sys.exit(0) + if cmd in ['-c', 'copyright', 'copying']: + log.info(__copyright__) + sys.exit(0) + if cmd in ['-m', '--machine']: + config.mread = True + del(sys.argv[1]) + continue + if cmd in ['-x', '--tohex']: + if len(sys.argv) < 3: + log.critical('not enough arguments') + sys.exit(1) + cpuspec = sys.argv[2] + import cset + try: + print cset.cpuspec_to_hex(cpuspec) + except (ValueError, OSError, IOError, CpusetException, CmdException), err: + log.critical('**> ' + str(err)) + if debug_level: + raise + else: + sys.exit(2) + sys.exit(0) + + break + + # re-build the command line arguments + cmd = commands.canonical_cmd(cmd) + sys.argv[0] += ' %s' % cmd + del(sys.argv[1]) + log.debug('cmdline: ' + ' '.join(sys.argv)) + try: # importing the cset class creates the model log.debug("creating cpuset model") @@ -193,11 +224,10 @@ def main(): command = commands[cmd] usage = command.usage.split('\n')[0].strip() parser = OptionParser(usage = usage, option_list = command.options) - parser.disable_interspersed_args() options, args = parser.parse_args() command.func(parser, options, args) except (ValueError, OSError, IOError, CpusetException, CmdException), err: - log.critical('%s: %s' % (cmd, err)) + log.critical('**> ' + str(err)) if str(err).find('Permission denied') != -1: log.critical('insufficient permissions, you probably need to be root') if str(err).find('invalid literal') != -1: diff --git a/cpuset/util.py b/cpuset/util.py index 8ff14f7..90d98bd 100644 --- a/cpuset/util.py +++ b/cpuset/util.py @@ -2,7 +2,7 @@ """ __copyright__ = """ -Copyright (C) 2008 Novell Inc. +Copyright (C) 2008, 2009 Novell Inc. Author: Alex Tsariounov This program is free software; you can redistribute it and/or modify @@ -20,6 +20,7 @@ Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA """ import sys, time +from cpuset import config class CpusetException(Exception): pass @@ -45,7 +46,8 @@ class TwirlyBar: self.__bar = ('|', '/', '-', '\\') def tick(self): - print '\b' + self.__bar[self.__state] + '\b', + if not config.mread: + print '\b' + self.__bar[self.__state] + '\b', self.__state = self.__state + 1 if self.__state > 3: self.__state = 0 @@ -65,6 +67,8 @@ class ProgressBar: self.block=chr(178) else: self.block=progresschar + if config.mread: + return self.f=sys.stdout if not self.finalcount: return self.f.write('[') @@ -82,11 +86,12 @@ class ProgressBar: percentcomplete=100 blockcount=int(percentcomplete/2) - if blockcount > self.blockcount: - for i in range(self.blockcount,blockcount): - self.f.write(self.block) - self.f.flush() + if not config.mread: + if blockcount > self.blockcount: + for i in range(self.blockcount,blockcount): + self.f.write(self.block) + self.f.flush() - if percentcomplete == 100: self.f.write("]\n") + if percentcomplete == 100: self.f.write("]\n") self.blockcount=blockcount diff --git a/cpuset/version.py b/cpuset/version.py index 334b0da..046d58e 100644 --- a/cpuset/version.py +++ b/cpuset/version.py @@ -1,5 +1,5 @@ __copyright__ = """ -Copyright (C) 2008 Novell Inc. +Copyright (C) 2008, 2009 Novell Inc. Author: Alex Tsariounov This program is free software; you can redistribute it and/or modify @@ -16,4 +16,4 @@ along with this program; if not, write to the Free Software Foundation, Inc., 59 Temple Place, Suite 330, Boston, MA 02111-1307 USA """ -version = '1.04' +version = '1.5.0' diff --git a/doc/Makefile b/doc/Makefile index 0b62d28..29d283c 100644 --- a/doc/Makefile +++ b/doc/Makefile @@ -48,8 +48,8 @@ install: man -include doc.dep clean: - #rm -f *.xml *.html *.pdf *.1 *.7 doc.dep rm -f *.xml *.pdf doc.dep + #rm -f *.xml *.html *.pdf *.1 *.7 doc.dep %.html : %.txt $(ASCIIDOC) -b xhtml11 -d manpage -f asciidoc.conf $(ASCIIDOC_EXTRA) $< @@ -63,8 +63,8 @@ clean: %.pdf: %.xml xmlto pdf $< -# special formatting rules +#special formatting rules tutorial.html : %.html : %.txt - $(ASCIIDOC) -b xhtml11 -d article -f tutorial.conf $(ASCIIDOC_EXTRA) $< + $(ASCIIDOC) -b xhtml11 -d article -f tutorial.conf -a toc -a numbered -a toclevels=4 $(ASCIIDOC_EXTRA) $< tutorial.xml : %.xml : %.txt $(ASCIIDOC) -b docbook -d article -f tutorial.conf $< diff --git a/doc/asciidoc.conf b/doc/asciidoc.conf index bc73828..740574e 100644 --- a/doc/asciidoc.conf +++ b/doc/asciidoc.conf @@ -47,7 +47,7 @@ endif::backend-docbook[] ifdef::backend-xhtml11[] [csetlink-inlinemacro] -cset-{target}(1) +cset {target}(1) endif::backend-xhtml11[] # csetdesc diff --git a/doc/cset-proc.1 b/doc/cset-proc.1 index c3467ba..a7d725b 100644 --- a/doc/cset-proc.1 +++ b/doc/cset-proc.1 @@ -2,7 +2,7 @@ .\" It was generated using the DocBook XSL Stylesheets (version 1.69.1). .\" Instead of manually editing it, you probably should edit the DocBook XML .\" source for it and then use the DocBook XSL Stylesheets to regenerate it. -.TH "CSET\-PROC" "1" "05/30/2008" "" "" +.TH "CSET\-PROC" "1" "02/07/2009" "" "" .\" disable hyphenation .nh .\" disable justification (adjust text to left margin only) @@ -47,6 +47,9 @@ move specified tasks to specified cpuset; to move a PIDSPEC to a cpuset, use \-m \-p PIDSPEC, \-\-pid=PIDSPEC specify pid or tid specification .TP +\-\-threads +if specified, any processes found in the PIDSPEC to have multiple threads will automatically have all their threads added to the PIDSPEC (use to move all related threads to a cpuset) +.TP \-s CPUSET, \-\-set=CPUSET specify name of immediate cpuset .TP @@ -59,7 +62,7 @@ specify name of origination cpuset \-k, \-\-kthread move, or include moving, unbound kernel threads .TP -\-a, \-\-all +\-\-force force all processes and threads to be moved .TP \-v, \-\-verbose @@ -71,9 +74,9 @@ To list which tasks are running in a particular cpuset, use the \-\-list command For example: -\fB# cset proc \-\-list \-\-set priset\fR +\fB# cset proc \-\-list \-\-set myset\fR -This command will list all the tasks running in the cpuset called "priset". +This command will list all the tasks running in the cpuset called "myset". Processes are created by specifying the path to the executable and specifying the cpuset that the process is to be created in. @@ -83,6 +86,14 @@ For example: This command will execute the /usr/bin/fast_code program on the "blazing_cpuset" cpuset. +Note that if your command takes options, then use the traditional "\-\-" marker to separate cset\(cqs options from your command\(cqs options. + +For example: + +\fB# cset proc \-\-set myset \-\-exec \(em ls \-l\fR + +This command will execute "ls \-l" on the cpuset called "myset". + The PIDSPEC argument taken for the move command is a comma separated list of PIDs or TIDs. The list can also include brackets of PIDs or TIDs (i.e. tasks) that are inclusive of the endpoints. For example: @@ -109,7 +120,7 @@ The move command accepts multiple common calling methods. For example, the follo These commands move the tasks defined as 2442 and any running task between 3000 and 3200 inclusive of the ends to the cpuset called "reserved_set". -Specifying the \-\-fromset is not necesary since the tasks will be moved to the destination cpuset no matter which cpuset they are currently running on. +Specifying the \-\-fromset is not necessary since the tasks will be moved to the destination cpuset no matter which cpuset they are currently running on. .sp .it 1 an-trap .nr an-no-space-flag 1 @@ -118,6 +129,8 @@ Specifying the \-\-fromset is not necesary since the tasks will be moved to the \fBNote\fR However, if you do specify a cpuset with the \-\-fromset option, then only those tasks that are both in the PIDSPEC \fBand\fR are running in the cpuset specified by \-\-fromset will be moved. I.e., if there is a task running on the system but not in \-\-fromset that is in PIDSPEC, it will not be moved. +If the \-\-threads switch is used, then the proc command will gather any threads of belonging to any processes or threads that are specified in the PIDSPEC and move them. This provides an easy way to move all related threads: just pick one TID from the set and use the \-\-threads option. + To move all userspace tasks from one cpuset to another, you need to specify the source and destination cpuset by name. For example: @@ -131,14 +144,14 @@ This command specifies that all processes and threads running on cpuset "comp1" .nr an-break-flag 1 .br \fBNote\fR -This move command will not move kernel threads unless the \-k/\-\-kthread switch is specified. If it is, then all unbound kernel threads will be added to the move. Unbound kernel threads are those that can run on any CPU. If you \fBalso specify\fR the \-a/\-\-all switch, then all tasks, kernel or not, bound or not, will be moved. +This move command will not move kernel threads unless the \-k/\-\-kthread switch is specified. If it is, then all unbound kernel threads will be added to the move. Unbound kernel threads are those that can run on any CPU. If you \fBalso specify\fR the \-\-force switch, then all tasks, kernel or not, bound or not, will be moved. .sp .it 1 an-trap .nr an-no-space-flag 1 .nr an-break-flag 1 .br \fBCaution\fR -Please be cautious with the \-\-all switch, since moving a kernel thread that is bound to a specific CPU to a cpuset that does not include that CPU can cause a system hang. +Please be cautious with the \-\-force switch, since moving a kernel thread that is bound to a specific CPU to a cpuset that does not include that CPU can cause a system hang. You must specify unique cpuset names for the both exec and move commands. If a simple name passed to the \-\-fromset, \-\-toset and \-\-set parameters is unique on the system then that command executes. However, if there are multiple cpusets by that name, then you will need to specify which one you mean with a full path rooted at the base cpuset tree. @@ -157,17 +170,19 @@ Then, to move a process from myset in group1 to yourset in group2, you would hav # cset proc \-\-move \-\-pid=50 \-\-fromset=/group1/myset \-\-toset=/group2/yourset -You do not have to worry about where in the Linux filesystem the cpuset filesystem is mounted. The cset command takes care of that. Any cpusets that are specifed by path (such as above), are done with respect to the root of the cpuset filesystem. +You do not have to worry about where in the Linux filesystem the cpuset filesystem is mounted. The cset command takes care of that. Any cpusets that are specified by path (such as above), are done with respect to the root of the cpuset filesystem. .SH "LICENSE" Cpuset is licensed under the GNU GPL V2 only. .SH "COPYRIGHT" -Copyright (c) 2008 Novell Inc. +Copyright (c) 2008\-2009 Novell Inc. .SH "AUTHOR" Written by Alex Tsariounov . - -Some substrate code and ideas were taken from the excellent Stacked GIT (stgit) v0.13 (see http://gna.org/projects/stgit and http://www.procode.org/stgit). Stacked GIT is under GPL V2 or later. .SH "SEE ALSO" -\fBcset\fR(1), \fBcset\-set\fR(1), \fBcset\-shield\fR(1), \fBtaskset\fR(1), \fBchrt\fR(1) +cset(1), cset\-set(1), cset\-shield(1) + +/usr/share/doc/packages/cpuset/html/tutorial.html + +taskset(1), chrt(1) /usr/src/linux/Documentation/cpusets.txt diff --git a/doc/cset-proc.html b/doc/cset-proc.html index 6864441..b05faa6 100644 --- a/doc/cset-proc.html +++ b/doc/cset-proc.html @@ -3,7 +3,8 @@ - + +cset-proc(1) -cset-proc(1) -

OPTIONS

+

OPTIONS

-
-
+
+
-h, --help
@@ -327,7 +435,7 @@ cset-proc(1) Manual Page prints the list of options for this command

-
+
-l, --list
@@ -335,7 +443,7 @@ cset-proc(1) Manual Page list processes in the specified cpuset

-
+
-e, --exec
@@ -343,7 +451,7 @@ cset-proc(1) Manual Page execute arguments in the specified cpuset

-
+
-u USER, --user=USER
@@ -351,7 +459,7 @@ cset-proc(1) Manual Page use this USER to --exec (id or name)

-
+
-g GROUP, --group=GROUP
@@ -359,7 +467,7 @@ cset-proc(1) Manual Page use this GROUP to --exec (id or name)

-
+
-m, --move
@@ -369,7 +477,7 @@ cset-proc(1) Manual Page and --toset

-
+
-p PIDSPEC, --pid=PIDSPEC
@@ -377,7 +485,17 @@ cset-proc(1) Manual Page specify pid or tid specification

-
+
+--threads +
+
+

+ if specified, any processes found in the PIDSPEC to have multiple + threads will automatically have all their threads added to the + PIDSPEC (use to move all related threads to a cpuset) +

+
+
-s CPUSET, --set=CPUSET
@@ -385,7 +503,7 @@ cset-proc(1) Manual Page specify name of immediate cpuset

-
+
-t TOSET, --toset=TOSET
@@ -393,7 +511,7 @@ cset-proc(1) Manual Page specify name of destination cpuset

-
+
-f FROMSET, --fromset=FROMSET
@@ -401,7 +519,7 @@ cset-proc(1) Manual Page specify name of origination cpuset

-
+
-k, --kthread
@@ -409,15 +527,15 @@ cset-proc(1) Manual Page move, or include moving, unbound kernel threads

-
--a, --all +
+--force

force all processes and threads to be moved

-
+
-v, --verbose
@@ -425,31 +543,36 @@ cset-proc(1) Manual Page prints more detailed output, additive

-
+
-

DESCRIPTION

+

DESCRIPTION

-

This command is used to run and manage arbitrary processes on +

This command is used to run and manage arbitrary processes on specified cpusets. It is also used to move pre-existing processes and threads to specified cpusets. You may note there is no "kill" or -"destroy" option — use the standard OS ^C or kill commands for that.

-

To list which tasks are running in a particular cpuset, use the --list -command.

-

For example:

-

# cset proc --list --set priset

-

This command will list all the tasks running in the cpuset called -"priset".

-

Processes are created by specifying the path to the executable -and specifying the cpuset that the process is to be created in.

-

For example:

-

# cset proc --set=blazing_cpuset --exec /usr/bin/fast_code

-

This command will execute the /usr/bin/fast_code program on the -"blazing_cpuset" cpuset.

-

The PIDSPEC argument taken for the move command is a comma +"destroy" option — use the standard OS ^C or kill commands for that.

+

To list which tasks are running in a particular cpuset, use the --list +command.

+

For example:

+

# cset proc --list --set myset

+

This command will list all the tasks running in the cpuset called +"myset".

+

Processes are created by specifying the path to the executable +and specifying the cpuset that the process is to be created in.

+

For example:

+

# cset proc --set=blazing_cpuset --exec /usr/bin/fast_code

+

This command will execute the /usr/bin/fast_code program on the +"blazing_cpuset" cpuset.

+

Note that if your command takes options, then use the traditional +"--" marker to separate cset’s options from your command’s options.

+

For example:

+

# cset proc --set myset --exec — ls -l

+

This command will execute "ls -l" on the cpuset called "myset".

+

The PIDSPEC argument taken for the move command is a comma separated list of PIDs or TIDs. The list can also include brackets of PIDs or TIDs (i.e. tasks) that are inclusive of the -endpoints.

-

For example:

+endpoints.

+

For example:

1,2,5         Means processes 1, 2 and 5
@@ -466,20 +589,20 @@ there is only one process, say PID 57, in the range of 50-65,
 then only that process will be moved.
-

To move a PIDSPEC to a specific cpuset, you can either specify +

To move a PIDSPEC to a specific cpuset, you can either specify the PIDSPEC with --pid and the destination cpuset with --toset, or use the short hand and list the cpuset name after the PIDSPEC -for the --move arguments.

-

The move command accepts multiple common calling methods. -For example, the following commands are equivalent:

-

# cset proc --move 2442,3000-3200 reserved_set

-

# cset proc --move --pid=2442,3000-3200 --toset=reserved_set

-

These commands move the tasks defined as 2442 and any running task +for the --move arguments.

+

The move command accepts multiple common calling methods. +For example, the following commands are equivalent:

+

# cset proc --move 2442,3000-3200 reserved_set

+

# cset proc --move --pid=2442,3000-3200 --toset=reserved_set

+

These commands move the tasks defined as 2442 and any running task between 3000 and 3200 inclusive of the ends to the cpuset called -"reserved_set".

-

Specifying the --fromset is not necesary since the tasks will be +"reserved_set".

+

Specifying the --fromset is not necessary since the tasks will be moved to the destination cpuset no matter which cpuset they are -currently running on.

+currently running on.

@@ -492,12 +615,17 @@ I.e., if there is a task running on the system but not in --fromset that is in PIDSPEC, it will not be moved.
-

To move all userspace tasks from one cpuset to another, you need -to specify the source and destination cpuset by name.

-

For example:

-

# cset proc --move --fromset=comp1 --toset=comp42

-

This command specifies that all processes and threads -running on cpuset "comp1" be moved to cpuset "comp42".

+

If the --threads switch is used, then the proc command will +gather any threads of belonging to any processes or threads that +are specified in the PIDSPEC and move them. This provides an easy +way to move all related threads: just pick one TID from the set +and use the --threads option.

+

To move all userspace tasks from one cpuset to another, you need +to specify the source and destination cpuset by name.

+

For example:

+

# cset proc --move --fromset=comp1 --toset=comp42

+

This command specifies that all processes and threads +running on cpuset "comp1" be moved to cpuset "comp42".

@@ -507,7 +635,7 @@ running on cpuset "comp1" be moved to cpuset "comp42".

the -k/--kthread switch is specified. If it is, then all unbound kernel threads will be added to the move. Unbound kernel threads are those that can run on any CPU. If you also specify the --a/--all switch, then all tasks, kernel or not, bound or not, +--force switch, then all tasks, kernel or not, bound or not, will be moved.
@@ -516,18 +644,18 @@ will be moved.
Caution
-Please be cautious with the --all switch, since moving a -kernel thread that is bound to a specific CPU to a cpuset that +Please be cautious with the --force switch, since moving +a kernel thread that is bound to a specific CPU to a cpuset that does not include that CPU can cause a system hang.
-

You must specify unique cpuset names for the both exec and move +

You must specify unique cpuset names for the both exec and move commands. If a simple name passed to the --fromset, --toset and --set parameters is unique on the system then that command executes. However, if there are multiple cpusets by that name, then you will need to specify which one you mean with a full path -rooted at the base cpuset tree.

-

For example, suppose you have the following cpuset tree:

+rooted at the base cpuset tree.

+

For example, suppose you have the following cpuset tree:

/cpusets
@@ -538,39 +666,37 @@ rooted at the base cpuset tree.

     /myset
     /yourset
-

Then, to move a process from myset in group1 to yourset in -group2, you would have to issue the following command:

-

# cset proc --move --pid=50 --fromset=/group1/myset --toset=/group2/yourset

-

You do not have to worry about where in the Linux filesystem the +

Then, to move a process from myset in group1 to yourset in +group2, you would have to issue the following command:

+

# cset proc --move --pid=50 --fromset=/group1/myset --toset=/group2/yourset

+

You do not have to worry about where in the Linux filesystem the cpuset filesystem is mounted. The cset command takes care of that. -Any cpusets that are specifed by path (such as above), are done with -respect to the root of the cpuset filesystem.

+Any cpusets that are specified by path (such as above), are done with +respect to the root of the cpuset filesystem.

-

LICENSE

+

LICENSE

-

Cpuset is licensed under the GNU GPL V2 only.

+

Cpuset is licensed under the GNU GPL V2 only.

-

COPYRIGHT

+

COPYRIGHT

-

Copyright (c) 2008 Novell Inc.

+

Copyright (c) 2008-2009 Novell Inc.

-

AUTHOR

+

AUTHOR

-

Written by Alex Tsariounov <alext@novell.com>.

-

Some substrate code and ideas were taken from the excellent Stacked -GIT (stgit) v0.13 (see http://gna.org/projects/stgit and -http://www.procode.org/stgit). Stacked GIT is under GPL V2 or later.

+

Written by Alex Tsariounov <alext@novell.com>.

-

SEE ALSO

+

SEE ALSO

-

cset(1), cset-set(1), cset-shield(1), taskset(1), -chrt(1)

-

/usr/src/linux/Documentation/cpusets.txt

+

cset(1), cset-set(1), cset-shield(1)

+

/usr/share/doc/packages/cpuset/html/tutorial.html

+

taskset(1), chrt(1)

+

/usr/src/linux/Documentation/cpusets.txt

diff --git a/doc/cset-proc.txt b/doc/cset-proc.txt index e933bab..e63f237 100644 --- a/doc/cset-proc.txt +++ b/doc/cset-proc.txt @@ -1,7 +1,7 @@ cset-proc(1) ============ Alex Tsariounov -v1.0, May 2008 +v1.5.0, February 2009 NAME ---- @@ -47,6 +47,11 @@ OPTIONS -p PIDSPEC, --pid=PIDSPEC:: specify pid or tid specification +--threads:: + if specified, any processes found in the PIDSPEC to have multiple + threads will automatically have all their threads added to the + PIDSPEC (use to move all related threads to a cpuset) + -s CPUSET, --set=CPUSET:: specify name of immediate cpuset @@ -59,7 +64,7 @@ OPTIONS -k, --kthread:: move, or include moving, unbound kernel threads --a, --all:: +--force:: force all processes and threads to be moved -v, --verbose:: @@ -77,10 +82,10 @@ command. For example: -*+# cset proc --list --set priset+* +*+# cset proc --list --set myset+* This command will list all the tasks running in the cpuset called -"priset". +"myset". Processes are created by specifying the path to the executable and specifying the cpuset that the process is to be created in. @@ -92,6 +97,15 @@ For example: This command will execute the /usr/bin/fast_code program on the "blazing_cpuset" cpuset. +Note that if your command takes options, then use the traditional +"--" marker to separate cset's options from your command's options. + +For example: + +*+# cset proc --set myset --exec -- ls -l+* + +This command will execute "ls -l" on the cpuset called "myset". + The PIDSPEC argument taken for the move command is a comma separated list of PIDs or TIDs. The list can also include brackets of PIDs or TIDs (i.e. tasks) that are inclusive of the @@ -123,7 +137,7 @@ These commands move the tasks defined as 2442 and any running task between 3000 and 3200 inclusive of the ends to the cpuset called "reserved_set". -Specifying the --fromset is not necesary since the tasks will be +Specifying the --fromset is not necessary since the tasks will be moved to the destination cpuset no matter which cpuset they are currently running on. @@ -133,6 +147,12 @@ are running in the cpuset specified by --fromset will be moved. I.e., if there is a task running on the system but not in --fromset that is in PIDSPEC, it will not be moved. +If the --threads switch is used, then the proc command will +gather any threads of belonging to any processes or threads that +are specified in the PIDSPEC and move them. This provides an easy +way to move all related threads: just pick one TID from the set +and use the --threads option. + To move all userspace tasks from one cpuset to another, you need to specify the source and destination cpuset by name. @@ -147,11 +167,11 @@ NOTE: This move command will not move kernel threads unless the -k/--kthread switch is specified. If it is, then all unbound kernel threads will be added to the move. Unbound kernel threads are those that can run on any CPU. If you *also specify* the --a/--all switch, then all tasks, kernel or not, bound or not, +--force switch, then all tasks, kernel or not, bound or not, will be moved. -CAUTION: Please be cautious with the --all switch, since moving a -kernel thread that is bound to a specific CPU to a cpuset that +CAUTION: Please be cautious with the --force switch, since moving +a kernel thread that is bound to a specific CPU to a cpuset that does not include that CPU can cause a system hang. You must specify unique cpuset names for the both exec and move @@ -178,7 +198,7 @@ group2, you would have to issue the following command: You do not have to worry about where in the Linux filesystem the cpuset filesystem is mounted. The cset command takes care of that. -Any cpusets that are specifed by path (such as above), are done with +Any cpusets that are specified by path (such as above), are done with respect to the root of the cpuset filesystem. LICENSE @@ -187,19 +207,18 @@ Cpuset is licensed under the GNU GPL V2 only. COPYRIGHT --------- -Copyright (c) 2008 Novell Inc. +Copyright (c) 2008-2009 Novell Inc. AUTHOR ------ -Written by Alex Tsariounov . - -Some substrate code and ideas were taken from the excellent Stacked -GIT (stgit) v0.13 (see http://gna.org/projects/stgit and -http://www.procode.org/stgit). Stacked GIT is under GPL V2 or later. +Written by Alex Tsariounov <\alext@novell.com>. SEE ALSO -------- -manlink:cset[1], csetlink:set[], csetlink:shield[], manlink:taskset[1], -manlink:chrt[1] +cset(1), cset-set(1), cset-shield(1) + +/usr/share/doc/packages/cpuset/html/tutorial.html + +taskset(1), chrt(1) /usr/src/linux/Documentation/cpusets.txt diff --git a/doc/cset-set.1 b/doc/cset-set.1 index be98b65..7773bfa 100644 --- a/doc/cset-set.1 +++ b/doc/cset-set.1 @@ -2,7 +2,7 @@ .\" It was generated using the DocBook XSL Stylesheets (version 1.69.1). .\" Instead of manually editing it, you probably should edit the DocBook XML .\" source for it and then use the DocBook XSL Stylesheets to regenerate it. -.TH "CSET\-SET" "1" "05/30/2008" "" "" +.TH "CSET\-SET" "1" "02/07/2009" "" "" .\" disable hyphenation .nh .\" disable justification (adjust text to left margin only) @@ -15,11 +15,9 @@ cset\-set \- manage sets of cpus \fIcset\fR [cset options] \fIset\fR [set options] [args] \fIcset\fR set \-\-help \fIcset\fR set -\fIcset\fR set \-\-all \fIcset\fR set \-\-recurse \fIcset\fR set \-\-list myset \fIcset\fR set myset -\fIcset\fR set \-\-all myset \fIcset\fR set \-\-recurse \-\-list myset \fIcset\fR set \-\-cpu 2\-5 \-\-mem 0 \-\-set newset \fIcset\fR set \-\-cpu 2\-5 newset @@ -47,9 +45,6 @@ destroy specified cpuset \-s CPUSET, \-\-set=CPUSET specify cpuset name to be acted on .TP -\-a, \-\-all -also do listing of members of specified cpuset, for use with \-\-list -.TP \-r, \-\-recurse do recursive listing, for use with \-\-list .TP @@ -64,7 +59,7 @@ mark this cpuset as owning its MEMs exclusively .SH "DESCRIPTION" This command is used to create, modify, and destroy cpusets. Cpusets form a tree\-like structure rooted at the root cpuset which always includes all system CPUs and all system memory nodes. -A cpuset is an organizational unit that defines a group of CPUs and a group of memory nodes where a process or thread (i.e. task) is allowed to run on. For non\-NUMA machines, the memory node is always 0 (zero) and cannot be set to anything else. For NUMA machines, the memory node can be set to a similar specifcation as the CPU definition and will tie those memory nodes to that cpuset. You will usually want the memory nodes that belong to the CPUs defined to be in the same cpuset. +A cpuset is an organizational unit that defines a group of CPUs and a group of memory nodes where a process or thread (i.e. task) is allowed to run on. For non\-NUMA machines, the memory node is always 0 (zero) and cannot be set to anything else. For NUMA machines, the memory node can be set to a similar specification as the CPU definition and will tie those memory nodes to that cpuset. You will usually want the memory nodes that belong to the CPUs defined to be in the same cpuset. A cpuset can have exclusive right to the CPUs defined in it. This means that only this cpuset can own these CPUs. Similarly, a cpuset can have exclusive right to the memory nodes defined in it. This means that only this cpuset can own these memory nodes. @@ -91,19 +86,21 @@ This command creates a cpuset called "new_set" located off the root cpuset which Create a cpuset that specifies both CPUs and memory nodes \fB# cset set \-\-cpu=3 \-\-mem=3 /rad/set_one\fR -Note that this command uses the full path method to specify the name of the new cpuset "/rad/set_one". It also names the new cpuset implicitily (i.e. no \-\-set option, although you can use that if you want to). If the "set_one" name is unique, you can subsequently refer to is just by that. Memory node 3 is assigned to this cpuset as well as CPU 3. +Note that this command uses the full path method to specify the name of the new cpuset "/rad/set_one". It also names the new cpuset implicitly (i.e. no \-\-set option, although you can use that if you want to). If the "set_one" name is unique, you can subsequently refer to is just by that. Memory node 3 is assigned to this cpuset as well as CPU 3. The above commands will create the new cpusets, or if they already exist, they will modify them to the new specifications. .SH "LICENSE" Cpuset is licensed under the GNU GPL V2 only. .SH "COPYRIGHT" -Copyright (c) 2008 Novell Inc. +Copyright (c) 2008\-2009 Novell Inc. .SH "AUTHOR" Written by Alex Tsariounov . - -Some substrate code and ideas were taken from the excellent Stacked GIT (stgit) v0.13 (see http://gna.org/projects/stgit and http://www.procode.org/stgit). Stacked GIT is under GPL V2 or later. .SH "SEE ALSO" -\fBcset\fR(1), \fBcset\-proc\fR(1), \fBcset\-shield\fR(1), \fBtaskset\fR(1), \fBchrt\fR(1) +cset(1), cset\-proc(1), cset\-shield(1) + +/usr/share/doc/packages/cpuset/html/tutorial.html + +taskset(1), chrt(1) /usr/src/linux/Documentation/cpusets.txt diff --git a/doc/cset-set.html b/doc/cset-set.html index 6e5c53e..15ae171 100644 --- a/doc/cset-set.html +++ b/doc/cset-set.html @@ -3,7 +3,8 @@ - + +cset-set(1) -cset-set(1) -

OPTIONS

+

OPTIONS

-
-
+
+
-h, --help
@@ -330,7 +436,7 @@ cset-set(1) Manual Page prints the list of options for this command

-
+
-l, --list
@@ -339,7 +445,7 @@ cset-set(1) Manual Page cpuset; if -r is used, will list recursively

-
+
-c CPUSPEC, --cpu=CPUSPEC
@@ -348,7 +454,7 @@ cset-set(1) Manual Page specification

-
+
-m MEMSPEC, --mem=MEMSPEC
@@ -357,7 +463,7 @@ cset-set(1) Manual Page cpuset

-
+
-d, --destroy
@@ -365,7 +471,7 @@ cset-set(1) Manual Page destroy specified cpuset

-
+
-s CPUSET, --set=CPUSET
@@ -373,15 +479,7 @@ cset-set(1) Manual Page specify cpuset name to be acted on

-
--a, --all -
-
-

- also do listing of members of specified cpuset, for use with --list -

-
-
+
-r, --recurse
@@ -389,7 +487,7 @@ cset-set(1) Manual Page do recursive listing, for use with --list

-
+
-v, --verbose
@@ -398,7 +496,7 @@ cset-set(1) Manual Page will not chop listing to fit in 80 columns

-
+
--cpu_exclusive
@@ -406,7 +504,7 @@ cset-set(1) Manual Page mark this cpuset as owning its CPUs exclusively

-
+
--mem_exclusive
@@ -414,64 +512,64 @@ cset-set(1) Manual Page mark this cpuset as owning its MEMs exclusively

-
+
-

DESCRIPTION

+

DESCRIPTION

-

This command is used to create, modify, and destroy cpusets. +

This command is used to create, modify, and destroy cpusets. Cpusets form a tree-like structure rooted at the root cpuset which always includes all system CPUs and all system memory -nodes.

-

A cpuset is an organizational unit that defines a group of CPUs +nodes.

+

A cpuset is an organizational unit that defines a group of CPUs and a group of memory nodes where a process or thread (i.e. task) is allowed to run on. For non-NUMA machines, the memory node is always 0 (zero) and cannot be set to anything else. For NUMA -machines, the memory node can be set to a similar specifcation +machines, the memory node can be set to a similar specification as the CPU definition and will tie those memory nodes to that cpuset. You will usually want the memory nodes that belong to -the CPUs defined to be in the same cpuset.

-

A cpuset can have exclusive right to the CPUs defined in it. +the CPUs defined to be in the same cpuset.

+

A cpuset can have exclusive right to the CPUs defined in it. This means that only this cpuset can own these CPUs. Similarly, a cpuset can have exclusive right to the memory nodes defined in it. This means that only this cpuset can own these memory -nodes.

-

Cpusets can be specified by name or by path; however, care +nodes.

+

Cpusets can be specified by name or by path; however, care should be taken when specifying by name if the name is not unique. This tool will generally not let you do destructive -things to non-unique cpuset names.

-

Cpusets are uniquely specified by path. The path starts at where +things to non-unique cpuset names.

+

Cpusets are uniquely specified by path. The path starts at where the cpusets filesystem is mounted so you generally do not have to know where that is. For example, so specify a cpuset that is called "two" which is a subset of "one" which in turn is a subset of the root cpuset, use the path "/one/two" regardless of where -the cpusets filesystem is mounted.

-

When specifying CPUs, a so-called CPUSPEC is used. The CPUSPEC +the cpusets filesystem is mounted.

+

When specifying CPUs, a so-called CPUSPEC is used. The CPUSPEC will accept a comma-separated list of CPUs and inclusive range specifications. For example, --cpu=1,3,5-7 will assign CPU1, -CPU3, CPU5, CPU6, and CPU7 to the specified cpuset.

-

Note that cpusets follow certain rules. For example, children +CPU3, CPU5, CPU6, and CPU7 to the specified cpuset.

+

Note that cpusets follow certain rules. For example, children can only include CPUs that the parents already have. If you do not follow those rules, the kernel cpuset subsystem will not let you create that cpuset. For example, if you create a cpuset that contains CPU3, and then attempt to create a child of that cpuset with a CPU other than 3, you will get an error, and the cpuset will not be active. The error is somewhat cryptic in that it is -usually a "Permission denied" error.

-

Memory nodes are specified with a MEMSPEC in a similar way to +usually a "Permission denied" error.

+

Memory nodes are specified with a MEMSPEC in a similar way to the CPUSPEC. For example, --mem=1,3-6 will assign MEM1, MEM3, -MEM4, MEM5, and MEM6 to the specified cpuset.

-

Note that if you attempt to create or modify a cpuset with a +MEM4, MEM5, and MEM6 to the specified cpuset.

+

Note that if you attempt to create or modify a cpuset with a memory node specification that is not valid, you may get a cryptic error message, "No space left on device", and the -modification will not be allowed.

-

When you destroy a cpuset, then the tasks running in that set are +modification will not be allowed.

+

When you destroy a cpuset, then the tasks running in that set are moved to the parent of that cpuset. If this is not what you want, then manually move those tasks to the cpuset of your choice with the cset proc command (see cset proc --help for more -information).

-

EXAMPLES

-
-
+information).

+

EXAMPLES

+
+
Create a cpuset with the default memory specification
@@ -479,14 +577,14 @@ Create a cpuset with the default memory specification # cset set --cpu=2,4,6-8 --set=new_set

-
-

This command creates a cpuset called "new_set" located +

+

This command creates a cpuset called "new_set" located off the root cpuset which holds CPUS 2,4,6,7,8 and node 0 (interleaved) memory. Note that --set is optional, and you can just specify the name for the new cpuset after -all arguments.

-
-
+all arguments.

+
+
Create a cpuset that specifies both CPUs and memory nodes
@@ -494,42 +592,40 @@ Create a cpuset that specifies both CPUs and memory nodes # cset set --cpu=3 --mem=3 /rad/set_one

-
-

Note that this command uses the full path method to +

+

Note that this command uses the full path method to specify the name of the new cpuset "/rad/set_one". It -also names the new cpuset implicitily (i.e. no --set +also names the new cpuset implicitly (i.e. no --set option, although you can use that if you want to). If the "set_one" name is unique, you can subsequently refer to is just by that. Memory node 3 is assigned to this -cpuset as well as CPU 3.

-

The above commands will create the new cpusets, or if they -already exist, they will modify them to the new specifications.

+cpuset as well as CPU 3.

+

The above commands will create the new cpusets, or if they +already exist, they will modify them to the new specifications.

-

LICENSE

+

LICENSE

-

Cpuset is licensed under the GNU GPL V2 only.

+

Cpuset is licensed under the GNU GPL V2 only.

-

COPYRIGHT

+

COPYRIGHT

-

Copyright (c) 2008 Novell Inc.

+

Copyright (c) 2008-2009 Novell Inc.

-

AUTHOR

+

AUTHOR

-

Written by Alex Tsariounov <alext@novell.com>.

-

Some substrate code and ideas were taken from the excellent Stacked -GIT (stgit) v0.13 (see http://gna.org/projects/stgit and -http://www.procode.org/stgit). Stacked GIT is under GPL V2 or later.

+

Written by Alex Tsariounov <alext@novell.com>.

-

SEE ALSO

+

SEE ALSO

-

cset(1), cset-proc(1), cset-shield(1), taskset(1), -chrt(1)

-

/usr/src/linux/Documentation/cpusets.txt

+

cset(1), cset-proc(1), cset-shield(1)

+

/usr/share/doc/packages/cpuset/html/tutorial.html

+

taskset(1), chrt(1)

+

/usr/src/linux/Documentation/cpusets.txt

diff --git a/doc/cset-set.txt b/doc/cset-set.txt index 50c015f..3fe6101 100644 --- a/doc/cset-set.txt +++ b/doc/cset-set.txt @@ -1,7 +1,7 @@ cset-set(1) =========== Alex Tsariounov -v1.0, May 2008 +v1.5.0, February 2009 NAME ---- @@ -13,11 +13,9 @@ SYNOPSIS 'cset' [cset options] 'set' [set options] [args] 'cset' set --help 'cset' set -'cset' set --all 'cset' set --recurse 'cset' set --list myset 'cset' set myset -'cset' set --all myset 'cset' set --recurse --list myset 'cset' set --cpu 2-5 --mem 0 --set newset 'cset' set --cpu 2-5 newset @@ -48,9 +46,6 @@ OPTIONS -s CPUSET, --set=CPUSET:: specify cpuset name to be acted on --a, --all:: - also do listing of members of specified cpuset, for use with --list - -r, --recurse:: do recursive listing, for use with --list @@ -75,7 +70,7 @@ A cpuset is an organizational unit that defines a group of CPUs and a group of memory nodes where a process or thread (i.e. task) is allowed to run on. For non-NUMA machines, the memory node is always 0 (zero) and cannot be set to anything else. For NUMA -machines, the memory node can be set to a similar specifcation +machines, the memory node can be set to a similar specification as the CPU definition and will tie those memory nodes to that cpuset. You will usually want the memory nodes that belong to the CPUs defined to be in the same cpuset. @@ -146,7 +141,7 @@ Create a cpuset that specifies both CPUs and memory nodes:: Note that this command uses the full path method to specify the name of the new cpuset "/rad/set_one". It -also names the new cpuset implicitily (i.e. no --set +also names the new cpuset implicitly (i.e. no --set option, although you can use that if you want to). If the "set_one" name is unique, you can subsequently refer to is just by that. Memory node 3 is assigned to this @@ -161,19 +156,18 @@ Cpuset is licensed under the GNU GPL V2 only. COPYRIGHT --------- -Copyright (c) 2008 Novell Inc. +Copyright (c) 2008-2009 Novell Inc. AUTHOR ------ -Written by Alex Tsariounov . - -Some substrate code and ideas were taken from the excellent Stacked -GIT (stgit) v0.13 (see http://gna.org/projects/stgit and -http://www.procode.org/stgit). Stacked GIT is under GPL V2 or later. +Written by Alex Tsariounov <\alext@novell.com>. SEE ALSO -------- -manlink:cset[1], csetlink:proc[], csetlink:shield[], manlink:taskset[1], -manlink:chrt[1] +cset(1), cset-proc(1), cset-shield(1) + +/usr/share/doc/packages/cpuset/html/tutorial.html + +taskset(1), chrt(1) /usr/src/linux/Documentation/cpusets.txt diff --git a/doc/cset-shield.1 b/doc/cset-shield.1 index 80b145c..cfeec0c 100644 --- a/doc/cset-shield.1 +++ b/doc/cset-shield.1 @@ -2,7 +2,7 @@ .\" It was generated using the DocBook XSL Stylesheets (version 1.69.1). .\" Instead of manually editing it, you probably should edit the DocBook XML .\" source for it and then use the DocBook XSL Stylesheets to regenerate it. -.TH "CSET\-SHIELD" "1" "05/30/2008" "" "" +.TH "CSET\-SHIELD" "1" "02/07/2009" "" "" .\" disable hyphenation .nh .\" disable justification (adjust text to left margin only) @@ -19,8 +19,8 @@ cset\-shield \- cpuset supercommand which implements cpu shielding \fIcset\fR shield \-\-cpu 1\-7 \-\-kthread=on \fIcset\fR shield \-\-exec /opt/software/myapp/doit \-\-my_opt1 \-\-my_opt2 \fIcset\fR shield \-\-user appuser \-\-exec run_benchmark.sh -\fIcset\fR shield \-\-shield 1024,2048,5000\-1000 -\fIcset\fR shield \-\-unshield 6000\-8500 +\fIcset\fR shield \-\-shield \-\-pid 1024,2048,5000\-1000 +\fIcset\fR shield \-\-unshield \-\-pid 6000\-8500 \fIcset\fR shield \-\-kthread=off \fIcset\fR shield \-\-kthread=on \fIcset\fR shield \-\-shield bash @@ -45,11 +45,14 @@ use this USER for \-\-exec (id or name) \-\-group=GROUP use this GROUP for \-\-exec (id or name) .TP -\-s PIDSPEC, \-\-shield=PIDSPEC -shield specified PIDSPEC of processes or threads +\-s, \-\-shield +shield PIDSPEC specified with \-p/\-\-pid of processes or threads .TP -\-u PIDSPEC, \-\-unshield=PIDSPEC -remove specified PIDSPEC of processes or threads from the shield, the task keep running in the unshielded cpuset +\-u, \-\-unshield +remove PIDSPEC specified with \-p/\-\-pid of processes or threads from the shield, the tasks keep running in the unshielded cpuset +.TP +\-\-threads +if specified, any processes found in the PIDSPEC to have multiple threads will automatically have all their threads added to the PIDSPEC (use to shield or unshield all related threads) .TP \-k on|off, \-\-kthread=on|off shield from unbound interrupt threads as well @@ -74,7 +77,7 @@ Usually, CPU zero would be in the system set and the rest of the CPUs would be i If the optional \-\-kthread=on option is given to the shield command, then all kernel threads (with exception of the per\-CPU bound interrupt kernel threads) are also moved to the system set. -One executes processes on the shielded user cpuset with the \-\-exec subcommand or moves processes or threads to the shielded cpuset with the \-\-shield subcommand. +One executes processes on the shielded user cpuset with the \-\-exec subcommand or moves processes or threads to the shielded cpuset with the \-\-shield subcommand with a \-\-pid option. .sp .it 1 an-trap .nr an-no-space-flag 1 @@ -87,11 +90,11 @@ To create a shield, you would execute the shield command with the \-\-cpu option For example: -\fB# cset shield \-\-cpu=3\fR +\fB# cset shield \-\-cpu=1\-3\fR -On a 4\-way machine, this command will dedicate the first 3 processors, CPU0\-CPU2, for the system set (unshielded) and only the last processor, CPU3, for the user set (shielded). +On a 4\-way machine, this command will dedicate the first processor, CPU0, for the system set (unshielded) and the last three processors, CPU1, CPU2, CPU3, for the user set (shielded). -The CPUSPEC will accept a comma separated list of CPUs and inclusive range specifications. For example, \-\-cpu=1,3,5\-7 will assign CPU1, CPU3, CPU5, CPU6, and CPU7 to the user (or shielded) cpuset. +The CPUSPEC will accept a comma separated list of CPUs and inclusive range specifications. For example, \-\-cpu=1,3,5\-7 will assign CPU1, CPU3, CPU5, CPU6, and CPU7 to the user (or shielded) cpuset and the inverse of that to the system (or unshielded) cpuset. If you do not like the names "system" and "user" for the unshielded and shielded sets respectively, or if those names are used already, then use the \-\-sysset and \-\-userset options. @@ -108,9 +111,17 @@ The above command will use the name "free" for the unshielded system cpuset, the \fBNote\fR If you do use the \-\-syset/\-\-userset options, then you must continue to use those for every invocation of the shield supercommand. -After initialization, you can run the process of interest on the shielded cpuset with the \-\-exec subcommand, or move processes or threads already running to the shielded cpuset with the \-\-shield subcommand. +After initialization, you can run the process of interest on the shielded cpuset with the \-\-exec subcommand, or move processes or threads already running to the shielded cpuset with the \-\-shield subcommand and the \-\-pid option. -The PIDSPEC argument taken for the \-\-shield (or \-s) subcommand is a comma separated list of PIDs or TIDs. The list can also include brackets of PIDs or TIDs that are inclusive of the endpoints. +Note that if your execed command takes options, then use the traditional "\-\-" marker to separate cset\(cqs options from your command\(cqs options. + +For example: + +\fB# cset shield \-\-exec \(em ls \-l\fR + +This command will execute "ls \-l" inside the shield. + +The PIDSPEC argument taken for the \-\-pid (or \-p) option is a comma separated list of PIDs or TIDs. The list can also include brackets of PIDs or TIDs that are inclusive of the endpoints. For example: .sp @@ -118,9 +129,9 @@ For example: 1,2,5 Means processes 1, 2 and 5 1,2,600\-700 Means processes 1, 2 and from 600 to 700 .fi -\fB# cset shield \-\-shield=50\-65\fR +\fB# cset shield \-\-shield \-\-pid=50\-65\fR -The above command moves all processes and threads with PID or TID in the range 50\-65 inclusive, from any cpuset they may be running in into the shielded user cpuset. +The above command moves all processes and threads with PID or TID in the range 50\-65 inclusive, from the system cpuset into the shielded user cpuset. If they are running in the root cpuset, you must use the \-\-force option to actually move them into the shield. .sp .it 1 an-trap .nr an-no-space-flag 1 @@ -128,17 +139,10 @@ The above command moves all processes and threads with PID or TID in the range 5 .br \fBNote\fR The range of PIDs or TIDs does not need to have every position populated. In other words, for the example above, if there is only one process, say PID 57, in the range of 50\-65, then only that process will be moved. -.sp -.it 1 an-trap -.nr an-no-space-flag 1 -.nr an-break-flag 1 -.br -\fBCaution\fR -Please note that there is no checking of processes you request to move into the shield with the \-\-shield command. This means that the tool will happily move, for example, kernel threads that are bound to specific CPUs with this command. You can hang your system by indiscriminately specifying arbitrary PIDs to the \-\-shield command so please be careful. -The \-\-unshield (or \-u) subcommand will remove the specified processes or threads from the shielded cpuset and move them into the unshielded (or system) cpuset. This option also takes a PIDSPEC argument, the same as for the \-\-shield subcommand. +The \-\-unshield (or \-u) subcommand will remove the specified processes or threads from the shielded cpuset and move them into the unshielded (or system) cpuset. This command is also used in conjuction with a \-p/\-\-pid option that specifies a PIDSPEC argument, the same as for the \-\-shield subcommand. -Both the \-\-shield and the \-\-unshield commands will also finally output the number of tasks running in the shield and out of the shield. If you do not specify a PIDSPEC to these commands, then just that status is output. By specifying also a \-\-verbose in addition, then you will get a listing of every task that is running in either the shield or out of the shield. +Both the \-\-shield and the \-\-unshield commands will also finally output the number of tasks running in the shield and out of the shield if you do not specify a PIDSPEC with \-\-pid. By specifying also a \-\-verbose in addition, then you will get a listing of every task that is running in either the shield or out of the shield. Using no subcommand, ie. only "cset shield", will output the status of both shield and non\-shield. Tasks will be listed if \-\-verbose is used. @@ -161,13 +165,15 @@ Even though you can mix general usage of cpusets with the shielding concepts des .SH "LICENSE" Cpuset is licensed under the GNU GPL V2 only. .SH "COPYRIGHT" -Copyright (c) 2008 Novell Inc. +Copyright (c) 2008\-2009 Novell Inc. .SH "AUTHOR" Written by Alex Tsariounov . - -Some substrate code and ideas were taken from the excellent Stacked GIT (stgit) v0.13 (see http://gna.org/projects/stgit and http://www.procode.org/stgit). Stacked GIT is under GPL V2 or later. .SH "SEE ALSO" -\fBcset\fR(1), \fBcset\-set\fR(1), \fBcset\-proc\fR(1), \fBtaskset\fR(1), \fBchrt\fR(1) +cset(1), cset\-set(1), cset\-proc(1) + +/usr/share/doc/packages/cpuset/html/tutorial.html + +taskset(1), chrt(1) /usr/src/linux/Documentation/cpusets.txt diff --git a/doc/cset-shield.html b/doc/cset-shield.html index 076b054..d7f9888 100644 --- a/doc/cset-shield.html +++ b/doc/cset-shield.html @@ -3,7 +3,8 @@ - + +cset-shield(1) -cset-shield(1) -

OPTIONS

+

OPTIONS

-
-
+
+
-h, --help
@@ -328,7 +436,7 @@ cset-shield(1) Manual Page prints the list of options for this command

-
+
-c CPUSPEC, --cpu=CPUSPEC
@@ -336,7 +444,7 @@ cset-shield(1) Manual Page modifies or initializes the shield cpusets

-
+
-r, --reset
@@ -344,7 +452,7 @@ cset-shield(1) Manual Page destroys the shield

-
+
-e, --exec
@@ -352,7 +460,7 @@ cset-shield(1) Manual Page executes args in the shield

-
+
--user=USER
@@ -360,7 +468,7 @@ cset-shield(1) Manual Page use this USER for --exec (id or name)

-
+
--group=GROUP
@@ -368,24 +476,34 @@ cset-shield(1) Manual Page use this GROUP for --exec (id or name)

-
--s PIDSPEC, --shield=PIDSPEC +
+-s, --shield

- shield specified PIDSPEC of processes or threads + shield PIDSPEC specified with -p/--pid of processes or threads

-
--u PIDSPEC, --unshield=PIDSPEC +
+-u, --unshield

- remove specified PIDSPEC of processes or threads from the shield, - the task keep running in the unshielded cpuset + remove PIDSPEC specified with -p/--pid of processes or threads from + the shield, the tasks keep running in the unshielded cpuset

-
+
+--threads +
+
+

+ if specified, any processes found in the PIDSPEC to have multiple + threads will automatically have all their threads added to the + PIDSPEC (use to shield or unshield all related threads) +

+
+
-k on|off, --kthread=on|off
@@ -393,7 +511,7 @@ cset-shield(1) Manual Page shield from unbound interrupt threads as well

-
+
-f, --force
@@ -401,7 +519,7 @@ cset-shield(1) Manual Page force operation, use with care

-
+
-v, --verbose
@@ -409,7 +527,7 @@ cset-shield(1) Manual Page prints more detailed output, additive

-
+
--sysset=SYSSET
@@ -417,7 +535,7 @@ cset-shield(1) Manual Page optionally specify system cpuset name

-
+
--userset=USERSET
@@ -425,32 +543,32 @@ cset-shield(1) Manual Page optionally specify user cpuset name

-
+
-

DESCRIPTION

+

DESCRIPTION

-

This is a supercommand that creates basic cpu shielding. The +

This is a supercommand that creates basic cpu shielding. The normal cset commands can of course be used to create this basic shield, but the shield command combines many such commands to -create and manage a common type of cpu shielding setup.

-

The concept of shielding implies at minimum three cpusets, for +create and manage a common type of cpu shielding setup.

+

The concept of shielding implies at minimum three cpusets, for example: root, user and system. The root cpuset always exists in all implementations of cpusets and contains all available CPUs on the machine. The system cpuset is so named because normal system tasks are made to run on it. The user cpuset is so named because that is the "shielded" cpuset on which you would run your tasks -of interest.

-

Usually, CPU zero would be in the system set and the rest of the +of interest.

+

Usually, CPU zero would be in the system set and the rest of the CPUs would be in the user set. After creation of the cpusets, all processes running in the root cpuset are moved to the system cpuset. Thus any new processes or threads spawned from these -processes will also run the system cpuset.

-

If the optional --kthread=on option is given to the shield +processes will also run the system cpuset.

+

If the optional --kthread=on option is given to the shield command, then all kernel threads (with exception of the per-CPU -bound interrupt kernel threads) are also moved to the system set.

-

One executes processes on the shielded user cpuset with the +bound interrupt kernel threads) are also moved to the system set.

+

One executes processes on the shielded user cpuset with the --exec subcommand or moves processes or threads to the shielded -cpuset with the --shield subcommand.

+cpuset with the --shield subcommand with a --pid option.

@@ -460,32 +578,32 @@ cpuset with the --shield subcommand.

is running in initially when using the --shield subcommand.
-

To create a shield, you would execute the shield command with the +

To create a shield, you would execute the shield command with the --cpu option that specifies CPUSPEC argument that assigns CPUs to be under the shield (this means assigned to the user cpuset, all -other cpus will be assigned to the system set).

-

For example:

-

# cset shield --cpu=3

-

On a 4-way machine, this command will dedicate the first -3 processors, CPU0-CPU2, for the system set (unshielded) -and only the last processor, CPU3, for the user set -(shielded).

-

The CPUSPEC will accept a comma separated list of CPUs and +other cpus will be assigned to the system set).

+

For example:

+

# cset shield --cpu=1-3

+

On a 4-way machine, this command will dedicate the first +processor, CPU0, for the system set (unshielded) and the last +three processors, CPU1, CPU2, CPU3, for the user set (shielded).

+

The CPUSPEC will accept a comma separated list of CPUs and inclusive range specifications. For example, --cpu=1,3,5-7 will assign CPU1, CPU3, CPU5, CPU6, and CPU7 to the user (or shielded) -cpuset.

-

If you do not like the names "system" and "user" for the +cpuset and the inverse of that to the system (or unshielded) +cpuset.

+

If you do not like the names "system" and "user" for the unshielded and shielded sets respectively, or if those names are -used already, then use the --sysset and --userset options.

-

For example:

-

# cset shield --sysset=free --userset=cage --cpu=2,3 --kthread=on

-

The above command will use the name "free" for the unshielded +used already, then use the --sysset and --userset options.

+

For example:

+

# cset shield --sysset=free --userset=cage --cpu=2,3 --kthread=on

+

The above command will use the name "free" for the unshielded system cpuset, the name "cage" for the shielded user cpuset, initialize these cpusets and dedicate CPU0 and CPU1 to the "free" set and (on a 4-way machine) dedicate CPU2 and CPU3 to the "cage" set. Further, the command moves all processes and threads, including kernel threads from the root cpuset to the "free" -cpuset.

+cpuset.

@@ -495,24 +613,31 @@ cpuset.

continue to use those for every invocation of the shield supercommand.
-

After initialization, you can run the process of interest on the +

After initialization, you can run the process of interest on the shielded cpuset with the --exec subcommand, or move processes or threads already running to the shielded cpuset with the --shield -subcommand.

-

The PIDSPEC argument taken for the --shield (or -s) subcommand is -a comma separated list of PIDs or TIDs. The list can also -include brackets of PIDs or TIDs that are inclusive of the -endpoints.

-

For example:

+subcommand and the --pid option.

+

Note that if your execed command takes options, then use the +traditional "--" marker to separate cset’s options from your +command’s options.

+

For example:

+

# cset shield --exec — ls -l

+

This command will execute "ls -l" inside the shield.

+

The PIDSPEC argument taken for the --pid (or -p) option is a +comma separated list of PIDs or TIDs. The list can also include +brackets of PIDs or TIDs that are inclusive of the endpoints.

+

For example:

1,2,5               Means processes 1, 2 and 5
 1,2,600-700         Means processes 1, 2 and from 600 to 700
-

# cset shield --shield=50-65

-

The above command moves all processes and threads with PID or -TID in the range 50-65 inclusive, from any cpuset they may -be running in into the shielded user cpuset.

+

# cset shield --shield --pid=50-65

+

The above command moves all processes and threads with PID or TID +in the range 50-65 inclusive, from the system cpuset into the +shielded user cpuset. If they are running in the root cpuset, +you must use the --force option to actually move them into the +shield.

@@ -524,51 +649,39 @@ there is only one process, say PID 57, in the range of 50-65, then only that process will be moved.
-
- - - -
-
Caution
-		Please note that there is no checking of processes you -request to move into the shield with the --shield command. This -means that the tool will happily move, for example, kernel -threads that are bound to specific CPUs with this command. You -can hang your system by indiscriminately specifying arbitrary -PIDs to the --shield command so please be careful.
-
-

The --unshield (or -u) subcommand will remove the specified +

The --unshield (or -u) subcommand will remove the specified processes or threads from the shielded cpuset and move them into -the unshielded (or system) cpuset. This option also takes a -PIDSPEC argument, the same as for the --shield subcommand.

-

Both the --shield and the --unshield commands will also finally +the unshielded (or system) cpuset. This command is also used in +conjuction with a -p/--pid option that specifies a PIDSPEC +argument, the same as for the --shield subcommand.

+

Both the --shield and the --unshield commands will also finally output the number of tasks running in the shield and out of the -shield. If you do not specify a PIDSPEC to these commands, then -just that status is output. By specifying also a --verbose in -addition, then you will get a listing of every task that is -running in either the shield or out of the shield.

-

Using no subcommand, ie. only "cset shield", will output the +shield if you do not specify a PIDSPEC with --pid. By specifying +also a --verbose in addition, then you will get a listing of +every task that is running in either the shield or out of the +shield.

+

Using no subcommand, ie. only "cset shield", will output the status of both shield and non-shield. Tasks will be listed if ---verbose is used.

-

You can adjust which CPUs are in the shielded cpuset by issuing +--verbose is used.

+

You can adjust which CPUs are in the shielded cpuset by issuing the --cpu subcommand again anytime after the shield has been -initialized.

-

For example if the original shield contained CPU0 and CPU1 in the +initialized.

+

For example if the original shield contained CPU0 and CPU1 in the system set and CPU2 and CPU3 in the user set, if you then issue -the following command:

-

# cset shield --cpu=1,2,3

-

then that command will move CPU1 into the shielded "user" cpuset. +the following command:

+

# cset shield --cpu=1,2,3

+

then that command will move CPU1 into the shielded "user" cpuset. Any processes or threads that were running on CPU1 that belonged to the unshielded "system" cpuset are migrated to CPU0 by the -system.

-

The --reset subcommand will in essence destroy the shield. For +system.

+

The --reset subcommand will in essence destroy the shield. For example, if there was a shield on a 4-way machine with CPU0 in system and CPUs 1-3 in user with processes running on the user cpuset (i.e. in the shield), and a --reset subcommand was issued, then all processes running in both system and user cpusets would be migrated to the root cpuset (which has access to all CPUs and never goes away), after which both system and user cpusets would -be destroyed.

+be destroyed.

@@ -582,31 +695,29 @@ and proc) directly.
-

LICENSE

+

LICENSE

-

Cpuset is licensed under the GNU GPL V2 only.

+

Cpuset is licensed under the GNU GPL V2 only.

-

COPYRIGHT

+

COPYRIGHT

-

Copyright (c) 2008 Novell Inc.

+

Copyright (c) 2008-2009 Novell Inc.

-

AUTHOR

+

AUTHOR

-

Written by Alex Tsariounov <alext@novell.com>.

-

Some substrate code and ideas were taken from the excellent Stacked -GIT (stgit) v0.13 (see http://gna.org/projects/stgit and -http://www.procode.org/stgit). Stacked GIT is under GPL V2 or later.

+

Written by Alex Tsariounov <alext@novell.com>.

-

SEE ALSO

+

SEE ALSO

-

cset(1), cset-set(1), cset-proc(1), taskset(1), -chrt(1)

-

/usr/src/linux/Documentation/cpusets.txt

+

cset(1), cset-set(1), cset-proc(1)

+

/usr/share/doc/packages/cpuset/html/tutorial.html

+

taskset(1), chrt(1)

+

/usr/src/linux/Documentation/cpusets.txt

diff --git a/doc/cset-shield.txt b/doc/cset-shield.txt index c7ea89d..167da33 100644 --- a/doc/cset-shield.txt +++ b/doc/cset-shield.txt @@ -1,7 +1,7 @@ cset-shield(1) ============== Alex Tsariounov -v1.0, May 2008 +v1.5.0, February 2009 NAME ---- @@ -17,8 +17,8 @@ SYNOPSIS 'cset' shield --cpu 1-7 --kthread=on 'cset' shield --exec /opt/software/myapp/doit --my_opt1 --my_opt2 'cset' shield --user appuser --exec run_benchmark.sh -'cset' shield --shield 1024,2048,5000-1000 -'cset' shield --unshield 6000-8500 +'cset' shield --shield --pid 1024,2048,5000-1000 +'cset' shield --unshield --pid 6000-8500 'cset' shield --kthread=off 'cset' shield --kthread=on 'cset' shield --shield bash @@ -43,12 +43,17 @@ OPTIONS --group=GROUP:: use this GROUP for --exec (id or name) --s PIDSPEC, --shield=PIDSPEC:: - shield specified PIDSPEC of processes or threads +-s, --shield:: + shield PIDSPEC specified with -p/--pid of processes or threads --u PIDSPEC, --unshield=PIDSPEC:: - remove specified PIDSPEC of processes or threads from the shield, - the task keep running in the unshielded cpuset +-u, --unshield:: + remove PIDSPEC specified with -p/--pid of processes or threads from + the shield, the tasks keep running in the unshielded cpuset + +--threads:: + if specified, any processes found in the PIDSPEC to have multiple + threads will automatically have all their threads added to the + PIDSPEC (use to shield or unshield all related threads) -k on|off, --kthread=on|off:: shield from unbound interrupt threads as well @@ -92,7 +97,7 @@ bound interrupt kernel threads) are also moved to the system set. One executes processes on the shielded user cpuset with the --exec subcommand or moves processes or threads to the shielded -cpuset with the --shield subcommand. +cpuset with the --shield subcommand with a --pid option. NOTE: You do not need to specify which cpuset a process or thread is running in initially when using the --shield subcommand. @@ -104,16 +109,16 @@ other cpus will be assigned to the system set). For example: -*+# cset shield --cpu=3+* +*+# cset shield --cpu=1-3+* On a 4-way machine, this command will dedicate the first -3 processors, CPU0-CPU2, for the system set (unshielded) -and only the last processor, CPU3, for the user set -(shielded). +processor, CPU0, for the system set (unshielded) and the last +three processors, CPU1, CPU2, CPU3, for the user set (shielded). The CPUSPEC will accept a comma separated list of CPUs and inclusive range specifications. For example, --cpu=1,3,5-7 will assign CPU1, CPU3, CPU5, CPU6, and CPU7 to the user (or shielded) +cpuset and the inverse of that to the system (or unshielded) cpuset. If you do not like the names "system" and "user" for the @@ -138,47 +143,52 @@ continue to use those for every invocation of the shield supercommand. After initialization, you can run the process of interest on the shielded cpuset with the --exec subcommand, or move processes or threads already running to the shielded cpuset with the --shield -subcommand. +subcommand and the --pid option. + +Note that if your execed command takes options, then use the +traditional "--" marker to separate cset's options from your +command's options. + +For example: + +*+# cset shield --exec -- ls -l+* -The PIDSPEC argument taken for the --shield (or -s) subcommand is -a comma separated list of PIDs or TIDs. The list can also -include brackets of PIDs or TIDs that are inclusive of the -endpoints. +This command will execute "ls -l" inside the shield. + +The PIDSPEC argument taken for the --pid (or -p) option is a +comma separated list of PIDs or TIDs. The list can also include +brackets of PIDs or TIDs that are inclusive of the endpoints. For example: 1,2,5 Means processes 1, 2 and 5 1,2,600-700 Means processes 1, 2 and from 600 to 700 -*+# cset shield --shield=50-65+* +*+# cset shield --shield --pid=50-65+* -The above command moves all processes and threads with PID or -TID in the range 50-65 inclusive, from any cpuset they may -be running in into the shielded user cpuset. +The above command moves all processes and threads with PID or TID +in the range 50-65 inclusive, from the system cpuset into the +shielded user cpuset. If they are running in the root cpuset, +you must use the --force option to actually move them into the +shield. NOTE: The range of PIDs or TIDs does not need to have every position populated. In other words, for the example above, if there is only one process, say PID 57, in the range of 50-65, then only that process will be moved. -CAUTION: Please note that there is no checking of processes you -request to move into the shield with the --shield command. This -means that the tool will happily move, for example, kernel -threads that are bound to specific CPUs with this command. You -can hang your system by indiscriminately specifying arbitrary -PIDs to the --shield command so please be careful. - The --unshield (or -u) subcommand will remove the specified processes or threads from the shielded cpuset and move them into -the unshielded (or system) cpuset. This option also takes a -PIDSPEC argument, the same as for the --shield subcommand. +the unshielded (or system) cpuset. This command is also used in +conjuction with a -p/--pid option that specifies a PIDSPEC +argument, the same as for the --shield subcommand. Both the --shield and the --unshield commands will also finally output the number of tasks running in the shield and out of the -shield. If you do not specify a PIDSPEC to these commands, then -just that status is output. By specifying also a --verbose in -addition, then you will get a listing of every task that is -running in either the shield or out of the shield. +shield if you do not specify a PIDSPEC with --pid. By specifying +also a --verbose in addition, then you will get a listing of +every task that is running in either the shield or out of the +shield. Using no subcommand, ie. only "cset shield", will output the status of both shield and non-shield. Tasks will be listed if @@ -220,19 +230,18 @@ Cpuset is licensed under the GNU GPL V2 only. COPYRIGHT --------- -Copyright (c) 2008 Novell Inc. +Copyright (c) 2008-2009 Novell Inc. AUTHOR ------ -Written by Alex Tsariounov . - -Some substrate code and ideas were taken from the excellent Stacked -GIT (stgit) v0.13 (see http://gna.org/projects/stgit and -http://www.procode.org/stgit). Stacked GIT is under GPL V2 or later. +Written by Alex Tsariounov <\alext@novell.com>. SEE ALSO -------- -manlink:cset[1], csetlink:set[], csetlink:proc[], manlink:taskset[1], -manlink:chrt[1] +cset(1), cset-set(1), cset-proc(1) + +/usr/share/doc/packages/cpuset/html/tutorial.html + +taskset(1), chrt(1) /usr/src/linux/Documentation/cpusets.txt diff --git a/doc/cset.1 b/doc/cset.1 index 4b5c143..5ef21b7 100644 --- a/doc/cset.1 +++ b/doc/cset.1 @@ -2,7 +2,7 @@ .\" It was generated using the DocBook XSL Stylesheets (version 1.69.1). .\" Instead of manually editing it, you probably should edit the DocBook XML .\" source for it and then use the DocBook XSL Stylesheets to regenerate it. -.TH "CSET" "1" "05/30/2008" "" "" +.TH "CSET" "1" "02/07/2009" "" "" .\" disable hyphenation .nh .\" disable justification (adjust text to left margin only) @@ -12,7 +12,7 @@ cset \- manage cpusets functions in the Linux kernel .SH "SYNOPSIS" .sp .nf -\fIcset\fR [\-\-version | \-\-help | \-\-log] +\fIcset\fR [\-\-version | \-\-help | \-\-tohex] \fIcset\fR [help | \-\-help] \fIcset\fR [cset options] [command options] [args] .fi @@ -31,60 +31,69 @@ Cpuset is a Python application to make using the cpusets facilities in the Linux Setting up and managing a simple shielded CPU environment The concept of \fIshielded\fR cpus is that a certain number of cpus are partitioned off on the system and only processes that are of interest are run on these cpus (i.e., inside the shield). -For a simple shielded configuration, one typically uses three cpusets: the root set, a system set and a user set. \fICset\fR includes a super command that implements this strategy and lets you easily manage it. See \fBcset\-shield\fR(1) for more details. +For a simple shielded configuration, one typically uses three cpusets: the root set, a system set and a user set. \fICset\fR includes a super command that implements this strategy and lets you easily manage it. See \fIcset\-shield(1)\fR for more details. .TP Setting up and managing a comples shielding environment -Shielding can be more complex of course where concepts such as priority cpusets and intersecting cpuset can be used. You can use \fIcset\fR to help manage this type of shielding as well. You will need to use the \fBcset\-set\fR(1) and \fBcset\-proc\fR(1) subcommands directly to do that. +Shielding can be more complex of course where concepts such as priority cpusets and intersecting cpuset can be used. You can use \fIcset\fR to help manage this type of shielding as well. You will need to use the \fIcset\-set(1)\fR and \fIcset\-proc(1)\fR subcommands directly to do that. .TP Managing cpusets on the system -The cset subcommand \fBcset\-set\fR(1) allows you to create and destroy arbitrary cpusets on the system and assign arbitrary cpus and memory nodes to them. The cpusets so created have to follow the Linux kernel cpuset rules. See the \fBcset\-set\fR(1) subcommand for more details. +The cset subcommand \fIcset\-set(1)\fR allows you to create and destroy arbitrary cpusets on the system and assign arbitrary cpus and memory nodes to them. The cpusets so created have to follow the Linux kernel cpuset rules. See the \fIcset\-set(1)\fR subcommand for more details. .TP Managing processes that run on various system cpusets -The cset subcommand \fBcset\-proc\fR(1) allows you to manage processes running on various cpusets created on the system. You can exec new processes in specific cpusets and move tasks around existing cpusets. See the \fBcset\-proc\fR(1) subcommand for more details. +The cset subcommand \fIcset\-proc(1)\fR allows you to manage processes running on various cpusets created on the system. You can exec new processes in specific cpusets and move tasks around existing cpusets. See the \fIcset\-proc(1)\fR subcommand for more details. .SH "OPTIONS" The following generic option flags are available. Additional options are available per\-command, and documented in the command\-specific documentation. .TP \fIcset\fR \-\-version -Prints the cpuset suite version that the \fIcset\fR program came from, as well as version of other components used, such as Python and the kernel. +Display version information and exits. .TP \fIcset\fR \-\-help Prints the synopsis and a list of all commands. .TP \fIcset\fR \-\-log -Creates a log file for the current run. All manner of useful information is stored in this file. This is usually used to debug cpuset when things don't go as planned. +Creates a log file for the current run. All manner of useful information is stored in this file. This is usually used to debug cpuset when things don\(cqt go as planned. .TP -\fIcset\fR help \fIcommand\fR -Although not strictly an "option", this is a very helpful way to execute help with cpuset. The command prints out a lengthy summary of how the specified subcommand works. +\fIcset\fR \-\-machine +Makes cset output information for all operations in a format that is machine readable (i.e. easy to parse). +.TP +\fIcset\fR \-\-tohex +Converts a CPUSPEC (see \fIcset\-set(1)\fR for definition) to a hexadecimal number and outputs it. Useful for setting IRQ stub affinity to a cpuset definition. .SH "CPUSET COMMANDS" The cpuset commands are divided into groups, according to the primary purpose of those commands. Following is a short description of each command. A more detailed description is available in individual command manpages. Those manpages are named \fIcset\-(1)\fR. The first command, \fIhelp\fR, is especially useful as it prints out a long summary of what a particular command does. .TP -\fIcset help\fR -print the detailed command usage -.TP -\fIcset version\fR -display version information +\fIcset help command\fR +print out a lengthy summary of how the specified subcommand works .TP -\fIcset copyright\fR -display copyright information +\fIcset command \-\-help\fR +print out an extended synopsis of the specified subcommand .TP \fIcset shield\fR -supercommand to set up and manage basic shielding (see \fBcset\-shield\fR(1)) +supercommand to set up and manage basic shielding (see \fIcset\-shield(1)\fR) .TP \fIcset set\fR -create, modify and destroy cpusets (see \fBcset\-set\fR(1)) +create, modify and destroy cpusets (see \fIcset\-set(1)\fR) .TP \fIcset proc\fR -create and manage processes within cpusets (see \fBcset\-proc\fR(1)) +create and manage processes within cpusets (see \fIcset\-proc(1)\fR) +.SH "FILES" +Cpuset uses a configuration file if present on the system. The file is \fI/etc/cset.conf\fR and may contain the following options. +.TP +mountpoint = +Specify the mountpoint where the cpuset filesystem is to be mounted. By default this is \fI/cpusets\fR; however, some people prefer to mount this in the more traditional \fI/dev/cpusets\fR. .SH "LICENSE" Cpuset is licensed under the GNU GPL V2 only. .SH "COPYRIGHT" -Copyright (c) 2008 Novell Inc. +Copyright (c) 2008\-2009 Novell Inc. .SH "AUTHOR" -Written by Alex Tsariounov . +Written by Alex Tsariounov Some substrate code and ideas were taken from the excellent Stacked GIT (stgit) v0.13 (see http://gna.org/projects/stgit and http://www.procode.org/stgit). Stacked GIT is under GPL V2 or later. .SH "SEE ALSO" -\fBcset\-set\fR(1), \fBcset\-proc\fR(1), \fBcset\-shield\fR(1), \fBtaskset\fR(1), \fBchrt\fR(1) +cset\-set(1), cset\-proc(1), cset\-shield(1) + +/usr/share/doc/packages/cpuset/html/tutorial.html + +taskset(1), chrt(1) /usr/src/linux/Documentation/cpusets.txt diff --git a/doc/cset.html b/doc/cset.html index ae45e97..3af961c 100644 --- a/doc/cset.html +++ b/doc/cset.html @@ -3,7 +3,8 @@ - + +cset(1) -cset(1) -

DESCRIPTION

+

DESCRIPTION

@@ -320,14 +428,14 @@ tool mounts the cpusets filesystem and manipulates it. Non-root users do not have permission for these actions.
-

Cpuset is a Python application to make using the cpusets facilities in +

Cpuset is a Python application to make using the cpusets facilities in the Linux kernel easier. The actual included command is called cset and it allows manipulation of cpusets on the system and provides higher level functions such as implementation and control of a basic -cpu shielding setup.

-

Typical uses of cpuset include

-
-
+cpu shielding setup.

+

Typical uses of cpuset include

+
+
Setting up and managing a simple shielded CPU environment
@@ -336,12 +444,12 @@ Setting up and managing a simple shielded CPU environment cpus are partitioned off on the system and only processes that are of interest are run on these cpus (i.e., inside the shield).

-

For a simple shielded configuration, one typically uses three cpusets: +

For a simple shielded configuration, one typically uses three cpusets: the root set, a system set and a user set. Cset includes a super command that implements this strategy and lets you easily manage it. -See cset-shield(1) for more details.

+See cset-shield(1) for more details.

-
+
Setting up and managing a comples shielding environment
@@ -349,53 +457,51 @@ Setting up and managing a comples shielding environment Shielding can be more complex of course where concepts such as priority cpusets and intersecting cpuset can be used. You can use cset to help manage this type of shielding as well. You - will need to use the cset-set(1) and cset-proc(1) + will need to use the cset-set(1) and cset-proc(1) subcommands directly to do that.

-
+
Managing cpusets on the system

- The cset subcommand cset-set(1) allows you to create and + The cset subcommand cset-set(1) allows you to create and destroy arbitrary cpusets on the system and assign arbitrary cpus and memory nodes to them. The cpusets so created have to - follow the Linux kernel cpuset rules. See the cset-set(1) + follow the Linux kernel cpuset rules. See the cset-set(1) subcommand for more details.

-
+
Managing processes that run on various system cpusets

- The cset subcommand cset-proc(1) allows you to manage + The cset subcommand cset-proc(1) allows you to manage processes running on various cpusets created on the system. You can exec new processes in specific cpusets and move tasks - around existing cpusets. See the cset-proc(1) subcommand + around existing cpusets. See the cset-proc(1) subcommand for more details.

-
+
-

OPTIONS

+

OPTIONS

-

The following generic option flags are available. Additional options +

The following generic option flags are available. Additional options are available per-command, and documented in the command-specific -documentation.

-
-
+documentation.

+
+
cset --version

- Prints the cpuset suite version that the cset program came - from, as well as version of other components used, such as - Python and the kernel. + Display version information and exits.

-
+
cset --help
@@ -403,114 +509,134 @@ documentation.

Prints the synopsis and a list of all commands.

-
+
cset --log <filename>

Creates a log file for the current run. All manner of useful information is stored in this file. This is usually used to - debug cpuset when things don't go as planned. + debug cpuset when things don’t go as planned.

-
-cset help command +
+cset --machine

- Although not strictly an "option", this is a very helpful way - to execute help with cpuset. The command prints out a lengthy - summary of how the specified subcommand works. + Makes cset output information for all operations in a format + that is machine readable (i.e. easy to parse).

-
+
+cset --tohex <CPUSPEC> +
+
+

+ Converts a CPUSPEC (see cset-set(1) for definition) to a + hexadecimal number and outputs it. Useful for setting IRQ + stub affinity to a cpuset definition. +

+
+
-

CPUSET COMMANDS

+

CPUSET COMMANDS

-

The cpuset commands are divided into groups, according to the primary +

The cpuset commands are divided into groups, according to the primary purpose of those commands. Following is a short description of each command. A more detailed description is available in individual command manpages. Those manpages are named cset-<command>(1). The first command, help, is especially useful as it prints out a long -summary of what a particular command does.

-
-
-cset help -
-
-

- print the detailed command usage -

-
-
-cset version +summary of what a particular command does.

+
+
+cset help command

- display version information + print out a lengthy summary of how the specified subcommand + works

-
-cset copyright +
+cset command --help

- display copyright information + print out an extended synopsis of the specified subcommand

-
+
cset shield

supercommand to set up and manage basic shielding (see - cset-shield(1)) + cset-shield(1))

-
+
cset set

- create, modify and destroy cpusets (see cset-set(1)) + create, modify and destroy cpusets (see cset-set(1))

-
+
cset proc

create and manage processes within cpusets (see - cset-proc(1)) + cset-proc(1)) +

+
+
+
+

FILES

+
+

Cpuset uses a configuration file if present on the system. The file +is /etc/cset.conf and may contain the following options.

+
+
+mountpoint = <directory_name> +
+
+

+ Specify the mountpoint where the cpuset filesystem is to be + mounted. By default this is /cpusets; however, some people + prefer to mount this in the more traditional /dev/cpusets.

-
+
-

LICENSE

+

LICENSE

-

Cpuset is licensed under the GNU GPL V2 only.

+

Cpuset is licensed under the GNU GPL V2 only.

-

COPYRIGHT

+

COPYRIGHT

-

Copyright (c) 2008 Novell Inc.

+

Copyright (c) 2008-2009 Novell Inc.

-

AUTHOR

+

AUTHOR

-

Written by Alex Tsariounov <alext@novell.com>.

-

Some substrate code and ideas were taken from the excellent Stacked +

Written by Alex Tsariounov <alext@novell.com>

+

Some substrate code and ideas were taken from the excellent Stacked GIT (stgit) v0.13 (see http://gna.org/projects/stgit and -http://www.procode.org/stgit). Stacked GIT is under GPL V2 or later.

+http://www.procode.org/stgit). Stacked GIT is under GPL V2 or later.

-

SEE ALSO

+

SEE ALSO

-

cset-set(1), cset-proc(1), cset-shield(1), taskset(1), -chrt(1)

-

/usr/src/linux/Documentation/cpusets.txt

+

cset-set(1), cset-proc(1), cset-shield(1)

+

/usr/share/doc/packages/cpuset/html/tutorial.html

+

taskset(1), chrt(1)

+

/usr/src/linux/Documentation/cpusets.txt

diff --git a/doc/cset.txt b/doc/cset.txt index 163a115..6f2b7f3 100644 --- a/doc/cset.txt +++ b/doc/cset.txt @@ -10,7 +10,7 @@ cset - manage cpusets functions in the Linux kernel SYNOPSIS -------- [verse] -'cset' [--version | --help | --log] +'cset' [--version | --help | --tohex] 'cset' [help | --help] 'cset' [cset options] [command options] [args] @@ -37,27 +37,27 @@ Setting up and managing a simple shielded CPU environment:: For a simple shielded configuration, one typically uses three cpusets: the root set, a system set and a user set. 'Cset' includes a super command that implements this strategy and lets you easily manage it. -See csetlink:shield[] for more details. +See 'cset-shield(1)' for more details. Setting up and managing a comples shielding environment:: Shielding can be more complex of course where concepts such as priority cpusets and intersecting cpuset can be used. You can use 'cset' to help manage this type of shielding as well. You - will need to use the csetlink:set[] and csetlink:proc[] + will need to use the 'cset-set(1)' and 'cset-proc(1)' subcommands directly to do that. Managing cpusets on the system:: - The cset subcommand csetlink:set[] allows you to create and + The cset subcommand 'cset-set(1)' allows you to create and destroy arbitrary cpusets on the system and assign arbitrary cpus and memory nodes to them. The cpusets so created have to - follow the Linux kernel cpuset rules. See the csetlink:set[] + follow the Linux kernel cpuset rules. See the 'cset-set(1)' subcommand for more details. Managing processes that run on various system cpusets:: - The cset subcommand csetlink:proc[] allows you to manage + The cset subcommand 'cset-proc(1)' allows you to manage processes running on various cpusets created on the system. You can exec new processes in specific cpusets and move tasks - around existing cpusets. See the csetlink:proc[] subcommand + around existing cpusets. See the 'cset-proc(1)' subcommand for more details. OPTIONS @@ -67,9 +67,7 @@ are available per-command, and documented in the command-specific documentation. 'cset' --version:: - Prints the cpuset suite version that the 'cset' program came - from, as well as version of other components used, such as - Python and the kernel. + Display version information and exits. 'cset' --help:: Prints the synopsis and a list of all commands. @@ -79,10 +77,14 @@ documentation. information is stored in this file. This is usually used to debug cpuset when things don't go as planned. -'cset' help 'command':: - Although not strictly an "option", this is a very helpful way - to execute help with cpuset. The command prints out a lengthy - summary of how the specified subcommand works. +'cset' --machine:: + Makes cset output information for all operations in a format + that is machine readable (i.e. easy to parse). + +'cset' --tohex :: + Converts a CPUSPEC (see 'cset-set(1)' for definition) to a + hexadecimal number and outputs it. Useful for setting IRQ + stub affinity to a cpuset definition. CPUSET COMMANDS --------------- @@ -93,20 +95,30 @@ command manpages. Those manpages are named 'cset-(1)'. The first command, 'help', is especially useful as it prints out a long summary of what a particular command does. -'cset help':: - print the detailed command usage -'cset version':: - display version information -'cset copyright':: - display copyright information +'cset help command':: + print out a lengthy summary of how the specified subcommand + works +'cset command --help':: + print out an extended synopsis of the specified subcommand + 'cset shield':: supercommand to set up and manage basic shielding (see - csetlink:shield[]) + 'cset-shield(1)') 'cset set':: - create, modify and destroy cpusets (see csetlink:set[]) + create, modify and destroy cpusets (see 'cset-set(1)') 'cset proc':: create and manage processes within cpusets (see - csetlink:proc[]) + 'cset-proc(1)') + +FILES +----- +Cpuset uses a configuration file if present on the system. The file +is '/etc/cset.conf' and may contain the following options. + +mountpoint = :: + Specify the mountpoint where the cpuset filesystem is to be + mounted. By default this is '/cpusets'; however, some people + prefer to mount this in the more traditional '/dev/cpusets'. LICENSE ------- @@ -114,19 +126,22 @@ Cpuset is licensed under the GNU GPL V2 only. COPYRIGHT --------- -Copyright (c) 2008 Novell Inc. +Copyright (c) 2008-2009 Novell Inc. AUTHOR ------ -Written by Alex Tsariounov . +Written by Alex Tsariounov <\alext@novell.com> Some substrate code and ideas were taken from the excellent Stacked -GIT (stgit) v0.13 (see http://gna.org/projects/stgit and -http://www.procode.org/stgit). Stacked GIT is under GPL V2 or later. +GIT (stgit) v0.13 (see \http://gna.org/projects/stgit and +\http://www.procode.org/stgit). Stacked GIT is under GPL V2 or later. SEE ALSO -------- -csetlink:set[], csetlink:proc[], csetlink:shield[], manlink:taskset[1], -manlink:chrt[1] +cset-set(1), cset-proc(1), cset-shield(1) + +/usr/share/doc/packages/cpuset/html/tutorial.html + +taskset(1), chrt(1) /usr/src/linux/Documentation/cpusets.txt diff --git a/doc/tutorial.html b/doc/tutorial.html new file mode 100644 index 0000000..1c1d44d --- /dev/null +++ b/doc/tutorial.html @@ -0,0 +1,1800 @@ + + + + + +Cpuset (cset) Tutorial + + + + + +
+
+

Alex Tsariounov <alext@novell.com>
+Copyright (c) 2009 Novell Inc., cset v1.5.0
+Verbatim copying and distribution of this entire article are permitted +worldwide, without royalty, in any medium, provided this notice is preserved.

+

This tutorial describes basic and advanced usage of the cset command to +manipulate cpusets on a Linux system. See also the manpages that come with +the cset command: cset(1), cset-shield(1), cset-set(1), and cset-proc(1) for +more details. Additionally, the cset command has online help.

+
+
+

1. Introduction

+
+

In the Linux kernel, the cpuset facility provides a mechanism for creating +logical entities called "cpusets" that encompass definitions of CPUs and NUMA +Memory Nodes (if NUMA is available). Cpusets constrain the CPU and Memory +placement of a task to only the resources defined within that cpuset. These +cpusets can then be arranged into a nested hierarchy visible in the "cpuset" +virtual filesystem. Sets of tasks can be assigned to these cpusets to +constrain the resources that they use. The tasks can be moved from one cpuset +to another to utilize other resources defined in those other cpusets.

+

The cset command is a Python application that provides a command line front +end for the Linux cpusets functionality. Working with cpusets directly can be +confusing and slightly complex. The cset tool hides that complexity behind +an easy-to-use command line interface.

+

There are two distinct use cases for cset: the basic shielding use case and +the "advanced" case of using raw set and proc subcommands. The basic +shielding function is accessed with the shield subcommand and described in +the next section. Using the raw set and proc subcommands allows one to +set up arbitrarily complex cpusets and is described in the later sections.

+

Note that in general, one either uses the shield subcommand or a +combination of the set and proc subcommands. One rarely, if ever, uses +all of these subcommands together. Doing so will likely become too +confusing. Additionally, the shield subcommand sets up its required cpusets +with exclusively marked CPUs. This can interfere with your cpuset +strategy. If you find that you need more functionality for your strategy than +shield provides, go ahead and transition to using set and proc +exclusively. It is straightforward to implement what shield does with a few +extra set and proc subcommands.

+

1.1. Obtaining Online Help

+
+
+For a full list of cset subcommands +
+
+

+ # cset help +

+
+
+For in-depth help on individual subcommands +
+
+

+ # cset help <subcommand> +

+
+
+For options of individual subcommands +
+
+

+ # cset <subcommand> (-h | --help) +

+
+
+
+

2. The Basic Shielding Model

+
+

Although any set up of cpusets can really be described as "shielding," there +is one prevalent shielding model in use that is so common that cset has a +subcommand that is dedicated to its use. This subcommand is called shield.

+

The concept behind this model is the use of three cpusets. The root cpuset +which is always present in all configurations and contains all CPUs. The +system cpuset which contains CPUs which are used for system tasks. These +are the normal tasks that are not "important," but which need to run on the +system. And finally, the user cpuset which contains CPUs which are used for +"important" tasks. The user cpuset is the shield. Only those tasks that +are somehow important, usually tasks whose performance determines the overall +rating for the machine, are run in the user cpuset.

+

The shield subcommand manages all of these cpusets and lets you define the +CPUs and Memory Nodes that are in the shielded and unshielded sets. The +subcommand automatically moves all movable tasks on the system into the +unshielded cpuset on shield activation, and back into the root cpuset on +shield tear down. The subcommand then lets you move tasks into and out of the +shield. Additionally, you can move special tasks (kernel threads) which +normally run in the root cpuset into the unshielded set so that your +shield will have even less disturbance.

+

The shield subcommand abstracts the management of these cpusets away from +you and provides options that drive how the shield is set up, which tasks are +to be shielded and which tasks are not, and status of the shield. In fact, +you need not be bothered with the naming of the required cpusets or even where +the cpuset filesystem is mounted. Cset and the shield subcommand takes +care of all that.

+

If you find yourself needing to define more cpusets for your application, then +it is likely that this simple shielding is not a rich enough model for you. +In this case, you should transition to using the set and proc subcommands +described in a later section.

+

2.1. A Simple Shielding Example

+

Assume that we have a 4-way machine that is not NUMA. This means there are 4 +CPUs at our disposal and there is only one Memory Node available. On such +machines, we do not need to specify any memory node parameters to cset, it +sets up the only available memory node by default.

+

Usually, one wants to dedicate as many CPUs to the shield as possible and +leave a minimal set of CPUs for normal system processing. The reasoning for +this is because the performance of the important tasks will rule the +performance of the installation as a whole and these important tasks need as +many resources available to them as possible, exclusive of other, unimportant +tasks that are running on the system.

+
+ + + +
+
Note
+		I use the word "task" to represent either a process or a thread that is +running on the system.
+
+

2.1.1. Setup and Teardown of the Shield

+

To set up a shield of 3 CPUs with 1 CPU left for low priority system +processing, issue the following command.

+
+
+
[zuul:cpuset-trunk]# cset shield -c 1-3
+cset: --> activating shielding:
+cset: moving 176 tasks from root into system cpuset...
+[==================================================]%
+cset: "system" cpuset of CPUSPEC(0) with 176 tasks running
+cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running
+
+

This command does a number of things. First, a user cpuset is created with +what’s called a CPUSPEC (CPU specification) from the -c/--cpu option. This +CPUSPEC specifies to use CPUs 1 through 3 inclusively. Next, the command +creates a system cpuset with a CPUSPEC that is the inverse of the -c +option for the current machine. On this machine that cpuset will only contain +the first CPU, CPU0. Next, all userspace processes running in the root +cpuset are transfered to the system cpuset. This makes all those processes +run only on CPU0. The effect of this is that the shield consists of CPUs 1 +through 3 and they are now idling.

+

Note that the command did not move the kernel threads that are running in the +root cpuset to the system cpuset. This is because you may want these +kernel threads to use all available CPUs. If you do not, the you can use the +-k/--kthread option as described below.

+

The shield setup command above outputs the information of which cpusets were +created and how many tasks are running on each. If you want to see the +current status of the shield again, issue this command:

+
+
+
[zuul:cpuset-trunk]# cset shield
+cset: --> shielding system active with
+cset: "system" cpuset of CPUSPEC(0) with 176 tasks running
+cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running
+
+

Which shows us that the shield is set up and that 176 tasks are running in the +system cpuset—the "unshielded" cpuset.

+

It is important to move all possible tasks from the root cpuset to the +unshielded system cpuset because a task’s cpuset property is inherited by +its children. Since we’ve moved all running tasks (including init) to the +unshielded system cpuset, that means that any new tasks that are spawned +will also run in the unshielded system cpuset.

+

Some kernel threads can be moved into the unshielded system cpuset as well. +These are the threads that are not bound to specific CPUs. If a kernel thread +is bound to a specific CPU, then it is generally not a good idea to move that +thread to the system set because at worst it may hang the system and at best +it will slow the system down significantly. These threads are usually the IRQ +threads on a real time Linux kernel, for example, and you may want to not move +these kernel threads into system. If you leave them in the root cpuset, +then they will have access to all CPUs.

+

However, if your application demands an even "quieter" shield, then you can +move all movable kernel threads into the unshielded system set with the +following command.

+
+
+
[zuul:cpuset-trunk]# cset shield -k on
+cset: --> activating kthread shielding
+cset: kthread shield activated, moving 70 tasks into system cpuset...
+[==================================================]%
+cset: done
+
+

You can see that this moved an additional 70 tasks to the unshielded system +cpuset. Note that the -k/--kthread on parameter can be given at the shield +creation time as well and you do not need to perform these two steps +separately if you know that you will want kernel thread shielding as well. +Executing cset shield again shows us the current state of the shield.

+
+
+
[zuul:cpuset-trunk]# cset shield
+cset: --> shielding system active with
+cset: "system" cpuset of CPUSPEC(0) with 246 tasks running
+cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running
+
+

You can get a detailed listing of what is running in the shield by specifying +either -s/--shield or -u/--unshield to the shield subcommand and using +the verbose flag. You will get output similar to the following.

+
+
+
[zuul:cpuset-trunk]# cset shield --unshield -v
+cset: "system" cpuset of CPUSPEC(0) with 251 tasks running
+   USER       PID  PPID SPPr TASK NAME
+   -------- ----- ----- ---- ---------
+   root         1     0 Soth init [5]
+   root         2     0 Soth [kthreadd]
+   root        84     2 Sf50 [IRQ-9]
+...
+   alext    31796 31789 Soth less
+   root     32653 25222 Roth python ./cset shield --unshield -v
+
+

Note that I abbreviated the listing; we do have 251 tasks running in the +system set. The output is self-explanatory; however, the "SPPr" field may +need a little explanation. "SPPr" stands for State, Policy and Priority. You +can see that the initial two tasks are Stopped and running in timeshare +priority, marked as "oth" (for "other"). The [IRQ-9] task is also stopped, +but marked at real time FIFO policy with a priority of 50. The last task in +the listing is the cset command itself and is marked as running. Also note +that adding a second -v/--verbose option will not restrict the output to +fit into an 80 character screen.

+

Tear down of the shield, stopping the shield in other words, is done with the +-r/--reset option to the shield subcommand. When this command is issued, +both the system and user cpusets are deleted and any tasks that are +running in both of those cpusets are moved to the root cpuset. Once so +moved, all tasks will have access to all resources on the system. For +example:

+
+
+
[zuul:cpuset-trunk]# cset shield --reset
+cset: --> deactivating/reseting shielding
+cset: moving 0 tasks from "/user" user set to root set...
+cset: moving 250 tasks from "/system" system set to root set...
+[==================================================]%
+cset: deleting "/user" and "/system" sets
+cset: done
+
+

2.1.2. Moving Interesting Tasks Into and Out of the Shield

+

Now that we have a shield running, the objective is to run our "important" +processes in that shield. These processes can be anything, but usually they +are directly related to the purpose of the machine. There are two ways to run +tasks in the shield:

+
    +
  1. +

    +Exec a process into the shield +

    +
    2. +
  2. +

    +Move an already running task into the shield +

    +
    3. +
+
Execing a Process into the Shield
+

Running a new process in the shield can be done with the -e/--exec option +to the shield subcommand. This is the simplest way to get a task to run in +the shield. For this example, let’s exec a new bash shell into the shield +with the following commands.

+
+
+
[zuul:cpuset-trunk]# cset shield -s
+cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running
+cset: done
+
+[zuul:cpuset-trunk]# cset shield -e bash
+cset: --> last message, executed args into cpuset "/user", new pid is: 13300
+
+[zuul:cpuset-trunk]# cset shield -s -v
+cset: "user" cpuset of CPUSPEC(1-3) with 2 tasks running
+   USER       PID  PPID SPPr TASK NAME
+   -------- ----- ----- ---- ---------
+   root     13300  8583 Soth bash
+   root     13329 13300 Roth python ./cset shield -s -v
+
+[zuul:cpuset-trunk]# exit
+
+[zuul:cpuset-trunk]# cset shield -s
+cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running
+cset: done
+
+

The first command above lists the status of the shield. We see that the +shield is defined as CPUs 1 through 3 inclusive and currently there are no +tasks running in it.

+

The second command execs the bash shell into the shield with the -e +option. The last message of cset lists the PID of the new process.

+
+ + + +
+
Note
+		cset follows the tradition of separating the tool options from the +command to be execed options with a double dash (--). This is not shown in +this simple example, but if the command you want to exec also takes options, +separate them with the double dash like so: # cset shield -e mycommand -- -v +The -v will be passed to mycommand, and not to cset.
+
+

The next command lists the status of the shield again. You will note that +there are actually two tasks running shielded: our new shell and the cset +status command itself. Remember that the cpuset property of a task is +inherited by its children. Since we ran the new shell in the shield, its +child, which is the status command, also ran in the shield.

+
+ + + +
+
Tip
+		Execing a shell into the shield is a useful way to experiment with +running tasks in the shield since all children of the shell will also run in +the shield.
+
+

The last command exits the shell after which we request a shield status again +and see that once again, it does not contain any tasks.

+

You may have noticed in the output above that both the new shell and the +status command are running as the root user. This is because cset needs to +run as root and so all it’s children will also run as root. If you need to +run a process under a different user and or group, you may use the --user +and --group options for exec as follows.

+
+
+
[zuul:cpuset-trunk]# cset shield --user=alext --group=users -e bash
+cset: --> last message, executed args into cpuset "/user", new pid is: 14212
+
+alext@zuul> cset shield -s -v
+cset: "user" cpuset of CPUSPEC(1-3) with 2 tasks running
+   USER       PID  PPID SPPr TASK NAME
+   -------- ----- ----- ---- ---------
+   alext    14212  8583 Soth bash
+   alext    14241 14212 Roth python ./cset shield -s -v
+
+
Moving a Running Task into and out of the Shield
+

While execing a process into the shield is undoubtably useful, most of the +time, you’ll want to move already running tasks into and out of the shield. +The cset shield subcommand includes two options for doing this: +-s/--shield and -u/--unshield. These options require what’s called a +PIDSPEC (process specification) to also be specified with the -p/--pid +option. The PIDSPEC defines which tasks get operated on. The PIDSPEC can be +a single process ID, a list of process IDs separated by commas, and a list of +process ID ranges separated by dashes, groups of which are separated by +commas. For example:

+
+
+--shield --pid 1234 +
+
+

+This PIDSPEC argument specifies that PID 1234 be shielded. +

+
+
+--shield --pid 1234,42,1934,15000,15001,15002 +
+
+

+This PIDSPEC argument specifies that this list of PIDs only be moved into the +shield. +

+
+
+--unshield -p 5000,5100,6010-7000,9232 +
+
+

+This PIDSPEC argument specifies that PIDs 5000,5100 and 9232 be unshielded +(moved out of the shield) along with any existing PID that is in the range +6010 through 7000 inclusive. +

+
+
+
+ + + +
+
Note
+		A range in a PIDSPEC does not have to have tasks running for every +number in that range. In fact, it is not even an error if there are no tasks +running in that range; none will be moved in that case. The range simply +specifies to act on any tasks that have a PID or TID that is within that +range.
+
+

Use of the appropriate PIDSPEC can thus be handy to move tasks and groups of +tasks into and out of the shield. Additionally, there is one more option that +can help with multi-threaded processes, and that is the --threads flag. If +this flag is present in a shield or unshield command with a PIDSPEC and if +any of the task IDs in the PIDSPEC belong to a thread in a process container, +then all the sibling threads in that process container will get shielded or +unshielded as well. This flag provides an easy mechanism to shield/unshield +all threads of a process by simply specifying one thread in that process.

+

In the following example, we move the current shell into the shield with a +range PIDSPEC and back out with the bash variable for the current PID.

+
+
+
[zuul:cpuset-trunk]# echo $$
+22018
+
+[zuul:cpuset-trunk]# cset shield -s -p 22010-22020
+cset: --> shielding following pidspec: 22010-22020
+cset: done
+
+[zuul:cpuset-trunk]# cset shield -s -v
+cset: "user" cpuset of CPUSPEC(1-3) with 2 tasks running
+   USER       PID  PPID SPPr TASK NAME
+   -------- ----- ----- ---- ---------
+   root      3770 22018 Roth python ./cset shield -s -v
+   root     22018  5034 Soth bash
+cset: done
+
+[zuul:cpuset-trunk]# cset shield -u -p $$
+cset: --> unshielding following pidspec: 22018
+cset: done
+
+[zuul:cpuset-trunk]# cset shield -s
+cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running
+cset: done
+
+
+ + + +
+
Note
+		Ordinarily, the shield option will shield a PIDSPEC only if it is +currently running in the system set—the unshielded set. The unshield +option will unshield a PIDSPEC only if it is currently running in the user +set—the shielded set. If you want to shield/unshield a process that +happens to be running in the root set (not common), then use the --force +option for these commands.
+
+
+

3. Full Featured Cpuset Manipulation Commands

+
+

While basic shielding as described above is useful and a common use model for +cset, there comes a time when more functionality will be desired to +implement your strategy. To implement this, cset provides two subcommands: +set, which allows you to manipulate cpusets; and proc, which allows you to +manipulate processes within those cpusets.

+

3.1. The Set Subcommand

+

In order to do anything with cpusets, you must be able to create, adjust, +rename, move and destroy them. The set subcommand allows the management of +cpusets in such a manner.

+

3.1.1. Creating and Destroying Cpusets with Set

+

The basic syntax of set for cpuset creation is:

+
+
+
[zuul:cpuset-trunk]# cset set -c 1-3 -s my_cpuset1
+cset: --> created cpuset "my_cpuset1"
+
+

This creates a cpuset named "my_cpuset1" with a CPUSPEC of CPU1, CPU2 and +CPU3. The CPUSPEC is the same concept as described in the "Setup and +Teardown of the Shield" section above. The set subcommand also takes a +-m/--mem option that lets you specify the memory nodes the set will use as +well as flags to make the CPUs and MEMs exclusive to the cpuset. If you are +on a non-NUMA machine, just leave the -m option out and the default memory +node 0 will be used.

+

Just like with shield, you can adjust the CPUs and MEMs with subsequent +calls to set. If, for example, you wish to adjust the "my_cpuset1" cpuset +to only use CPUs 1 and 3 (and omit CPU2), then issue the following command.

+
+
+
[zuul:cpuset-trunk]# cset set -c 1,3 -s my_cpuset1
+cset: --> modified cpuset "my_cpuset
+
+

cset will then adjust the CPUs that are assigned to the "my_cpuset1" set to + only use CPU1 and CPU3.

+

To rename a cpuset, use the -n/--newname option. For example:

+
+
+
[zuul:cpuset-trunk]# cset set -s my_cpuset1 -n super_set
+cset: --> renaming "/cpusets/my_cpuset1" to "super_set"
+
+

Renames the cpuset called "my_cpuset1" to "super_set".

+

To destroy a cpuset, use the -d/--destroy option as follows.

+
+
+
[zuul:cpuset-trunk]# cset set -d super_set
+cset: --> processing cpuset "super_set", moving 0 tasks to parent "/"...
+cset: --> deleting cpuset "/super_set"
+cset: done
+
+

This command destroys the newly created cpuset called "super_set". When a +cpuset is destroyed, all the tasks running in it are moved to the parent +cpuset. The root cpuset, which always exists and always contains all CPUs, +can not be destroyed. You may also give the --destroy option a list of +cpusets to destroy.

+
+ + + +
+
Note
+		The cset subcommand creates the cpusets based on a mounted cpuset +filesystem. You do not need to know where that filesystem is mounted, +although it is easy to figure out (by default it’s on /cpusets). When you +give the set subcommand a name for a new cpuset, it is created wherever the +cpuset filesystem is mounted at.
+
+

If you want to create a cpuset hierarchy, then you must give a path to the +cset set subcommand. This path will always begin with the root cpuset, +for which the path is /. For example.

+
+
+
[zuul:cpuset-trunk]# cset set -c 1,3 -s top_set
+cset: --> created cpuset "top_set"
+
+[zuul:cpuset-trunk]# cset set -c 3 -s /top_set/sub_set
+cset: --> created cpuset "/top_set/sub_set"
+
+

These commands created two cpusets: top_set and sub_set. The top_set +uses CPU1 and CPU3. It has a subset of sub_set which only uses CPU3. Once +you have created a subset with a path, then if the name is unique, you do not +have to specify the path in order to affect it. If the name is not unique, +then cset will complain and ask you to use the path. For example:

+
+
+
[zuul:cpuset-trunk]# cset set -c 1,3 -s sub_set
+cset: --> modified cpuset "sub_set
+
+

This command adds CPU1 to the sub_set cpuset for it’s use. Note that using +the path in this case is optional.

+

If you attempt to destroy a cpuset which has sub-cpusets, cset will complain +and not do it unless you use the -r/--recurse and the --force options. +If you do use --force, then all the tasks running in all subsets of the +deletion target cpuset will be moved to the target’s parent cpuset and all +cpusets.

+

Moving a cpuset from under a certain cpuset to a different location is +currently not implemented and is slated for a later release of cset.

+

3.1.2. Listing Cpusets with Set

+

To list cpusets, use the set subcommand with the -l/--list option. For +example:

+
+
+
[zuul:cpuset-trunk]# cset set -l
+cset:
+         Name       CPUs-X    MEMs-X Tasks Subs Path
+ ------------ ---------- - ------- - ----- ---- ----------
+         root        0-3 y       0 y   320    1 /
+          one          3 n       0 n     0    1 /one
+
+

This shows that there is currently one cpuset present called one. (Of course +that there is also the root set, which is always present.) The output shows +that the one cpuset has no tasks running in it. The root cpuset has 320 +tasks running. The "-X" for "CPUs" and "MEMs" fields denotes whether the CPUs +and MEMs in the cpusets are marked exclusive to those cpusets. Note that the +one cpuset has subsets as indicated by a 1 in the Subs field. You can +specify a cpuset to list with the set subcommand as follows.

+
+
+
[zuul:cpuset-trunk]# cset set -l -s one
+cset:
+         Name       CPUs-X    MEMs-X Tasks Subs Path
+ ------------ ---------- - ------- - ----- ---- ----------
+          one          3 n       0 n     0    1 /one
+          two          3 n       0 n     0    1 /one/two
+
+

This output shows that there is a cpuset called two in cpuset one and it +also has subset. You can also ask for a recursive listing as follows.

+
+
+
[zuul:cpuset-trunk]# cset set -l -r
+cset:
+         Name       CPUs-X    MEMs-X Tasks Subs Path
+ ------------ ---------- - ------- - ----- ---- ----------
+         root        0-3 y       0 y   320    1 /
+          one          3 n       0 n     0    1 /one
+          two          3 n       0 n     0    1 /one/two
+        three          3 n       0 n     0    0 /one/two/three
+
+

This command lists all cpusets existing on the system since it asks for a +recursive listing beginning at the root cpuset. Incidentally, should you +need to specify the root cpuset you can use either root or / to specify it +explicitely—just remember that the root cpuset cannot be deleted or modified.

+

3.2. The Proc Subcommand

+

Now that we know how to create, rename and destroy cpusets with the set +subcommand, the next step is to manage threads and processes in those +cpusets. The subcommand to do this is called proc and it allows you to exec +processes into a cpuset, move existing tasks around existing cpusets, and list +tasks running in specified cpusets. For the following examples, let us +assume a cpuset setup of two sets as follows:

+
+
+
[zuul:cpuset-trunk]# cset set -l
+cset:
+         Name       CPUs-X    MEMs-X Tasks Subs Path
+ ------------ ---------- - ------- - ----- ---- ----------
+         root        0-3 y       0 y   309    2 /
+          two          2 n       0 n     3    0 /two
+        three          3 n       0 n    10    0 /three
+
+

3.2.1. Listing Tasks with Proc

+

Operation of the proc subcommand follows the same model as the set +subcommand. For example, to list tasks in a cpuset, you need to use the +-l/--list option and specify the cpuset by name or, if the name exists +multiple times in the cpuset hierarchy, by path. For example:

+
+
+
[zuul:cpuset-trunk]# cset proc -l -s two
+cset: "two" cpuset of CPUSPEC(2) with 3 tasks running
+ USER       PID  PPID SPPr TASK NAME
+ -------- ----- ----- ---- ---------
+ root     16141  4300 Soth bash
+ root     16171 16141 Soth bash
+ root     16703 16171 Roth python ./cset proc -l two
+
+

This output shows us that the cpuset called two has CPU2 only attached to it +and is running three tasks: two shells and the python command to list it. +Note that cpusets are inherited so that if a process is contained in a cpuset, +then any children it spawns also run within that set. In this case, the +python command to list set two was run from a shell already running in set +two. This can be seen by the PPID (parent process ID) of the python +command matching the PID of the shell.

+

Additionally, the "SPPr" field needs explanation. "SPPr" stands for State, +Policy and Priority. You can see that the initial two tasks are Stopped and +running in timeshare priority, marked as "oth" (for "other"). The last task +is marked as running, "R" and also at timeshare priority, "oth." If any of +these tasks would have been at real time priority, then the policy would be +shown as "f" for FIFO or "r" for round robin, and the priority would be a +number from 1 to 99. See below for an example.

+
+
+
[zuul:cpuset-trunk]# cset proc -l -s root | head -7
+cset: "root" cpuset of CPUSPEC(0-3) with 309 tasks running
+ USER       PID  PPID SPPr TASK NAME
+ -------- ----- ----- ---- ---------
+ root         1     0 Soth init [5]
+ root         2     0 Soth [kthreadd]
+ root         3     2 Sf99 [migration/0]
+ root         4     2 Sf99 [posix_cpu_timer]
+
+

This output shows the first few tasks in the root cpuset. Note that both +init and [kthread] are running at timeshare; however, the [migration/0] +and [posix_cpu_timer] kernel threads are running at real time policy of FIFO +and priority of 99. Incidentally, this output is from a system running the +real time Linux kernel which runs some kernel threads at real time +priorities. And finally, note that you can of course use cset as any other +Linux tool and include it in pipelines as in the example above.

+

Taking a peek into the third cpuset called three, we see:

+
+
+
[zuul:cpuset-trunk]# cset proc -l -s three
+cset: "three" cpuset of CPUSPEC(3) with 10 tasks running
+ USER       PID  PPID SPPr TASK NAME
+ -------- ----- ----- ---- ---------
+ alext    16165     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    16169     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    16170     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    16237     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    16491     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    16492     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    16493     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    17243     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    17244     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    17265     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+
+

This output shows that a lot of beagled tasks are running in this cpuset and +it also shows an ellipsis (…) at the end of their listings. If you see this +ellipsis, that means that the command was too long to fit onto an 80 character +screen. To see the entire commandline, use the -v/--verbose flag, as per +following.

+
+
+
[zuul:cpuset-trunk]# cset proc -l -s three -v | head -4
+cset: "three" cpuset of CPUSPEC(3) with 10 tasks running
+ USER       PID  PPID SPPr TASK NAME
+ -------- ----- ----- ---- ---------
+ alext    16165     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg --autostarted --indexing-delay 300
+
+

3.2.2. Execing Tasks with Proc

+

To exec a task into a cpuset, the proc subcommand needs to be employed with +the -e/--exec option. Let’s exec a shell into the cpuset named two in +our set. First we check to see what is running that set:

+
+
+
[zuul:cpuset-trunk]# cset proc -l -s two
+cset: "two" cpuset of CPUSPEC(2) with 0 tasks running
+
+[zuul:cpuset-trunk]# cset proc -s two -e bash
+cset: --> last message, executed args into cpuset "/two", new pid is: 20955
+
+[zuul:cpuset-trunk]# cset proc -l -s two
+cset: "two" cpuset of CPUSPEC(2) with 2 tasks running
+ USER       PID  PPID SPPr TASK NAME
+ -------- ----- ----- ---- ---------
+ root     20955 19253 Soth bash
+ root     20981 20955 Roth python ./cset proc -l two
+
+

You can see that initially, two had nothing running in it. After the +completion of the second command, we list two again and see that there are +two tasks running: the shell which we execed and the python cset command +that is listing the cpuset. The reason for the second task is that the cpuset +property of a running task is inherited by all its children. Since we +executed the listing command from the new shell which was bound to cpuset +two, the resulting process for the listing is also bound to cpuset two. +Let’s test that by just running a new shell with no prefixed cset command.

+
+
+
[zuul:cpuset-trunk]# bash
+
+[zuul:cpuset-trunk]# cset proc -l -s two
+cset: "two" cpuset of CPUSPEC(2) with 3 tasks running
+ USER       PID  PPID SPPr TASK NAME
+ -------- ----- ----- ---- ---------
+ root     20955 19253 Soth bash
+ root     21118 20955 Soth bash
+ root     21147 21118 Roth python ./cset proc -l two
+
+

Here again we see that the second shell, PID 21118, has a parent PID of 20955 +which is the first shell. Both shells as well as the listing command are +running in the two cpuset.

+
+ + + +
+
Note
+		cset follows the tradition of separating the tool options from the +command to be execed options with a double dash (--). This is not shown in +this simple example, but if the command you want to exec also takes options, +separate them with the double dash like so: # cset proc -s myset -e mycommand +-- -v The -v will be passed to mycommand, and not to cset.
+
+
+ + + +
+
Tip
+		Execing a shell into a cpuset is a useful way to experiment with running +tasks in that cpuset since all children of the shell will also run in the same +cpuset.
+
+

Finally, if you misspell the command to be execed, the result may be +puzzling. For example:

+
+
+
[zuul:cpuset-trunk]# cset proc -s two -e blah-blah
+cset: --> last message, executed args into cpuset "/two", new pid is: 21655
+cset: **> [Errno 2] No such file or directory
+
+

The result is no new process even though a new PID is output. The reason for +the message is of course that the cset process forked in preparation for +exec, but the command blah-blah was not found in order to exec it.

+

3.2.3. Moving Tasks with Proc

+

Although the ability to exec a task into a cpuset is fundamental, you will +most likely be moving tasks between cpusets more often. Moving tasks is +accomplished with the -m/--move and -p/--pid options to the proc +subcommand of cset. The move option tells the proc subcommand that a task +move is requested. The -p/--pid option takes an argument called a PIDSPEC +(PID Specification). The PIDSPEC defines which tasks get operated on.

+

The PIDSPEC can be a single process ID, a list of process IDs separated by +commas, and a list of process ID ranges also separated by commas. For +example:

+
+
+--move --pid 1234 +
+
+

+This PIDSPEC argument specifies that task 1234 be moved. +

+
+
+--move --pid 1234,42,1934,15000,15001,15002 +
+
+

+This PIDSPEC argument specifies that this list of tasks only be moved. +

+
+
+--move --pid 5000,5100,6010-7000,9232 +
+
+

+This PIDSPEC argument specifies that tasks 5000,5100 and 9232 be moved along +with any existing task that is in the range 6010 through 7000 inclusive. +

+
+
+
+ + + +
+
Note
+		A range in a PIDSPEC does not have to have running tasks for every +number in that range. In fact, it is not even an error if there are no tasks +running in that range; none will be moved in that case. The range simply +specifies to act on any tasks that have a PID or TID that is within that +range.
+
+

In the following example, we move the current shell into the cpuset named +two with a range PIDSPEC and back out to the root cpuset with the bash +variable for the current PID.

+
+
+
[zuul:cpuset-trunk]# cset proc -l -s two
+cset: "two" cpuset of CPUSPEC(2) with 0 tasks running
+
+[zuul:cpuset-trunk]# echo $$
+19253
+
+[zuul:cpuset-trunk]# cset proc -m -p 19250-19260 -t two
+cset: moving following pidspec: 19253
+cset: moving 1 userspace tasks to /two
+cset: done
+
+[zuul:cpuset-trunk]# cset proc -l -s two
+cset: "two" cpuset of CPUSPEC(2) with 2 tasks running
+ USER       PID  PPID SPPr TASK NAME
+ -------- ----- ----- ---- ---------
+ root     19253 16447 Roth bash
+ root     29456 19253 Roth python ./cset proc -l -s two
+
+[zuul:cpuset-trunk]# cset proc -m -p $$ -t root
+cset: moving following pidspec: 19253
+cset: moving 1 userspace tasks to /
+cset: done
+
+[zuul:cpuset-trunk]# cset proc -l -s two
+cset: "two" cpuset of CPUSPEC(2) with 0 tasks running
+
+

Use of the appropriate PIDSPEC can thus be handy to move tasks and groups of +tasks. Additionally, there is one more option that can help with +multi-threaded processes, and that is the --threads flag. If this flag is +present in a proc move command with a PIDSPEC and if any of the task IDs in +the PIDSPEC belongs to a thread in a process container, then all the sibling +threads in that process container will also get moved. This flag provides an +easy mechanism to move all threads of a process by simply specifying one +thread in that process. In the following example, we move all the threads +running in cpuset three to cpuset two by using the --threads flag.

+
+
+
[zuul:cpuset-trunk]# cset set two three
+cset:
+         Name       CPUs-X    MEMs-X Tasks Subs Path
+ ------------ ---------- - ------- - ----- ---- ----------
+          two          2 n       0 n     0    0 /two
+        three          3 n       0 n    10    0 /three
+
+[zuul:cpuset-trunk]# cset proc -l -s three
+cset: "three" cpuset of CPUSPEC(3) with 10 tasks running
+ USER       PID  PPID SPPr TASK NAME
+ -------- ----- ----- ---- ---------
+ alext    16165     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    16169     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    16170     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    16237     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    16491     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    16492     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    16493     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    17243     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    17244     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    27133     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+
+[zuul:cpuset-trunk]# cset proc -m -p 16165 --threads -t two
+cset: moving following pidspec: 16491,16493,16492,16170,16165,16169,27133,17244,17243,16237
+cset: moving 10 userspace tasks to /two
+[==================================================]%
+cset: done
+
+[zuul:cpuset-trunk]# cset set two three
+cset:
+         Name       CPUs-X    MEMs-X Tasks Subs Path
+ ------------ ---------- - ------- - ----- ---- ----------
+          two          2 n       0 n    10    0 /two
+        three          3 n       0 n     0    0 /three
+
+
Moving All Tasks from one Cpuset to Another
+

There is a special case for moving all tasks currently running in one cpuset +to another. This can be a common use case, and when you need to do it, +specifying a PIDSPEC with -p is not necessary so long as you use the +-f/--fromset and the -t/--toset options.

+

In the following example, we move all 10 beagled threads back to cpuset +three with this method.

+
+
+
[zuul:cpuset-trunk]# cset proc -l two three
+cset: "two" cpuset of CPUSPEC(2) with 10 tasks running
+ USER       PID  PPID SPPr TASK NAME
+ -------- ----- ----- ---- ---------
+ alext    16165     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    16169     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    16170     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    16237     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    16491     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    16492     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    16493     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    17243     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    17244     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+ alext    27133     1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -...
+cset: "three" cpuset of CPUSPEC(3) with 0 tasks running
+
+[zuul:cpuset-trunk]# cset proc -m -f two -t three
+cset: moving all tasks from two to /three
+cset: moving 10 userspace tasks to /three
+[==================================================]%
+cset: done
+
+[zuul:cpuset-trunk]# cset set two three
+cset:
+         Name       CPUs-X    MEMs-X Tasks Subs Path
+ ------------ ---------- - ------- - ----- ---- ----------
+          two          2 n       0 n     0    0 /two
+        three          3 n       0 n    10    0 /three
+
+
Moving Kernel Threads with Proc
+

Kernel threads are special and cset detects tasks that are kernel threads +and will refuse to move them unless you also add a -k/--kthread option to +your proc move command. Even if you include -k, cset will still +refuse to move the kernel thread if they are bound to specific CPUs. The +reason for this is system protection.

+

A number of kernel threads, especially on the real time Linux kernel, are +bound to specific CPUs and depend on per-CPU kernel variables. If you move +these threads to a different CPU than what they are bound to, you risk at best +that the system will become horribly slow, and at worst a system hang. If you +still insist to move those threads (after all, cset needs to give the +knowledgeable user access to the keys), then you need to use the --force +option additionally.

+
+ + + +
+
Warning
+		Overriding a task move command with --force can have dire +consequences for the system. Please be sure of the command before you force +it.
+
+

In the following example, we move all unbound kernel threads running in the +root cpuset to the cpuset named two by using the -k option.

+
+
+
[zuul:cpuset-trunk]# cset proc -k -f root -t two
+cset: moving all kernel threads from / to /two
+cset: moving 70 kernel threads to: /two
+cset: --> not moving 76 threads (not unbound, use --force)
+[==================================================]%
+cset: done
+
+

You will note that we used the fromset→toset facility of the proc +subcommand and we only specified the -k option (not the -m option). This +has the effect of moving all kernel threads only.

+

Note that only 70 actual kernel threads were moved and 76 were not. The +reason that 76 kernel threads were not moved was because they are bound to +specific CPUs. Now, let’s move those kernel threads back to root.

+
+
+
[zuul:cpuset-trunk]# cset proc -k -f two -t root
+cset: moving all kernel threads from /two to /
+cset: ** no task matched move criteria
+cset: **> kernel tasks are bound, use --force if ok
+
+[zuul:cpuset-trunk]# cset set -l -s two
+cset:
+         Name       CPUs-X    MEMs-X Tasks Subs Path
+ ------------ ---------- - ------- - ----- ---- ----------
+          two          2 n       0 n    70    0 /two
+
+

Ah! What’s this? Cset refused to move the kernel threads back to root +because it says that they are "bound." Let’s check this with the Linux +taskset command.

+
+
+
[zuul:cpuset-trunk]# cset proc -l -s two | head -5
+cset: "two" cpuset of CPUSPEC(2) with 70 tasks running
+ USER       PID  PPID SPPr TASK NAME
+ -------- ----- ----- ---- ---------
+ root         2     0 Soth [kthreadd]
+ root        55     2 Soth [khelper]
+
+[zuul:cpuset-trunk]# taskset -p 2
+pid 2's current affinity mask: 4
+
+[zuul:cpuset-trunk]# cset set -l -s two
+cset:
+         Name       CPUs-X    MEMs-X Tasks Subs Path
+ ------------ ---------- - ------- - ----- ---- ----------
+          two          2 n       0 n    70    0 /two
+
+

Of course, since the cpuset named two only has CPU2 assigned to it, once we +moved the unbound kernel threads from root to two, their affinity masks +got automatically changed to only use CPU2. This is evident from the +taskset output which is a hex value. To really move these threads back to +root, we need to force the move as follows.

+
+
+
[zuul:cpuset-trunk]# cset proc -k -f two -t root --force
+cset: moving all kernel threads from /two to /
+cset: moving 70 kernel threads to: /
+[==================================================]%
+cset: done
+
+

3.2.4. Destroying Tasks

+

There actually is no cset subcommand or option to destroy tasks—it’s not +really needed. Tasks exist and are accessible on the system as normal, even +if they happen to be running in one cpuset or another. To destroy tasks, use +the usual ^C method or by using the kill(1) command.

+

3.3. Implementing "Shielding" with Set and Proc

+

With the preceding material on the set and proc subcommands, we now have +the background to implement the basic shielding model, just like the shield +subcommand.

+

One may pose the question as to why we want to do this, especially since +shield already does it? The answer is that sometimes one needs more +functionality than shield has to implement one’s shielding strategy. In +those case you need to first stop using shield since that subcommand will +interfere with the further application of set and proc; however, you will +still need to implement the functionality of shield in order to implement +successful shielding.

+

Remember from the above sections describing shield, that shielding has at +minimum three cpusets: root, which is always present and contains all CPUs; +system which is the "non-shielded" set of CPUs and runs unimportant system +tasks; and user, which is the "shielded" set of CPUs and runs your important +tasks. Remember also that shield moves all movable tasks into system +and, optionally, moves unbound kernel threads into system as well.

+

We start first by creating the system and user cpusets as follows. We +assume that the machine is a four-CPU machine without NUMA memory features. +The system cpuset should hold only CPU0 while the user cpuset should hold +the rest of the CPUs.

+
+
+
[zuul:cpuset-trunk]# cset set -c 0 -s system
+cset: --> created cpuset "system"
+
+[zuul:cpuset-trunk]# cset set -c 1-3 -s user
+cset: --> created cpuset "user"
+
+[zuul:cpuset-trunk]# cset set -l
+cset:
+         Name       CPUs-X    MEMs-X Tasks Subs Path
+ ------------ ---------- - ------- - ----- ---- ----------
+         root        0-3 y       0 y   333    2 /
+         user        1-3 n       0 n     0    0 /user
+       system          0 n       0 n     0    0 /system
+
+

Now, we need to move all running user processes into the system cpuset.

+
+
+
[zuul:cpuset-trunk]# cset proc -m -f root -t system
+cset: moving all tasks from root to /system
+cset: moving 188 userspace tasks to /system
+[==================================================]%
+cset: done
+
+[zuul:cpuset-trunk]# cset set -l
+cset:
+         Name       CPUs-X    MEMs-X Tasks Subs Path
+ ------------ ---------- - ------- - ----- ---- ----------
+         root        0-3 y       0 y   146    2 /
+         user        1-3 n       0 n     0    0 /user
+       system          0 n       0 n   187    0 /system
+
+

We now have the basic shielding set up. Since all userspace tasks are running +in system, anything that is spawned from them will also run in system. +The user cpuset has nothing running in it unless you put tasks there with +the proc subcommand as described above. If you also want to move movable +kernel threads from root to system (in order to achieve a form of +"interrupt shielding" on a real time Linux kernel for example), you would +execute the following command as well.

+
+
+
[zuul:cpuset-trunk]# cset proc -k -f root -t system
+cset: moving all kernel threads from / to /system
+cset: moving 70 kernel threads to: /system
+cset: --> not moving 76 threads (not unbound, use --force)
+[==================================================]%
+cset: done
+
+[zuul:cpuset-trunk]# cset set -l
+cset:
+         Name       CPUs-X    MEMs-X Tasks Subs Path
+ ------------ ---------- - ------- - ----- ---- ----------
+         root        0-3 y       0 y    76    2 /
+         user        1-3 n       0 n     0    0 /user
+       system          0 n       0 n   257    0 /system
+
+

At this point, you have achieved the simple shielding model that the shield +subcommand provides. You can now add other cpuset definitions to expand your +shielding strategy beyond that simple model.

+

3.4. Implementing Hierarchy with Set and Proc

+

One popular extended "shielding" model is based on hierarchical cpusets, each +with diminishing numbers of CPUs. This model is used to create "priority +cpusets" that allow assignment of CPU resources to tasks based on some +arbitrary priority definition. The idea being that a higher priority task +will get access to more CPU resources than a lower priority task.

+

The example provided here once again assumes a machine with four CPUs and no +NUMA memory features. This base serves to illustrate the point well; however, +note that if your machine has (many) more CPUs, then strategies such as this +and others get more interesting.

+

We define a shielding set up as in the previous section where we have a +system cpuset with just CPU0 that takes care of "unimportant" system tasks. +One usually requires this type of cpuset since it forms the basis of +shielding. We modify the strategy to not use a user cpuset, instead we +create a number of new cpusets each holding one more CPU than the other. +These cpusets will be called prio_low with one CPU, prio_med with two +CPUs, prio_high with three CPUs, and prio_all with all CPUs.

+
+ + + +
+
Note
+		One may ask, why create a prio_all with all CPUs when that is +substantially the definition of the root cpuset? The answer is that it is +best to keep a separation between the root cpuset and everything else, even +if a particular cpuset duplicates root exactly. Usually, one builds +automation on top of a cpuset strategy. In these cases, it is best to avoid +using invariant names of cpusets, such as root for example, in this +automation.
+
+

All of these prio_* cpusets can be created under root, in a flat way; +however, it is advantageous to create them as a hierarchy. The reasoning for +this is twofold: first, if a cpuset is destroyed, all its tasks are moved to +its parent; second, one can use exclusive CPUs in a hierarchy.

+

There is a planned addition to the proc subcommand that will allow moving a +specified PIDSPEC of tasks running in a specified cpuset to its parent. This +addition will ease the automation burden.

+

If a cpuset has CPUs that are exclusive to it, then other cpusets may not make +use of those CPUs unless they are children of that cpuset. This has more +relevance to machines with many CPUs and more complex strategies.

+

Now, we start with a clean slate and build the appropriate cpusets as +follows.

+
+
+
[zuul:cpuset-trunk]# cset set -r
+cset:
+         Name       CPUs-X    MEMs-X Tasks Subs Path
+ ------------ ---------- - ------- - ----- ---- ----------
+         root        0-3 y       0 y   344    0 /
+
+[zuul:cpuset-trunk]# cset set -c 0-3 prio_all
+cset: --> created cpuset "prio_all"
+
+[zuul:cpuset-trunk]# cset set -c 1-3 /prio_all/prio_high
+cset: --> created cpuset "/prio_all/prio_high"
+
+[zuul:cpuset-trunk]# cset set -c 2-3 /prio_all/prio_high/prio_med
+cset: --> created cpuset "/prio_all/prio_high/prio_med"
+
+[zuul:cpuset-trunk]# cset set -c 3 /prio_all/prio_high/prio_med/prio_low
+cset: --> created cpuset "/prio_all/prio_high/prio_med/prio_low"
+
+[zuul:cpuset-trunk]# cset set -c 0 system
+cset: --> created cpuset "system"
+
+[zuul:cpuset-trunk]# cset set -l -r
+cset:
+         Name       CPUs-X    MEMs-X Tasks Subs Path
+ ------------ ---------- - ------- - ----- ---- ----------
+         root        0-3 y       0 y   344    2 /
+       system          0 n       0 n     0    0 /system
+     prio_all        0-3 n       0 n     0    1 /prio_all
+    prio_high        1-3 n       0 n     0    1 /prio_all/prio_high
+     prio_med        2-3 n       0 n     0    1 /prio_all/prio_high/prio_med
+     prio_low          3 n       0 n     0    0 /prio_all/pr...rio_med/prio_low
+
+
+ + + +
+
Note
+		We used the -r/--recurse switch to list all the sets in the last +command above. If we had not, then the prio_med and prio_low cpusets +would not have been listed.
+
+

The strategy is now implemented and we now move all userspace tasks as well as +all movable kernel threads into the system cpuset to activate the shield.

+
+
+
[zuul:cpuset-trunk]# cset proc -m -k -f root -t system
+cset: moving all tasks from root to /system
+cset: moving 198 userspace tasks to /system
+cset: moving 70 kernel threads to: /system
+cset: --> not moving 76 threads (not unbound, use --force)
+[==================================================]%
+cset: done
+
+[zuul:cpuset-trunk]# cset set -l -r
+cset:
+         Name       CPUs-X    MEMs-X Tasks Subs Path
+ ------------ ---------- - ------- - ----- ---- ----------
+         root        0-3 y       0 y    76    2 /
+       system          0 n       0 n   268    0 /system
+     prio_all        0-3 n       0 n     0    1 /prio_all
+    prio_high        1-3 n       0 n     0    1 /prio_all/prio_high
+     prio_med        2-3 n       0 n     0    1 /prio_all/prio_high/prio_med
+     prio_low          3 n       0 n     0    0 /prio_all/pr...rio_med/prio_low
+
+

The shield is now active. Since the prio_* cpuset names are unique, one can +assign tasks to them either via either their simple name, or their full path +(as described in the proc section above).

+

You may have noted that there is an ellipsis in the path of the prio_low +cpuset in the listing above. This is done in order to fit the output onto an +80 character screen. If you want to see the entire line, then you need to use +the -v/--verbose flag as follows.

+
+
+
[zuul:cpuset-trunk]# cset set -l -r -v
+cset:
+         Name       CPUs-X    MEMs-X Tasks Subs Path
+ ------------ ---------- - ------- - ----- ---- ----------
+         root        0-3 y       0 y    76    2 /
+       system          0 n       0 n   268    0 /system
+     prio_all        0-3 n       0 n     0    1 /prio_all
+    prio_high        1-3 n       0 n     0    1 /prio_all/prio_high
+     prio_med        2-3 n       0 n     0    1 /prio_all/prio_high/prio_med
+     prio_low          3 n       0 n     0    0 /prio_all/prio_high/prio_med/prio_low
+
+
+

4. Using Shortcuts

+
+

The commands listed in the previous sections always used all the required +options. Cset however does have a shortcut facility that will execute +certain commands without specifying all options. An effort has been made to +do this with the "principle of least surprise." This means that if you do not +specify options, but do specify parameters, then the outcome of the command +should be intuitive. As much as possible.

+

Using shortcuts is of course not necessary. In fact, you can not only not use +shortcuts, but you can use long options instead of short, in case you really +enjoy typing… All kidding aside, using long options and not using shortcuts +does have a use case: when you write a script intended to be self-documenting, +or perhaps when you generate cset documentation.

+

To begin, the subcommands shield, set and proc can themselves be +shortened to the fewest number of characters that are unambiguous. For +example, the following commands are identical:

+
+
+
# cset shield -s -p 1234         <-->    # cset sh -s -p 1234
+# cset set -c 1,3 -s newset      <-->    # cset se -c 1,3 -s newset
+# cset proc -s newset -e bash    <-->    # cset p -s newset -e bash
+
+

Note that proc can be shortened to just p, while shield and set need +two letters to disambiguate.

+

4.1. Shield Subcommand Shortcuts

+

The shield subcommand supports two areas with shortcuts: the case when there +is no options given where to shield is the common use case, and making the +-p/--pid option optional for the -s/--shield and -u/--unshield +options.

+

For the common use case of actually shielding either a PIDSPEC or execing a +command into the shield, the following cset commands are equivalent.

+
+
+
# cset shield -s -p 1234,500-649   <-->    # cset sh 1234,500-649
+# cset shield -s -e bash           <-->    # cset sh bash
+
+

When using the -s or -u shield/unshield options, it is optional to use the +-p option to specify a PIDSPEC. For example:

+
+
+
# cset shield -s -p 1234     <-->    # cset sh -s 1234
+# cset shield -u -p 1234     <-->    # cset sh -u 1234
+
+

4.2. Set Subcommand Shortcuts

+

The set subcommand has a limited number of shortcuts. Basically, the +-s/--set option is optional in most cases and the -l/--list option is +also optional if you want to list sets. For example, these commands are +equivalent.

+
+
+
# cset set -l -s myset           <-->  # cset se -l myset
+# cset se -l myset               <-->  # cset se myset
+
+# cset set -c 1,2,3 -s newset    <-->  # cset se -c 1,2,3 newset
+# cset set -d -s newset          <-->  # cset se -d newset
+
+# cset set -n newname -s oldname <-->  # cset se -n newname oldname
+
+

In fact, if you want to apply either the list or the destroy options to +multiple cpusets with one cset command, you’ll need to not use the -s +option. For example:

+
+
+
# cset se -d myset yourset ourset
+   --> destroys cpusets: myset, yourset and ourset
+
+# cset se -l prio_high prio_med prio_low
+   --> lists only cpusets prio_high, prio_med and prio_low
+   --> the -l is optional in this case since list is default
+
+

4.3. Proc Subcommand Shortcuts

+

For the proc subcommand, the -s, -t and -f options to specify the +cpuset, the origination cpuset and the destination cpuset, can sometimes be +optional. For example, the following commands are equivalent.

+
+
+
To list tasks in cpusets:
+# cset proc -l -s myset        \
+# cset proc -l -f myset         -->  # cset p -l myset
+# cset proc -l -t myset        /
+
+# cset p -l myset              <-->  # cset p myset
+
+# cset proc -l -s one two      <-->  # cset p -l one two
+# cset p -l one two            <--> # cset p one two
+
+To exec a process into a cpuset:
+# cset proc -s myset -e bash   <-->  # cset p myset -e bash
+
+

Movement of tasks into and out of cpusets have the following shortcuts.

+
+
+
To move a PIDSPEC into a cpuset:
+# cset proc -m -p 4242,4243 -s myset <--> # cset p -m 4242,4243 myset
+# cset proc -m -p 12 -t myset        <--> # cset p -m 12 myset
+
+To move all tasks from one cpuset to another:
+# cset proc -m -f set1 -t set2      \
+# cset proc -m -s set1 -t set2       --> # cset p -m set1 set2
+# cset proc -m -f set1 -s set2      /
+
+
+

5. What To Do if There are Problems

+
+

If you encounter problems with the cset application, the best option is to +log a bug with the cset bugzilla instance found here:

+
+
+
http://devzilla.novell.com/cpuset
+
+

If you are using cset on a supported operating system such as SLES or SLERT +from Novell, then feel free to use that bugzilla instead.

+

If the problem is repeatable, there is an excellent chance that it will get +fixed quickly. Also, cset contains a logging facility that is invaluable +for the developers to diagnose problems. To create a log of a run, use the +-l/--log option with a filename as an argument to the main cset +application. For example.

+
+
+
# cset -l logfile.txt set -n newname oldname
+
+

That command saves a lot of debugging information in the logfile.txt file. +Please attach this file to the bug.

+
+ + + diff --git a/doc/tutorial.txt b/doc/tutorial.txt index 2a39ddf..ec1ce2c 100644 --- a/doc/tutorial.txt +++ b/doc/tutorial.txt @@ -1,51 +1,1348 @@ -Cpuset Tutorial -############### +Cpuset (cset) Tutorial +###################### +Alex Tsariounov + +Copyright (c) 2009 Novell Inc., cset v1.5.0 + +Verbatim copying and distribution of this entire article are permitted +worldwide, without royalty, in any medium, provided this notice is preserved. -Cpuset (cset) is a Python application that provides a command line -application front end for the cpusets functionality found in the Linux -kernel. +This tutorial describes basic and advanced usage of the *cset* command to +manipulate cpusets on a Linux system. See also the manpages that come with +the *cset* command: cset(1), cset-shield(1), cset-set(1), and cset-proc(1) for +more details. Additionally, the *cset* command has online help. -Basic Operation -=============== +Introduction +============ +In the Linux kernel, the cpuset facility provides a mechanism for creating +logical entities called "cpusets" that encompass definitions of CPUs and NUMA +Memory Nodes (if NUMA is available). Cpusets constrain the CPU and Memory +placement of a task to only the resources defined within that cpuset. These +cpusets can then be arranged into a nested hierarchy visible in the "cpuset" +virtual filesystem. Sets of tasks can be assigned to these cpusets to +constrain the resources that they use. The tasks can be moved from one cpuset +to another to utilize other resources defined in those other cpusets. + +The *cset* command is a Python application that provides a command line front +end for the Linux cpusets functionality. Working with cpusets directly can be +confusing and slightly complex. The *cset* tool hides that complexity behind +an easy-to-use command line interface. + +There are two distinct use cases for *cset*: the basic shielding use case and +the "advanced" case of using raw +set+ and +proc+ subcommands. The basic +shielding function is accessed with the +shield+ subcommand and described in +the next section. Using the raw +set+ and +proc+ subcommands allows one to +set up arbitrarily complex cpusets and is described in the later sections. + +Note that in general, one either uses the +shield+ subcommand 'or' a +combination of the +set+ and +proc+ subcommands. One rarely, if ever, uses +all of these subcommands together. Doing so will likely become too +confusing. Additionally, the +shield+ subcommand sets up its required cpusets +with exclusively marked CPUs. This can interfere with your cpuset +strategy. If you find that you need more functionality for your strategy than ++shield+ provides, go ahead and transition to using +set+ and +proc+ +exclusively. It is straightforward to implement what +shield+ does with a few +extra +set+ and +proc+ subcommands. + + +Obtaining Online Help +--------------------- +For a full list of *cset* subcommands:: + +# cset help+ + +For in-depth help on individual subcommands:: + +# cset help + + +For options of individual subcommands:: + +# cset (-h | --help)+ + + +The Basic Shielding Model +========================= +Although any set up of cpusets can really be described as "shielding," there +is one prevalent shielding model in use that is so common that *cset* has a +subcommand that is dedicated to its use. This subcommand is called +shield+. + +The concept behind this model is the use of three cpusets. The 'root' cpuset +which is always present in all configurations and contains all CPUs. The +'system' cpuset which contains CPUs which are used for system tasks. These +are the normal tasks that are not "important," but which need to run on the +system. And finally, the 'user' cpuset which contains CPUs which are used for +"important" tasks. The 'user' cpuset is the shield. Only those tasks that +are somehow important, usually tasks whose performance determines the overall +rating for the machine, are run in the 'user' cpuset. + +The +shield+ subcommand manages all of these cpusets and lets you define the +CPUs and Memory Nodes that are in the 'shielded' and 'unshielded' sets. The +subcommand automatically moves all movable tasks on the system into the +'unshielded' cpuset on shield activation, and back into the 'root' cpuset on +shield tear down. The subcommand then lets you move tasks into and out of the +shield. Additionally, you can move special tasks (kernel threads) which +normally run in the 'root' cpuset into the 'unshielded' set so that your +shield will have even less disturbance. + +The +shield+ subcommand abstracts the management of these cpusets away from +you and provides options that drive how the shield is set up, which tasks are +to be shielded and which tasks are not, and status of the shield. In fact, +you need not be bothered with the naming of the required cpusets or even where +the cpuset filesystem is mounted. *Cset* and the +shield+ subcommand takes +care of all that. + +If you find yourself needing to define more cpusets for your application, then +it is likely that this simple shielding is not a rich enough model for you. +In this case, you should transition to using the +set+ and +proc+ subcommands +described in a later section. + + +A Simple Shielding Example +-------------------------- +Assume that we have a 4-way machine that is not NUMA. This means there are 4 +CPUs at our disposal and there is only one Memory Node available. On such +machines, we do not need to specify any memory node parameters to *cset*, it +sets up the only available memory node by default. + +Usually, one wants to dedicate as many CPUs to the shield as possible and +leave a minimal set of CPUs for normal system processing. The reasoning for +this is because the performance of the important tasks will rule the +performance of the installation as a whole and these important tasks need as +many resources available to them as possible, exclusive of other, unimportant +tasks that are running on the system. + +NOTE: I use the word "task" to represent either a process or a thread that is +running on the system. + + +Setup and Teardown of the Shield +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +To set up a shield of 3 CPUs with 1 CPU left for low priority system +processing, issue the following command. + +------------------------------------------------------------------- +[zuul:cpuset-trunk]# cset shield -c 1-3 +cset: --> activating shielding: +cset: moving 176 tasks from root into system cpuset... +[==================================================]% +cset: "system" cpuset of CPUSPEC(0) with 176 tasks running +cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running +------------------------------------------------------------------- + +This command does a number of things. First, a 'user' cpuset is created with +what's called a CPUSPEC (CPU specification) from the +-c/\--cpu+ option. This +CPUSPEC specifies to use CPUs 1 through 3 inclusively. Next, the command +creates a 'system' cpuset with a CPUSPEC that is the inverse of the +-c+ +option for the current machine. On this machine that cpuset will only contain +the first CPU, CPU0. Next, all userspace processes running in the 'root' +cpuset are transfered to the 'system' cpuset. This makes all those processes +run only on CPU0. The effect of this is that the shield consists of CPUs 1 +through 3 and they are now idling. + +Note that the command did not move the kernel threads that are running in the +'root' cpuset to the 'system' cpuset. This is because you may want these +kernel threads to use all available CPUs. If you do not, the you can use the ++-k/\--kthread+ option as described below. + +The shield setup command above outputs the information of which cpusets were +created and how many tasks are running on each. If you want to see the +current status of the shield again, issue this command: + +------------------------------------------------------------------- +[zuul:cpuset-trunk]# cset shield +cset: --> shielding system active with +cset: "system" cpuset of CPUSPEC(0) with 176 tasks running +cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running +------------------------------------------------------------------- + +Which shows us that the shield is set up and that 176 tasks are running in the +'system' cpuset--the "unshielded" cpuset. + +It is important to move all possible tasks from the 'root' cpuset to the +unshielded 'system' cpuset because a task's cpuset property is inherited by +its children. Since we've moved all running tasks (including init) to the +unshielded 'system' cpuset, that means that any new tasks that are spawned +will *also* run in the unshielded 'system' cpuset. + +Some kernel threads can be moved into the unshielded 'system' cpuset as well. +These are the threads that are not bound to specific CPUs. If a kernel thread +is bound to a specific CPU, then it is generally not a good idea to move that +thread to the 'system' set because at worst it may hang the system and at best +it will slow the system down significantly. These threads are usually the IRQ +threads on a real time Linux kernel, for example, and you may want to not move +these kernel threads into 'system'. If you leave them in the 'root' cpuset, +then they will have access to all CPUs. + +However, if your application demands an even "quieter" shield, then you can +move all movable kernel threads into the unshielded 'system' set with the +following command. + +-------------------------------------------------------------------- +[zuul:cpuset-trunk]# cset shield -k on +cset: --> activating kthread shielding +cset: kthread shield activated, moving 70 tasks into system cpuset... +[==================================================]% +cset: done +-------------------------------------------------------------------- + +You can see that this moved an additional 70 tasks to the unshielded 'system' +cpuset. Note that the +-k/\--kthread on+ parameter can be given at the shield +creation time as well and you do not need to perform these two steps +separately if you know that you will want kernel thread shielding as well. +Executing *cset shield* again shows us the current state of the shield. + +-------------------------------------------------------------------- +[zuul:cpuset-trunk]# cset shield +cset: --> shielding system active with +cset: "system" cpuset of CPUSPEC(0) with 246 tasks running +cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running +-------------------------------------------------------------------- + +You can get a detailed listing of what is running in the shield by specifying +either +-s/\--shield+ or +-u/\--unshield+ to the +shield+ subcommand and using +the verbose flag. You will get output similar to the following. + +-------------------------------------------------------------------- +[zuul:cpuset-trunk]# cset shield --unshield -v +cset: "system" cpuset of CPUSPEC(0) with 251 tasks running + USER PID PPID SPPr TASK NAME + -------- ----- ----- ---- --------- + root 1 0 Soth init [5] + root 2 0 Soth [kthreadd] + root 84 2 Sf50 [IRQ-9] +... + alext 31796 31789 Soth less + root 32653 25222 Roth python ./cset shield --unshield -v +-------------------------------------------------------------------- + +Note that I abbreviated the listing; we do have 251 tasks running in the +'system' set. The output is self-explanatory; however, the "SPPr" field may +need a little explanation. "SPPr" stands for State, Policy and Priority. You +can see that the initial two tasks are Stopped and running in timeshare +priority, marked as "oth" (for "other"). The [IRQ-9] task is also stopped, +but marked at real time FIFO policy with a priority of 50. The last task in +the listing is the *cset* command itself and is marked as running. Also note +that adding a second +-v/\--verbose+ option will not restrict the output to +fit into an 80 character screen. + +Tear down of the shield, stopping the shield in other words, is done with the ++-r/\--reset+ option to the +shield+ subcommand. When this command is issued, +both the 'system' and 'user' cpusets are deleted and any tasks that are +running in both of those cpusets are moved to the 'root' cpuset. Once so +moved, all tasks will have access to all resources on the system. For +example: + +-------------------------------------------------------------------- +[zuul:cpuset-trunk]# cset shield --reset +cset: --> deactivating/reseting shielding +cset: moving 0 tasks from "/user" user set to root set... +cset: moving 250 tasks from "/system" system set to root set... +[==================================================]% +cset: deleting "/user" and "/system" sets +cset: done +-------------------------------------------------------------------- + + +Moving Interesting Tasks Into and Out of the Shield +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +Now that we have a shield running, the objective is to run our "important" +processes in that shield. These processes can be anything, but usually they +are directly related to the purpose of the machine. There are two ways to run +tasks in the shield: + + . Exec a process into the shield + . Move an already running task into the shield + + +Execing a Process into the Shield +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Running a new process in the shield can be done with the +-e/\--exec+ option +to the +shield+ subcommand. This is the simplest way to get a task to run in +the shield. For this example, let's exec a new bash shell into the shield +with the following commands. + +-------------------------------------------------------------------- +[zuul:cpuset-trunk]# cset shield -s +cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running +cset: done + +[zuul:cpuset-trunk]# cset shield -e bash +cset: --> last message, executed args into cpuset "/user", new pid is: 13300 + +[zuul:cpuset-trunk]# cset shield -s -v +cset: "user" cpuset of CPUSPEC(1-3) with 2 tasks running + USER PID PPID SPPr TASK NAME + -------- ----- ----- ---- --------- + root 13300 8583 Soth bash + root 13329 13300 Roth python ./cset shield -s -v + +[zuul:cpuset-trunk]# exit + +[zuul:cpuset-trunk]# cset shield -s +cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running +cset: done +-------------------------------------------------------------------- + +The first command above lists the status of the shield. We see that the +shield is defined as CPUs 1 through 3 inclusive and currently there are no +tasks running in it. + +The second command execs the bash shell into the shield with the +-e+ +option. The last message of cset lists the PID of the new process. + +NOTE: *cset* follows the tradition of separating the tool options from the +command to be execed options with a double dash (+\--+). This is not shown in +this simple example, but if the command you want to exec also takes options, +separate them with the double dash like so: +# cset shield -e mycommand \-- -v+ +The +-v+ will be passed to +mycommand+, and not to *cset*. + +The next command lists the status of the shield again. You will note that +there are actually two tasks running shielded: our new shell and the *cset* +status command itself. Remember that the cpuset property of a task is +inherited by its children. Since we ran the new shell in the shield, its +child, which is the status command, also ran in the shield. + +TIP: Execing a shell into the shield is a useful way to experiment with +running tasks in the shield since all children of the shell will also run in +the shield. + +The last command exits the shell after which we request a shield status again +and see that once again, it does not contain any tasks. + +You may have noticed in the output above that both the new shell and the +status command are running as the root user. This is because *cset* needs to +run as root and so all it's children will also run as root. If you need to +run a process under a different user and or group, you may use the +\--user+ +and +\--group+ options for +exec+ as follows. + +-------------------------------------------------------------------- +[zuul:cpuset-trunk]# cset shield --user=alext --group=users -e bash +cset: --> last message, executed args into cpuset "/user", new pid is: 14212 + +alext@zuul> cset shield -s -v +cset: "user" cpuset of CPUSPEC(1-3) with 2 tasks running + USER PID PPID SPPr TASK NAME + -------- ----- ----- ---- --------- + alext 14212 8583 Soth bash + alext 14241 14212 Roth python ./cset shield -s -v +-------------------------------------------------------------------- + + +Moving a Running Task into and out of the Shield +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +While execing a process into the shield is undoubtably useful, most of the +time, you'll want to move already running tasks into and out of the shield. +The *cset* shield subcommand includes two options for doing this: ++-s/\--shield+ and +-u/\--unshield+. These options require what's called a +PIDSPEC (process specification) to also be specified with the +-p/\--pid+ +option. The PIDSPEC defines which tasks get operated on. The PIDSPEC can be +a single process ID, a list of process IDs separated by commas, and a list of +process ID ranges separated by dashes, groups of which are separated by +commas. For example: + ++\--shield --pid 1234+:: +This PIDSPEC argument specifies that PID 1234 be shielded. + ++\--shield --pid 1234,42,1934,15000,15001,15002+:: +This PIDSPEC argument specifies that this list of PIDs only be moved into the +shield. + ++\--unshield -p 5000,5100,6010-7000,9232+:: +This PIDSPEC argument specifies that PIDs 5000,5100 and 9232 be unshielded +(moved out of the shield) along with any existing PID that is in the range +6010 through 7000 inclusive. + +NOTE: A range in a PIDSPEC does not have to have tasks running for every +number in that range. In fact, it is not even an error if there are no tasks +running in that range; none will be moved in that case. The range simply +specifies to act on any tasks that have a PID or TID that is within that +range. + +Use of the appropriate PIDSPEC can thus be handy to move tasks and groups of +tasks into and out of the shield. Additionally, there is one more option that +can help with multi-threaded processes, and that is the +\--threads+ flag. If +this flag is present in a +shield+ or +unshield+ command with a PIDSPEC and if +any of the task IDs in the PIDSPEC belong to a thread in a process container, +then *all* the sibling threads in that process container will get shielded or +unshielded as well. This flag provides an easy mechanism to shield/unshield +all threads of a process by simply specifying one thread in that process. + +In the following example, we move the current shell into the shield with a +range PIDSPEC and back out with the bash variable for the current PID. + +----------------------------------------------------------------------- +[zuul:cpuset-trunk]# echo $$ +22018 + +[zuul:cpuset-trunk]# cset shield -s -p 22010-22020 +cset: --> shielding following pidspec: 22010-22020 +cset: done + +[zuul:cpuset-trunk]# cset shield -s -v +cset: "user" cpuset of CPUSPEC(1-3) with 2 tasks running + USER PID PPID SPPr TASK NAME + -------- ----- ----- ---- --------- + root 3770 22018 Roth python ./cset shield -s -v + root 22018 5034 Soth bash +cset: done + +[zuul:cpuset-trunk]# cset shield -u -p $$ +cset: --> unshielding following pidspec: 22018 +cset: done + +[zuul:cpuset-trunk]# cset shield -s +cset: "user" cpuset of CPUSPEC(1-3) with 0 tasks running +cset: done +----------------------------------------------------------------------- + +NOTE: Ordinarily, the +shield+ option will shield a PIDSPEC only if it is +currently running in the 'system' set--the unshielded set. The +unshield+ +option will unshield a PIDSPEC only if it is currently running in the 'user' +set--the shielded set. If you want to +shield/unshield+ a process that +happens to be running in the 'root' set (not common), then use the +\--force+ +option for these commands. + + +Full Featured Cpuset Manipulation Commands +========================================== +While basic shielding as described above is useful and a common use model for +*cset*, there comes a time when more functionality will be desired to +implement your strategy. To implement this, *cset* provides two subcommands: ++set+, which allows you to manipulate cpusets; and +proc+, which allows you to +manipulate processes within those cpusets. + + +The Set Subcommand +------------------ +In order to do anything with cpusets, you must be able to create, adjust, +rename, move and destroy them. The +set+ subcommand allows the management of +cpusets in such a manner. + + +Creating and Destroying Cpusets with Set +~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ +The basic syntax of +set+ for cpuset creation is: + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset set -c 1-3 -s my_cpuset1 +cset: --> created cpuset "my_cpuset1" +---------------------------------------------------------- + +This creates a cpuset named "my_cpuset1" with a CPUSPEC of CPU1, CPU2 and +CPU3. The CPUSPEC is the same concept as described in the "Setup and +Teardown of the Shield" section above. The +set+ subcommand also takes a ++-m/\--mem+ option that lets you specify the memory nodes the set will use as +well as flags to make the CPUs and MEMs exclusive to the cpuset. If you are +on a non-NUMA machine, just leave the +-m+ option out and the default memory +node 0 will be used. + +Just like with +shield+, you can adjust the CPUs and MEMs with subsequent +calls to +set+. If, for example, you wish to adjust the "my_cpuset1" cpuset +to only use CPUs 1 and 3 (and omit CPU2), then issue the following command. + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset set -c 1,3 -s my_cpuset1 +cset: --> modified cpuset "my_cpuset +---------------------------------------------------------- + +*cset* will then adjust the CPUs that are assigned to the "my_cpuset1" set to + only use CPU1 and CPU3. + +To rename a cpuset, use the +-n/\--newname+ option. For example: + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset set -s my_cpuset1 -n super_set +cset: --> renaming "/cpusets/my_cpuset1" to "super_set" +---------------------------------------------------------- + +Renames the cpuset called "my_cpuset1" to "super_set". + +To destroy a cpuset, use the +-d/\--destroy+ option as follows. + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset set -d super_set +cset: --> processing cpuset "super_set", moving 0 tasks to parent "/"... +cset: --> deleting cpuset "/super_set" +cset: done +---------------------------------------------------------- + +This command destroys the newly created cpuset called "super_set". When a +cpuset is destroyed, all the tasks running in it are moved to the parent +cpuset. The 'root' cpuset, which always exists and always contains all CPUs, +can not be destroyed. You may also give the +\--destroy+ option a list of +cpusets to destroy. + +NOTE: The *cset* subcommand creates the cpusets based on a mounted cpuset +filesystem. You do not need to know where that filesystem is mounted, +although it is easy to figure out (by default it's on '/cpusets'). When you +give the +set+ subcommand a name for a new cpuset, it is created wherever the +cpuset filesystem is mounted at. + +If you want to create a cpuset hierarchy, then you must give a path to the +*cset* +set+ subcommand. This path will always begin with the 'root' cpuset, +for which the path is '/'. For example. + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset set -c 1,3 -s top_set +cset: --> created cpuset "top_set" + +[zuul:cpuset-trunk]# cset set -c 3 -s /top_set/sub_set +cset: --> created cpuset "/top_set/sub_set" +---------------------------------------------------------- + +These commands created two cpusets: 'top_set' and 'sub_set'. The 'top_set' +uses CPU1 and CPU3. It has a subset of 'sub_set' which only uses CPU3. Once +you have created a subset with a path, then if the name is unique, you do not +have to specify the path in order to affect it. If the name is not unique, +then *cset* will complain and ask you to use the path. For example: + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset set -c 1,3 -s sub_set +cset: --> modified cpuset "sub_set +---------------------------------------------------------- + +This command adds CPU1 to the 'sub_set' cpuset for it's use. Note that using +the path in this case is optional. + +If you attempt to destroy a cpuset which has sub-cpusets, *cset* will complain +and not do it unless you use the +-r/\--recurse+ and the +\--force+ options. +If you do use +\--force+, then all the tasks running in all subsets of the +deletion target cpuset will be moved to the target's parent cpuset and all +cpusets. + +Moving a cpuset from under a certain cpuset to a different location is +currently not implemented and is slated for a later release of *cset*. + + +Listing Cpusets with Set +~~~~~~~~~~~~~~~~~~~~~~~~ +To list cpusets, use the +set+ subcommand with the '-l/\--list' option. For +example: + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset set -l +cset: + Name CPUs-X MEMs-X Tasks Subs Path + ------------ ---------- - ------- - ----- ---- ---------- + root 0-3 y 0 y 320 1 / + one 3 n 0 n 0 1 /one +---------------------------------------------------------- + +This shows that there is currently one cpuset present called 'one'. (Of course +that there is also the 'root' set, which is always present.) The output shows +that the 'one' cpuset has no tasks running in it. The 'root' cpuset has 320 +tasks running. The "-X" for "CPUs" and "MEMs" fields denotes whether the CPUs +and MEMs in the cpusets are marked exclusive to those cpusets. Note that the +'one' cpuset has subsets as indicated by a 1 in the 'Subs' field. You can +specify a cpuset to list with the +set+ subcommand as follows. + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset set -l -s one +cset: + Name CPUs-X MEMs-X Tasks Subs Path + ------------ ---------- - ------- - ----- ---- ---------- + one 3 n 0 n 0 1 /one + two 3 n 0 n 0 1 /one/two +---------------------------------------------------------- + +This output shows that there is a cpuset called 'two' in cpuset 'one' and it +also has subset. You can also ask for a recursive listing as follows. + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset set -l -r +cset: + Name CPUs-X MEMs-X Tasks Subs Path + ------------ ---------- - ------- - ----- ---- ---------- + root 0-3 y 0 y 320 1 / + one 3 n 0 n 0 1 /one + two 3 n 0 n 0 1 /one/two + three 3 n 0 n 0 0 /one/two/three +---------------------------------------------------------- + +This command lists all cpusets existing on the system since it asks for a +recursive listing beginning at the 'root' cpuset. Incidentally, should you +need to specify the 'root' cpuset you can use either +root+ or +/+ to specify it +explicitely--just remember that the 'root' cpuset cannot be deleted or modified. + + +The Proc Subcommand +------------------- +Now that we know how to create, rename and destroy cpusets with the +set+ +subcommand, the next step is to manage threads and processes in those +cpusets. The subcommand to do this is called +proc+ and it allows you to exec +processes into a cpuset, move existing tasks around existing cpusets, and list +tasks running in specified cpusets. For the following examples, let us +assume a cpuset setup of two sets as follows: + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset set -l +cset: + Name CPUs-X MEMs-X Tasks Subs Path + ------------ ---------- - ------- - ----- ---- ---------- + root 0-3 y 0 y 309 2 / + two 2 n 0 n 3 0 /two + three 3 n 0 n 10 0 /three +---------------------------------------------------------- + +Listing Tasks with Proc +~~~~~~~~~~~~~~~~~~~~~~~ +Operation of the +proc+ subcommand follows the same model as the +set+ +subcommand. For example, to list tasks in a cpuset, you need to use the ++-l/\--list+ option and specify the cpuset by name or, if the name exists +multiple times in the cpuset hierarchy, by path. For example: + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset proc -l -s two +cset: "two" cpuset of CPUSPEC(2) with 3 tasks running + USER PID PPID SPPr TASK NAME + -------- ----- ----- ---- --------- + root 16141 4300 Soth bash + root 16171 16141 Soth bash + root 16703 16171 Roth python ./cset proc -l two +---------------------------------------------------------- + +This output shows us that the cpuset called 'two' has CPU2 only attached to it +and is running three tasks: two shells and the python command to list it. +Note that cpusets are inherited so that if a process is contained in a cpuset, +then any children it spawns also run within that set. In this case, the +python command to list set 'two' was run from a shell already running in set +'two'. This can be seen by the PPID (parent process ID) of the python +command matching the PID of the shell. + +Additionally, the "SPPr" field needs explanation. "SPPr" stands for State, +Policy and Priority. You can see that the initial two tasks are Stopped and +running in timeshare priority, marked as "oth" (for "other"). The last task +is marked as running, "R" and also at timeshare priority, "oth." If any of +these tasks would have been at real time priority, then the policy would be +shown as "f" for FIFO or "r" for round robin, and the priority would be a +number from 1 to 99. See below for an example. + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset proc -l -s root | head -7 +cset: "root" cpuset of CPUSPEC(0-3) with 309 tasks running + USER PID PPID SPPr TASK NAME + -------- ----- ----- ---- --------- + root 1 0 Soth init [5] + root 2 0 Soth [kthreadd] + root 3 2 Sf99 [migration/0] + root 4 2 Sf99 [posix_cpu_timer] +---------------------------------------------------------- + +This output shows the first few tasks in the 'root' cpuset. Note that both +'init' and '[kthread]' are running at timeshare; however, the '[migration/0]' +and '[posix_cpu_timer]' kernel threads are running at real time policy of FIFO +and priority of 99. Incidentally, this output is from a system running the +real time Linux kernel which runs some kernel threads at real time +priorities. And finally, note that you can of course use *cset* as any other +Linux tool and include it in pipelines as in the example above. + +Taking a peek into the third cpuset called 'three', we see: + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset proc -l -s three +cset: "three" cpuset of CPUSPEC(3) with 10 tasks running + USER PID PPID SPPr TASK NAME + -------- ----- ----- ---- --------- + alext 16165 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 16169 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 16170 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 16237 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 16491 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 16492 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 16493 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 17243 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 17244 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 17265 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... +---------------------------------------------------------- -Help ----- +This output shows that a lot of 'beagled' tasks are running in this cpuset and +it also shows an ellipsis (...) at the end of their listings. If you see this +ellipsis, that means that the command was too long to fit onto an 80 character +screen. To see the entire commandline, use the +-v/\--verbose+ flag, as per +following. -For a full list of cset commands: +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset proc -l -s three -v | head -4 +cset: "three" cpuset of CPUSPEC(3) with 10 tasks running + USER PID PPID SPPr TASK NAME + -------- ----- ----- ---- --------- + alext 16165 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg --autostarted --indexing-delay 300 +---------------------------------------------------------- - cset help -For help on individual subcommands: +Execing Tasks with Proc +~~~~~~~~~~~~~~~~~~~~~~~ +To exec a task into a cpuset, the +proc+ subcommand needs to be employed with +the +-e/\--exec+ option. Let's exec a shell into the cpuset named 'two' in +our set. First we check to see what is running that set: - cset help +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset proc -l -s two +cset: "two" cpuset of CPUSPEC(2) with 0 tasks running -For options of individual subcommands: +[zuul:cpuset-trunk]# cset proc -s two -e bash +cset: --> last message, executed args into cpuset "/two", new pid is: 20955 - cset (-h | --help) +[zuul:cpuset-trunk]# cset proc -l -s two +cset: "two" cpuset of CPUSPEC(2) with 2 tasks running + USER PID PPID SPPr TASK NAME + -------- ----- ----- ---- --------- + root 20955 19253 Soth bash + root 20981 20955 Roth python ./cset proc -l two +---------------------------------------------------------- -Built-in Super Commands ------------------------ +You can see that initially, 'two' had nothing running in it. After the +completion of the second command, we list 'two' again and see that there are +two tasks running: the shell which we execed and the python *cset* command +that is listing the cpuset. The reason for the second task is that the cpuset +property of a running task is inherited by all its children. Since we +executed the listing command from the new shell which was bound to cpuset +'two', the resulting process for the listing is also bound to cpuset 'two'. +Let's test that by just running a new shell with no prefixed *cset* command. -Cpuset has some built-in "super commands" that encapsulate common use -cases of cpusets and make their interface simple to use. +---------------------------------------------------------- +[zuul:cpuset-trunk]# bash -cset shield -~~~~~~~~~~~ +[zuul:cpuset-trunk]# cset proc -l -s two +cset: "two" cpuset of CPUSPEC(2) with 3 tasks running + USER PID PPID SPPr TASK NAME + -------- ----- ----- ---- --------- + root 20955 19253 Soth bash + root 21118 20955 Soth bash + root 21147 21118 Roth python ./cset proc -l two +---------------------------------------------------------- -This super command sets up a shielded set of cpus environment. +Here again we see that the second shell, PID 21118, has a parent PID of 20955 +which is the first shell. Both shells as well as the listing command are +running in the 'two' cpuset. -Basic cpuset Manipulation Commands ----------------------------------- +NOTE: *cset* follows the tradition of separating the tool options from the +command to be execed options with a double dash (+\--+). This is not shown in +this simple example, but if the command you want to exec also takes options, +separate them with the double dash like so: +# cset proc -s myset -e mycommand +\-- -v+ The +-v+ will be passed to +mycommand+, and not to *cset*. +TIP: Execing a shell into a cpuset is a useful way to experiment with running +tasks in that cpuset since all children of the shell will also run in the same +cpuset. -Advanced Usage -============== +Finally, if you misspell the command to be execed, the result may be +puzzling. For example: -This describes more advanced usage of cpusets. +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset proc -s two -e blah-blah +cset: --> last message, executed args into cpuset "/two", new pid is: 21655 +cset: **> [Errno 2] No such file or directory +---------------------------------------------------------- -Priority Realtime CPU Sets +The result is no new process even though a new PID is output. The reason for +the message is of course that the *cset* process forked in preparation for +exec, but the command +blah-blah+ was not found in order to exec it. + + +Moving Tasks with Proc +~~~~~~~~~~~~~~~~~~~~~~ +Although the ability to exec a task into a cpuset is fundamental, you will +most likely be moving tasks between cpusets more often. Moving tasks is +accomplished with the +-m/\--move+ and +-p/\--pid+ options to the +proc+ +subcommand of *cset*. The move option tells the +proc+ subcommand that a task +move is requested. The +-p/\--pid+ option takes an argument called a PIDSPEC +(PID Specification). The PIDSPEC defines which tasks get operated on. + +The PIDSPEC can be a single process ID, a list of process IDs separated by +commas, and a list of process ID ranges also separated by commas. For +example: + ++\--move --pid 1234+:: +This PIDSPEC argument specifies that task 1234 be moved. + ++\--move --pid 1234,42,1934,15000,15001,15002+:: +This PIDSPEC argument specifies that this list of tasks only be moved. + ++\--move --pid 5000,5100,6010-7000,9232+:: +This PIDSPEC argument specifies that tasks 5000,5100 and 9232 be moved along +with any existing task that is in the range 6010 through 7000 inclusive. + +NOTE: A range in a PIDSPEC does not have to have running tasks for every +number in that range. In fact, it is not even an error if there are no tasks +running in that range; none will be moved in that case. The range simply +specifies to act on any tasks that have a PID or TID that is within that +range. + +In the following example, we move the current shell into the cpuset named +'two' with a range PIDSPEC and back out to the 'root' cpuset with the bash +variable for the current PID. + +----------------------------------------------------------------------- +[zuul:cpuset-trunk]# cset proc -l -s two +cset: "two" cpuset of CPUSPEC(2) with 0 tasks running + +[zuul:cpuset-trunk]# echo $$ +19253 + +[zuul:cpuset-trunk]# cset proc -m -p 19250-19260 -t two +cset: moving following pidspec: 19253 +cset: moving 1 userspace tasks to /two +cset: done + +[zuul:cpuset-trunk]# cset proc -l -s two +cset: "two" cpuset of CPUSPEC(2) with 2 tasks running + USER PID PPID SPPr TASK NAME + -------- ----- ----- ---- --------- + root 19253 16447 Roth bash + root 29456 19253 Roth python ./cset proc -l -s two + +[zuul:cpuset-trunk]# cset proc -m -p $$ -t root +cset: moving following pidspec: 19253 +cset: moving 1 userspace tasks to / +cset: done + +[zuul:cpuset-trunk]# cset proc -l -s two +cset: "two" cpuset of CPUSPEC(2) with 0 tasks running +----------------------------------------------------------------------- + +Use of the appropriate PIDSPEC can thus be handy to move tasks and groups of +tasks. Additionally, there is one more option that can help with +multi-threaded processes, and that is the +\--threads+ flag. If this flag is +present in a +proc+ move command with a PIDSPEC and if any of the task IDs in +the PIDSPEC belongs to a thread in a process container, then *all* the sibling +threads in that process container will also get moved. This flag provides an +easy mechanism to move all threads of a process by simply specifying one +thread in that process. In the following example, we move all the threads +running in cpuset 'three' to cpuset 'two' by using the +\--threads+ flag. + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset set two three +cset: + Name CPUs-X MEMs-X Tasks Subs Path + ------------ ---------- - ------- - ----- ---- ---------- + two 2 n 0 n 0 0 /two + three 3 n 0 n 10 0 /three + +[zuul:cpuset-trunk]# cset proc -l -s three +cset: "three" cpuset of CPUSPEC(3) with 10 tasks running + USER PID PPID SPPr TASK NAME + -------- ----- ----- ---- --------- + alext 16165 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 16169 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 16170 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 16237 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 16491 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 16492 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 16493 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 17243 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 17244 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 27133 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + +[zuul:cpuset-trunk]# cset proc -m -p 16165 --threads -t two +cset: moving following pidspec: 16491,16493,16492,16170,16165,16169,27133,17244,17243,16237 +cset: moving 10 userspace tasks to /two +[==================================================]% +cset: done + +[zuul:cpuset-trunk]# cset set two three +cset: + Name CPUs-X MEMs-X Tasks Subs Path + ------------ ---------- - ------- - ----- ---- ---------- + two 2 n 0 n 10 0 /two + three 3 n 0 n 0 0 /three +---------------------------------------------------------- + + +Moving All Tasks from one Cpuset to Another +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +There is a special case for moving all tasks currently running in one cpuset +to another. This can be a common use case, and when you need to do it, +specifying a PIDSPEC with +-p+ is not necessary 'so long as' you use the ++-f/\--fromset+ *and* the +-t/\--toset+ options. + +In the following example, we move all 10 +beagled+ threads back to cpuset +'three' with this method. + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset proc -l two three +cset: "two" cpuset of CPUSPEC(2) with 10 tasks running + USER PID PPID SPPr TASK NAME + -------- ----- ----- ---- --------- + alext 16165 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 16169 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 16170 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 16237 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 16491 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 16492 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 16493 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 17243 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 17244 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... + alext 27133 1 Soth beagled /usr/lib64/beagle/BeagleDaemon.exe --bg -... +cset: "three" cpuset of CPUSPEC(3) with 0 tasks running + +[zuul:cpuset-trunk]# cset proc -m -f two -t three +cset: moving all tasks from two to /three +cset: moving 10 userspace tasks to /three +[==================================================]% +cset: done + +[zuul:cpuset-trunk]# cset set two three +cset: + Name CPUs-X MEMs-X Tasks Subs Path + ------------ ---------- - ------- - ----- ---- ---------- + two 2 n 0 n 0 0 /two + three 3 n 0 n 10 0 /three +---------------------------------------------------------- + + +Moving Kernel Threads with Proc +^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ +Kernel threads are special and *cset* detects tasks that are kernel threads +and will refuse to move them unless you also add a +-k/\--kthread+ option to +your +proc+ move command. Even if you include +-k+, *cset* will 'still' +refuse to move the kernel thread if they are bound to specific CPUs. The +reason for this is system protection. + +A number of kernel threads, especially on the real time Linux kernel, are +bound to specific CPUs and depend on per-CPU kernel variables. If you move +these threads to a different CPU than what they are bound to, you risk at best +that the system will become horribly slow, and at worst a system hang. If you +still insist to move those threads (after all, *cset* needs to give the +knowledgeable user access to the keys), then you need to use the +--force+ +option additionally. + +WARNING: Overriding a task move command with +--force+ can have dire +consequences for the system. Please be sure of the command before you force +it. + +In the following example, we move all unbound kernel threads running in the +'root' cpuset to the cpuset named 'two' by using the +-k+ option. + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset proc -k -f root -t two +cset: moving all kernel threads from / to /two +cset: moving 70 kernel threads to: /two +cset: --> not moving 76 threads (not unbound, use --force) +[==================================================]% +cset: done +---------------------------------------------------------- + +You will note that we used the fromset->toset facility of the +proc+ +subcommand and we only specified the +-k+ option (not the +-m+ option). This +has the effect of moving all kernel threads only. + +Note that only 70 actual kernel threads were moved and 76 were not. The +reason that 76 kernel threads were not moved was because they are bound to +specific CPUs. Now, let's move those kernel threads back to 'root'. + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset proc -k -f two -t root +cset: moving all kernel threads from /two to / +cset: ** no task matched move criteria +cset: **> kernel tasks are bound, use --force if ok + +[zuul:cpuset-trunk]# cset set -l -s two +cset: + Name CPUs-X MEMs-X Tasks Subs Path + ------------ ---------- - ------- - ----- ---- ---------- + two 2 n 0 n 70 0 /two +---------------------------------------------------------- + +Ah! What's this? *Cset* refused to move the kernel threads back to 'root' +because it says that they are "bound." Let's check this with the Linux +taskset command. + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset proc -l -s two | head -5 +cset: "two" cpuset of CPUSPEC(2) with 70 tasks running + USER PID PPID SPPr TASK NAME + -------- ----- ----- ---- --------- + root 2 0 Soth [kthreadd] + root 55 2 Soth [khelper] + +[zuul:cpuset-trunk]# taskset -p 2 +pid 2's current affinity mask: 4 + +[zuul:cpuset-trunk]# cset set -l -s two +cset: + Name CPUs-X MEMs-X Tasks Subs Path + ------------ ---------- - ------- - ----- ---- ---------- + two 2 n 0 n 70 0 /two +---------------------------------------------------------- + +Of course, since the cpuset named 'two' only has CPU2 assigned to it, once we +moved the unbound kernel threads from 'root' to 'two', their affinity masks +got automatically changed to only use CPU2. This is evident from the ++taskset+ output which is a hex value. To really move these threads back to +'root', we need to force the move as follows. + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset proc -k -f two -t root --force +cset: moving all kernel threads from /two to / +cset: moving 70 kernel threads to: / +[==================================================]% +cset: done +---------------------------------------------------------- + + +Destroying Tasks +~~~~~~~~~~~~~~~~ +There actually is no *cset* subcommand or option to destroy tasks--it's not +really needed. Tasks exist and are accessible on the system as normal, even +if they happen to be running in one cpuset or another. To destroy tasks, use +the usual ^C method or by using the *kill(1)* command. + + +Implementing "Shielding" with Set and Proc +------------------------------------------ +With the preceding material on the +set+ and +proc+ subcommands, we now have +the background to implement the basic shielding model, just like the +shield+ +subcommand. + +One may pose the question as to why we want to do this, especially since ++shield+ already does it? The answer is that sometimes one needs more +functionality than +shield+ has to implement one's shielding strategy. In +those case you need to first stop using +shield+ since that subcommand will +interfere with the further application of +set+ and +proc+; however, you will +still need to implement the functionality of +shield+ in order to implement +successful shielding. + +Remember from the above sections describing +shield+, that shielding has at +minimum three cpusets: 'root', which is always present and contains all CPUs; +'system' which is the "non-shielded" set of CPUs and runs unimportant system +tasks; and 'user', which is the "shielded" set of CPUs and runs your important +tasks. Remember also that +shield+ moves all movable tasks into 'system' +and, optionally, moves unbound kernel threads into 'system' as well. + +We start first by creating the 'system' and 'user' cpusets as follows. We +assume that the machine is a four-CPU machine without NUMA memory features. +The 'system' cpuset should hold only CPU0 while the 'user' cpuset should hold +the rest of the CPUs. + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset set -c 0 -s system +cset: --> created cpuset "system" + +[zuul:cpuset-trunk]# cset set -c 1-3 -s user +cset: --> created cpuset "user" + +[zuul:cpuset-trunk]# cset set -l +cset: + Name CPUs-X MEMs-X Tasks Subs Path + ------------ ---------- - ------- - ----- ---- ---------- + root 0-3 y 0 y 333 2 / + user 1-3 n 0 n 0 0 /user + system 0 n 0 n 0 0 /system +---------------------------------------------------------- + +Now, we need to move all running user processes into the 'system' cpuset. + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset proc -m -f root -t system +cset: moving all tasks from root to /system +cset: moving 188 userspace tasks to /system +[==================================================]% +cset: done + +[zuul:cpuset-trunk]# cset set -l +cset: + Name CPUs-X MEMs-X Tasks Subs Path + ------------ ---------- - ------- - ----- ---- ---------- + root 0-3 y 0 y 146 2 / + user 1-3 n 0 n 0 0 /user + system 0 n 0 n 187 0 /system +---------------------------------------------------------- + +We now have the basic shielding set up. Since all userspace tasks are running +in 'system', anything that is spawned from them will also run in 'system'. +The 'user' cpuset has nothing running in it unless you put tasks there with +the +proc+ subcommand as described above. If you also want to move movable +kernel threads from 'root' to 'system' (in order to achieve a form of +"interrupt shielding" on a real time Linux kernel for example), you would +execute the following command as well. + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset proc -k -f root -t system +cset: moving all kernel threads from / to /system +cset: moving 70 kernel threads to: /system +cset: --> not moving 76 threads (not unbound, use --force) +[==================================================]% +cset: done + +[zuul:cpuset-trunk]# cset set -l +cset: + Name CPUs-X MEMs-X Tasks Subs Path + ------------ ---------- - ------- - ----- ---- ---------- + root 0-3 y 0 y 76 2 / + user 1-3 n 0 n 0 0 /user + system 0 n 0 n 257 0 /system +---------------------------------------------------------- + +At this point, you have achieved the simple shielding model that the +shield+ +subcommand provides. You can now add other cpuset definitions to expand your +shielding strategy beyond that simple model. + + +Implementing Hierarchy with Set and Proc +---------------------------------------- +One popular extended "shielding" model is based on hierarchical cpusets, each +with diminishing numbers of CPUs. This model is used to create "priority +cpusets" that allow assignment of CPU resources to tasks based on some +arbitrary priority definition. The idea being that a higher priority task +will get access to more CPU resources than a lower priority task. + +The example provided here once again assumes a machine with four CPUs and no +NUMA memory features. This base serves to illustrate the point well; however, +note that if your machine has (many) more CPUs, then strategies such as this +and others get more interesting. + +We define a shielding set up as in the previous section where we have a +'system' cpuset with just CPU0 that takes care of "unimportant" system tasks. +One usually requires this type of cpuset since it forms the basis of +shielding. We modify the strategy to not use a 'user' cpuset, instead we +create a number of new cpusets each holding one more CPU than the other. +These cpusets will be called 'prio_low' with one CPU, 'prio_med' with two +CPUs, 'prio_high' with three CPUs, and 'prio_all' with all CPUs. + +NOTE: One may ask, why create a 'prio_all' with all CPUs when that is +substantially the definition of the 'root' cpuset? The answer is that it is +best to keep a separation between the 'root' cpuset and everything else, even +if a particular cpuset duplicates 'root' exactly. Usually, one builds +automation on top of a cpuset strategy. In these cases, it is best to avoid +using invariant names of cpusets, such as 'root' for example, in this +automation. + +All of these 'prio_*' cpusets can be created under root, in a flat way; +however, it is advantageous to create them as a hierarchy. The reasoning for +this is twofold: first, if a cpuset is destroyed, all its tasks are moved to +its parent; second, one can use exclusive CPUs in a hierarchy. + +There is a planned addition to the +proc+ subcommand that will allow moving a +specified PIDSPEC of tasks running in a specified cpuset to its parent. This +addition will ease the automation burden. + +If a cpuset has CPUs that are exclusive to it, then other cpusets may not make +use of those CPUs unless they are children of that cpuset. This has more +relevance to machines with many CPUs and more complex strategies. + +Now, we start with a clean slate and build the appropriate cpusets as +follows. + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset set -r +cset: + Name CPUs-X MEMs-X Tasks Subs Path + ------------ ---------- - ------- - ----- ---- ---------- + root 0-3 y 0 y 344 0 / + +[zuul:cpuset-trunk]# cset set -c 0-3 prio_all +cset: --> created cpuset "prio_all" + +[zuul:cpuset-trunk]# cset set -c 1-3 /prio_all/prio_high +cset: --> created cpuset "/prio_all/prio_high" + +[zuul:cpuset-trunk]# cset set -c 2-3 /prio_all/prio_high/prio_med +cset: --> created cpuset "/prio_all/prio_high/prio_med" + +[zuul:cpuset-trunk]# cset set -c 3 /prio_all/prio_high/prio_med/prio_low +cset: --> created cpuset "/prio_all/prio_high/prio_med/prio_low" + +[zuul:cpuset-trunk]# cset set -c 0 system +cset: --> created cpuset "system" + +[zuul:cpuset-trunk]# cset set -l -r +cset: + Name CPUs-X MEMs-X Tasks Subs Path + ------------ ---------- - ------- - ----- ---- ---------- + root 0-3 y 0 y 344 2 / + system 0 n 0 n 0 0 /system + prio_all 0-3 n 0 n 0 1 /prio_all + prio_high 1-3 n 0 n 0 1 /prio_all/prio_high + prio_med 2-3 n 0 n 0 1 /prio_all/prio_high/prio_med + prio_low 3 n 0 n 0 0 /prio_all/pr...rio_med/prio_low +---------------------------------------------------------- + +NOTE: We used the +-r/\--recurse+ switch to list all the sets in the last +command above. If we had not, then the 'prio_med' and 'prio_low' cpusets +would not have been listed. + +The strategy is now implemented and we now move all userspace tasks as well as +all movable kernel threads into the 'system' cpuset to activate the shield. + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset proc -m -k -f root -t system +cset: moving all tasks from root to /system +cset: moving 198 userspace tasks to /system +cset: moving 70 kernel threads to: /system +cset: --> not moving 76 threads (not unbound, use --force) +[==================================================]% +cset: done + +[zuul:cpuset-trunk]# cset set -l -r +cset: + Name CPUs-X MEMs-X Tasks Subs Path + ------------ ---------- - ------- - ----- ---- ---------- + root 0-3 y 0 y 76 2 / + system 0 n 0 n 268 0 /system + prio_all 0-3 n 0 n 0 1 /prio_all + prio_high 1-3 n 0 n 0 1 /prio_all/prio_high + prio_med 2-3 n 0 n 0 1 /prio_all/prio_high/prio_med + prio_low 3 n 0 n 0 0 /prio_all/pr...rio_med/prio_low +---------------------------------------------------------- + +The shield is now active. Since the 'prio_*' cpuset names are unique, one can +assign tasks to them either via either their simple name, or their full path +(as described in the +proc+ section above). + +You may have noted that there is an ellipsis in the path of the 'prio_low' +cpuset in the listing above. This is done in order to fit the output onto an +80 character screen. If you want to see the entire line, then you need to use +the +-v/\--verbose+ flag as follows. + +---------------------------------------------------------- +[zuul:cpuset-trunk]# cset set -l -r -v +cset: + Name CPUs-X MEMs-X Tasks Subs Path + ------------ ---------- - ------- - ----- ---- ---------- + root 0-3 y 0 y 76 2 / + system 0 n 0 n 268 0 /system + prio_all 0-3 n 0 n 0 1 /prio_all + prio_high 1-3 n 0 n 0 1 /prio_all/prio_high + prio_med 2-3 n 0 n 0 1 /prio_all/prio_high/prio_med + prio_low 3 n 0 n 0 0 /prio_all/prio_high/prio_med/prio_low +---------------------------------------------------------- + + +Using Shortcuts +=============== +The commands listed in the previous sections always used all the required +options. *Cset* however does have a shortcut facility that will execute +certain commands without specifying all options. An effort has been made to +do this with the "principle of least surprise." This means that if you do not +specify options, but do specify parameters, then the outcome of the command +should be intuitive. As much as possible. + +Using shortcuts is of course not necessary. In fact, you can not only not use +shortcuts, but you can use long options instead of short, in case you really +enjoy typing... All kidding aside, using long options and not using shortcuts +does have a use case: when you write a script intended to be self-documenting, +or perhaps when you generate *cset* documentation. + +To begin, the subcommands +shield+, +set+ and +proc+ can themselves be +shortened to the fewest number of characters that are unambiguous. For +example, the following commands are identical: + +---------------------------------------------------------- +# cset shield -s -p 1234 <--> # cset sh -s -p 1234 +# cset set -c 1,3 -s newset <--> # cset se -c 1,3 -s newset +# cset proc -s newset -e bash <--> # cset p -s newset -e bash +---------------------------------------------------------- + +Note that +proc+ can be shortened to just +p+, while +shield+ and +set+ need +two letters to disambiguate. + + +Shield Subcommand Shortcuts --------------------------- +The +shield+ subcommand supports two areas with shortcuts: the case when there +is no options given where to 'shield' is the common use case, and making the ++-p/\--pid+ option 'optional' for the +-s/\--shield+ and +-u/\--unshield+ +options. + +For the common use case of actually 'shielding' either a PIDSPEC or execing a +command into the shield, the following *cset* commands are equivalent. + +----------------------------------------------------------- +# cset shield -s -p 1234,500-649 <--> # cset sh 1234,500-649 +# cset shield -s -e bash <--> # cset sh bash +----------------------------------------------------------- + +When using the +-s+ or +-u+ shield/unshield options, it is optional to use the ++-p+ option to specify a PIDSPEC. For example: + +----------------------------------------------------------- +# cset shield -s -p 1234 <--> # cset sh -s 1234 +# cset shield -u -p 1234 <--> # cset sh -u 1234 +----------------------------------------------------------- + + +Set Subcommand Shortcuts +------------------------ +The +set+ subcommand has a limited number of shortcuts. Basically, the ++-s/\--set+ option is optional in most cases and the +-l/\--list+ option is +also optional if you want to list sets. For example, these commands are +equivalent. + +----------------------------------------------------------- +# cset set -l -s myset <--> # cset se -l myset +# cset se -l myset <--> # cset se myset + +# cset set -c 1,2,3 -s newset <--> # cset se -c 1,2,3 newset +# cset set -d -s newset <--> # cset se -d newset + +# cset set -n newname -s oldname <--> # cset se -n newname oldname +----------------------------------------------------------- + +In fact, if you want to apply either the list or the destroy options to +multiple cpusets with one *cset* command, you'll need to not use the +-s+ +option. For example: + +----------------------------------------------------------- +# cset se -d myset yourset ourset + --> destroys cpusets: myset, yourset and ourset + +# cset se -l prio_high prio_med prio_low + --> lists only cpusets prio_high, prio_med and prio_low + --> the -l is optional in this case since list is default +----------------------------------------------------------- + +Proc Subcommand Shortcuts +------------------------- +For the +proc+ subcommand, the +-s+, +-t+ and +-f+ options to specify the +cpuset, the origination cpuset and the destination cpuset, can sometimes be +optional. For example, the following commands are equivalent. + +----------------------------------------------------------- +To list tasks in cpusets: +# cset proc -l -s myset \ +# cset proc -l -f myset --> # cset p -l myset +# cset proc -l -t myset / + +# cset p -l myset <--> # cset p myset + +# cset proc -l -s one two <--> # cset p -l one two +# cset p -l one two <--> # cset p one two + +To exec a process into a cpuset: +# cset proc -s myset -e bash <--> # cset p myset -e bash +----------------------------------------------------------- + +Movement of tasks into and out of cpusets have the following shortcuts. + +----------------------------------------------------------- +To move a PIDSPEC into a cpuset: +# cset proc -m -p 4242,4243 -s myset <--> # cset p -m 4242,4243 myset +# cset proc -m -p 12 -t myset <--> # cset p -m 12 myset + +To move all tasks from one cpuset to another: +# cset proc -m -f set1 -t set2 \ +# cset proc -m -s set1 -t set2 --> # cset p -m set1 set2 +# cset proc -m -f set1 -s set2 / +----------------------------------------------------------- + + +What To Do if There are Problems +================================ +If you encounter problems with the *cset* application, the best option is to +log a bug with the *cset* bugzilla instance found here: + + http://devzilla.novell.com/cpuset + +If you are using *cset* on a supported operating system such as SLES or SLERT +from Novell, then feel free to use that bugzilla instead. + +If the problem is repeatable, there is an excellent chance that it will get +fixed quickly. Also, *cset* contains a logging facility that is invaluable +for the developers to diagnose problems. To create a log of a run, use the ++-l/\--log+ option with a filename as an argument to the main *cset* +application. For example. +------------------------------------------------------------- +# cset -l logfile.txt set -n newname oldname +------------------------------------------------------------- +That command saves a lot of debugging information in the 'logfile.txt' file. +Please attach this file to the bug. \ No newline at end of file diff --git a/setup.py b/setup.py index 4591a3e..15df15d 100755 --- a/setup.py +++ b/setup.py @@ -20,7 +20,7 @@ setup(name = 'cpuset', scripts = ['cset'], packages = ['cpuset', 'cpuset.commands'], data_files = [ - ('share/doc/packages/cpuset', ['README', 'COPYING']), + ('share/doc/packages/cpuset', ['README', 'COPYING', 'AUTHORS', 'NEWS', 'INSTALL']), ('share/doc/packages/cpuset', glob.glob('doc/*.html')), ('share/doc/packages/cpuset', glob.glob('doc/*.txt')), ]