Linux Certif

Toute la documentation sur la certification Linux LPI

mpi-selector

NAME

mpi-selector - A simple site-wide/per-user MPI selection tool for clusters with more than one MPI implementation installed.

SYNOPSIS

Commands for end users

mpi-selector [options] --list

mpi-selector [options] --set <name>

mpi-selector [options] --unset

mpi-selector [options] --query

mpi-selector [options] --version

Commands for MPI implementations

mpi-selector [options] --register <name> --source-dir <dir>

mpi-selector [options] --unregister <name>

DESCRIPTION

The mpi-selector command is a simplistic tool to select one of multiple MPI implementations. mpi-selector allows system administrators to set a site-wide default MPI implementation while also allowing users to set their own default MPI implementation (thereby overriding the system-wide default).

Note that both the site-wide and per-user defaults are independent from each other; a system administrator may choose not to have a site-wide default while a user may choose to have a personal default --- and vice versa.

The system is effected by having system-wide shell startup files that looks first at the user's MPI preferences. If found, the MPI implementation indicated by the user's preferences is setup in the current environment. If not found, look for a site-wide default. If found, the MPI implementation indicated in by the site-wide default is setup in the current environment. If not found, exit silently.

End Users / System Administrators

The mpi-selector command provides four main actions:

* List which MPI implementations are available

* Set a default (either on a per-user or site-wide basis)

* Unset a default (either on a per-user or site-wide basis)

* Query what the current default is

A common scenario is that a system administrator sets a site-wide default for a supported MPI implementation that most users will use. Power users then change their per-user defaults to use a different MPI implementation.

Another common scenario is for power users to frequently use mpi-selector to swap back and forth between multiple different MPI implementations.

NOTE: The mpi-selector command only changes the defaults for new shells. Specifically, after you invoke the mpi-selector command to change the default MPI implementation, this change does not take effect until you start a new shell. This is intentional. See the ``KNOWN LIMITATIONS'' section, below.

MPI Implementations

MPI implementations register themselves with mpi-selector when they are installed and unregister themselves when they are uninstalled. Each MPI installation provides two files that setup the environment for itself:

* mpivars.sh: File sourceable by Bourne-like shells (sh, bash, etc.)

* mpivars.csh: File sourceable by C-like shells (csh, tcsh, etc.)

These files are expected to be in a single directory and ``registered'' with mpi-selector using the --register and --source-dir options. mpi-selector will copy these files to its own internal store; it is safe to remove the originals after the mpi-selector registration completes successfully.

The <name> argument to --register must be simplistic --- it cannot contain any shell special characters (not even if they are escaped), nor can it contain spaces. The intent is to provide simple names that users can type without escaping or quoting. Names not conforming to these rules will be rejected and the registration will fail.

When an MPI implementation is uninstalled, it should unregister with mpi-selector via the --unregister option.

OPTIONS

--list: List which MPI implementations are available

--no: Assume ``no'' to any interactive questions asked.

--query: See what the current default is. If specified with no options, whichevery default has precedence --- if any --- will be shown. If specified with --user, only show the per-user default (if there is one). If specified with --system, only show the site-wide default (if there is one).

--register: Register a new MPI implementation. Must be combined with the --source-dir option.

--set <name>: Set the default MPI implementation. May be combined with --system or --user (--user is the default and does not need to be specified).

--source-dir: Specify the location where mpivars.* files should be copied from. Only meaningful when used with the --register option.

--system: When used with --set or --unset, specifies to work with the site-wide default (vs. the per-user default). When used with --query, it specifies to specifically query the site-wide default.

--unregister: Unregister an MPI implementation.

--user: When used with --set or --unset, specifies to work with the per-user default (vs. the site-wide default). When used with --query, it specifies to specifically query the per-user default.

--unset: Unset the default MPI implementation. May be combined with --system or --user (--user is the default and does not need to be explicitly specified).

--verbose: Be verbose.

--version: Return the version of mpi-selector.

--yes: Assume ``yes'' to any interactive questions asked.

EXAMPLES

Examples for End Users / System Administrators

The four main actions that system administrators and end users invoke are: listing which MPI implementations are available, setting a default, unsetting a default, and querying what the current default is.

Listing which MPI implementations are available

The --list option to the mpi-selector command shows a simple list of which MPI implementations are available:

   shell$ mpi-selector --list
   mympi-1.2.3
   mympi-4.5.6
   othermpi-7.8.9
   shell$
 
 

Setting a default

By default, MPI selections are performed on a per-user basis with the --set option, using a name from the list of available MPI implementations (which can be obtained via the --list command):

   shell$ mpi-selector --set mympi-4.5.6
   shell$
 
 

Note that the default takes effect in the next shell that is started; it does NOT take effect in the current shell!

If a default MPI is already set, setting a new default will cause an interactive confirmation prompt. This interactive prompt can be avoided by using the --yes option, which assumes a ``yes'' answer to all questions:

   shell$ mpi-selector --set mympi-4.5.6
   shell$ mpi-selector --set mympi-4.5.6 --yes
   shell$
 
 

If the --system option is used, the site-wide default is modified instead of the per-user default. Since this option typically reqires writing files into protected areas, root access may be required.

   shell# mpi-selector --set mympi-4.5.6 --system
   shell#
 
 

Unsetting a default

Unset the current default with the --unset option:

   shell$ mpi-selector --unset
   shell$
 
 

Similar to --set, the --system option can be used to unset the site-wide default

   shell# mpi-selector --unset --system
   shell#
 
 

Querying what the current default is

The --query option can be used to see what the current MPI implementation is (more specifically, what the MPI implementation will be for the next shell that is started). It indicates both which MPI is the default and at what level the default was set (per-user vs. site-wide):

   shell$ mpi-selector --set mympi-1.2.3
   shell$ mpi-selector --query
   default:mympi-1.2.3
   level:user
   shell$
 
 

Note that if there is no per-user default, the system default will be shown:

   shell# mpi-selector --set othermpi-7.8.9 --system
 
 

   shell$ mpi-selector --unset
   shell$ mpi-selector --query
   default:othermpi-7.8.9
   level:system
   shell$ mpi-selector --set mympi-1.2.3
   shell$ mpi-selector --query
   default:mympi-1.2.3
   level:user
   shell$
 
 

If there is no per-user default and no site-wide default, --query will return silently:

   shell$ mpi-selector --query
   shell$
 
 

Examples for MPI Implementations

Registering and unregistering typically writes files into protected areas, and therefore usually requires root access.

If there are no MPI implementations registered, --list will return silently:

   shell# mpi-selector --list
   shell#
 
 

An MPI with mpivars.sh and mpivars.csh in /opt/mympi/bin can be registered as follows:

   shell# mpi-selector --register mympi-1.2.3 \
          --source-dir /opt/mympi/bin
   shell# mpi-selector --list
   mympi-1.2.3
   shell#
 
 

Note that re-registering the same <name> will cause an interactive confirmation prompt; the --yes option can be supplied to assume ``yes'' to all questions asked:

   shell# mpi-selector --list
   mympi-1.2.3
   shell# mpi-selector --register mympi-1.2.3 \
          --source-dir /opt/mympi/bin --yes
   mympi-1.2.3 is already registered.
   Overwriting previously registered files.
   shell# mpi-selector --list
   mympi-1.2.3
   shell#
 
 

Unregistering is also simple:

   shell# mpi-selector --list
   mympi-1.2.3
   shell# mpi-selector --unregister mympi-1.2.3
   shell# mpi-selector --list
   shell#
 
 

Registering and Unregistering in RPMs

Registering and unregistering via RPM is unfortunately more complicated than it needs to be because of the following issues:

1. Although RPM obeys dependency ordering of ``rpm -i a b c''. That is,
   c will be installed before a if a requires c.
   Regardless, RPM's must know a) that the mpi-selector command is
   installed, and b) be able to find it in its path.

2. RPM does not obey dependency ordering of ``rpm -e a b c''. That is,
   c may be uninstalled before a, even if a requires c.
   Hence, the mpi-select command may disappear before an RPM using the
   mpi-select command in a scriptlet is uninstalled.

3. ``Updating'' RPMs will first uninstall the old RPM and then
   re-install the new one.

Additionally, the staged installations (such as the OFED installer) require telling the mpi-selector command additional information so that various internal data files can be found.

In general, MPI installations via RPMs should register during the %post scriptlet and unregister during the %preun scriptlet (not during the %postun scriptlet!).

If RPMs ``require'' the mpi-selector RPM, they can be assured that the mpi-selector command will exist and be installed properly, but they still need to be able to find mpi-selector in their PATH. Hence, if mpi-selector is not installed into a default PATH location, the %post scriptlet won't be able to find it, and the registration call will fail. The simplest workaround (at least for the moment) is to set the PATH to where mpi-selector is installed before installing any RPMs that use it.

With that in mind, here is a possible %post scriptlet for OFED-installed RPMS:

   mpi-selector --register <name> --source-dir <source_dir> \
       --yes --silent
 
 

Note the following:

1. The --yes option forces an overwrite if, for some reason, a
   previous MPI of the name name is already registered.

2. The --silent option makes mpi-selector run silently, since RPMs
   are supposed to install with no output.

Here is a possible %preun scriptlet for OFED-installed RPMs:

   mpi-selector --unregister <name> --yes || \
       /bin/true > /dev/null 2> /dev/null
 
 

Note the following:

1. We use %preun instead of %postun because of RPM's upgrade behavior.

2. Since RPM does not honor dependencies when uninstalling, it is
   possible that mpi-selector is no longer installed, and therefore
   the command may fail.  However, since mpi-selector is no longer
   installed, we don't care that it failed (i.e., there's nothing to
   unregister from), so just redirect all output to /dev/null and
   ensure that the return code from the overall command is ``true'' (RPM
   will abort if any individual scriptlet command fails).

KNOWN LIMITATIONS

The main known limitation of mpi-selector is that it only affects future shells --- running it does not affect the current shell. After you run mpi-selector to set a new default MPI (regardless of whether it is a system-level or user-specific default), that default will not take effect until you start a new shell --- even though --query will report the new default.

This behavior is because mpi-selector defaults are only read during shell startup. It was an intentional design decision --- mpi-selector is intended to be a simplistic tool, and an all-encompassing solution.

Other solutions for modifying the current environment exist, such as the Environment Modules package (<http://modules.sourceforge.net/>) and SoftEnv from Argonne National Laboratory (and probably others). Using these tools, you can immediately change the environment of the current shell (to include switching to use a different MPI implementation). As such, these already-existing, mature tools are better suited for such usage patterns; mpi-selector is not intended to replace them.

For rsh/ssh-based parallel environments, switching defaults frequently should be done with care. Specifically, rsh/ssh-based launchers may depends on a common environment across all nodes (e.g., to find helper executables and/or libraries for a specific MPI). Consider the following example:

   shell$ mpi-selector --set mympi-1.2.3
   shell$ mpicc my_mpi_app.c -o my_mpi_app
   shell$ mpirun -np 32 --hostfile myhosts my_mpi_app
 
 

While my_mpi_app is starting, it may be dangerous to switch the mpi-selector default (perhaps in a different window) because the rsh and/or ssh commands currently executing may be relying on finding the same MPI implementation on all nodes. Changing the default while the application is launching may cause a different MPI implementation to be found on some nodes, thereby causing undefined behavior.

FILES

$HOME/.mpi-selector Location of per-user default selection

/etc/sysconfig/mpi-selector Location of site-wide default selection.

/var/lib/mpi-selector/data/data Directory containing registered MPI shell startup files.

AUTHOR

Written by Jeff Squyres.

REPORTING BUGS

Send bug reports to the OpenFabrics general mailing list (see <http://www.openfabrics.org/>). This is a high-volume mailing list, so be sure to put ``mpi-selector'' in the subject to ensure that it is not missed.

COPYRIGHT

Copyright (c) 2007 Cisco Systems, Inc. All rights reserved.

This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.