MPI topic: Tools interface

Experimental html version of downloadable textbook, see http://www.tacc.utexas.edu/~eijkhout/istc/istc.html
\[ \newcommand\inv{^{-1}}\newcommand\invt{^{-t}} \newcommand\bbP{\mathbb{P}} \newcommand\bbR{\mathbb{R}} \newcommand\defined{ \mathrel{\lower 5pt \hbox{${\equiv\atop\mathrm{\scriptstyle D}}$}}} \] 13.1 : Initializing the tools interface
13.2 : Control variables
13.3 : Performance variables
13.3.1 : Performance experiment sessions
13.4 : Categories of variables
Back to Table of Contents

13 MPI topic: Tools interface

13.1 Initializing the tools interface

crumb trail: > mpi-tools > Initializing the tools interface

The tools interface requires a different initialization routine MPI_T_init_thread

int MPI_T_init_thread( int required,int *provided );

Likewise, there is MPI_T_finalize

int MPI_T_finalize();

These matching calls can be made multiple times, after MPI has already been initialized with MPI_Init or MPI_Init_thread .

Verbosity level is an integer parameter.

MPI_T_VERBOSITY_{USER,TUNER,MPIDEV}_{BASIC,DETAIL,ALL}

13.2 Control variables

crumb trail: > mpi-tools > Control variables

Control variables

that can be used to inspect and/or control the internal workings of MPI. Accessing control variables requires initializing the tools interface; section  13.1 .

We query how many control variables are available with


MPI_T_cvar_get_num .

%% int MPI_T_cvar_get_num( int *number_of_cvars );
%% 

A description of the control variable can be obtained from


MPI_T_cvar_get_info .

%% int MPI_T_cvar_get_info( int cvar_num,
%%   char *name, int *name_length,
%%   int *verbosity, MPI_Datatype *type,MPI_T_enum *enumtype,
%%   char *description,int *description_length,
%%   int *bind,int *scope);
%% 
  • An invalid index leads to a function result of MPI_T_ERR_INVALID_INDEX .
  • Any output parameter can be specified as NULL and MPI will not set this.
  • The bind variable is an object type or MPI_T_BIND_NO_OBJECT .
  • The enumtype variable is MPI_T_ENUM_NULL if the variable is not an enum type.
// cvar.c
MPI_T_cvar_get_num(&ncvar);
printf("#cvars: %d\n",ncvar);
for (int ivar=0; ivar<ncvar; ivar++) {
  char name[100]; int namelen = 100;
  char desc[256]; int desclen = 256;
  int verbosity,bind,scope;
  MPI_Datatype datatype;
  MPI_T_enum enumtype;
  MPI_T_cvar_get_info
	(ivar,
	 name,&namelen,
	 &verbosity,&datatype,&enumtype,desc,&desclen,&bind,&scope
	 );
  printf("cvar %3d: %s\n  %s\n",ivar,name,desc);
Remark

There is no constant indicating a maximum buffer length for these variables. However, you can do the following:

  1. Call the info routine with NULL values for the buffers, reading out the buffer lengths;
  2. allocate the buffers with sufficient length, that is, including an extra position for the null terminator; and
  3. calling the info routine a second time, filling in the string buffers.

Conversely, given a variable name, its index can be retrieved with MPI_T_cvar_get_index :

int MPI_T_cvar_get_index(const char *name, int *cvar_index)

If the name can not be matched, the index is MPI_T_ERR_INVALID_NAME .

Accessing a control variable is done through a control variable handle .

int MPI_T_cvar_handle_alloc
   (int cvar_index, void *obj_handle,
    MPI_T_cvar_handle *handle, int *count)

The handle is freed with MPI_T_cvar_handle_free :

int MPI_T_cvar_handle_free(MPI_T_cvar_handle *handle)

Control variable access is done through MPI_T_cvar_read and MPI_T_cvar_write :

int MPI_T_cvar_read(MPI_T_cvar_handle handle, void* buf);
int MPI_T_cvar_write(MPI_T_cvar_handle handle, const void* buf);

13.3 Performance variables

crumb trail: > mpi-tools > Performance variables

\begin{raggedlist} Performance variables come in classes: MPI_T_PVAR_CLASS_STATE MPI_T_PVAR_CLASS_LEVEL MPI_T_PVAR_CLASS_SIZE MPI_T_PVAR_CLASS_PERCENTAGE MPI_T_PVAR_CLASS_HIGHWATERMARK MPI_T_PVAR_CLASS_LOWWATERMARK MPI_T_PVAR_CLASS_COUNTER MPI_T_PVAR_CLASS_AGGREGATE MPI_T_PVAR_CLASS_TIMER MPI_T_PVAR_CLASS_GENERIC \end{raggedlist}

Query the number of performance variables with MPI_T_pvar_get_num :

int MPI_T_pvar_get_num(int *num_pvar);

Get information about each variable, by index, with MPI_T_pvar_get_info :

int MPI_T_pvar_get_info
   (int pvar_index, char *name, int *name_len,
    int *verbosity, int *var_class, MPI_Datatype *datatype,
    MPI_T_enum *enumtype, char *desc, int *desc_len, int *bind,
    int *readonly, int *continuous, int *atomic)

See general remarks about these in section  13.2 .

  • The readonly variable indicates that the variable can not be written.
  • The continuous variable requires use of MPI_T_pvar_start and MPI_T_pvar_stop .

Given a name, the index can be retried with MPI_T_pvar_get_index :

int MPI_T_pvar_get_index(const char *name, int var_class, int *pvar_index)

Again, see section  13.2 .

13.3.1 Performance experiment sessions

crumb trail: > mpi-tools > Performance variables > Performance experiment sessions

To prevent measurements from getting mixed up, they need to be done in performance experiment session s, to be called `sessions' in this chapter. However see section  8.3 .

Create a session with MPI_T_pvar_session_create

int MPI_T_pvar_session_create(MPI_T_pvar_session *session)

and release it with MPI_T_pvar_session_free :

int MPI_T_pvar_session_free(MPI_T_pvar_session *session)

which sets the session variable to MPI_T_PVAR_SESSION_NULL .

We access a variable through a handle, associated with a certain session. The handle is created with MPI_T_pvar_handle_alloc :

int MPI_T_pvar_handle_alloc
   (MPI_T_pvar_session session, int pvar_index,
    void *obj_handle, MPI_T_pvar_handle *handle, int *count)

(If a routine takes both a session and handle argument, and the two are not associated, an error of MPI_T_ERR_INVALID_HANDLE is returned.)

Free the handle with MPI_T_pvar_handle_free :

int MPI_T_pvar_handle_free
   (MPI_T_pvar_session session,
    MPI_T_pvar_handle *handle)

which sets the variable to MPI_T_PVAR_HANDLE_NULL .

Continuous variables (see MPI_T_pvar_get_info above, which outputs this) can be started and stopped with MPI_T_pvar_start and MPI_T_pvar_stop :

int MPI_T_pvar_start(MPI_T_pvar_session session, MPI_T_pvar_handle handle);
int MPI_T_pvar_stop(MPI_T_pvar_session session, MPI_T_pvar_handle handle)

Passing MPI_T_PVAR_ALL_HANDLES to the stop call attempts to stop all variables within the session. Failure to stop a variable returns MPI_T_ERR_PVAR_NO_STARTSTOP .

Variables can be read and written with MPI_T_pvar_read and MPI_T_pvar_write :

int MPI_T_pvar_read
   (MPI_T_pvar_session session, MPI_T_pvar_handle handle,
    void* buf)
int MPI_T_pvar_write
   (MPI_T_pvar_session session, MPI_T_pvar_handle handle,
    const void* buf)

If the variable can not be written (see the readonly parameter of MPI_T_pvar_get_info ), MPI_T_ERR_PVAR_NO_WRITE is returned.

A special case of writing the variable is to reset it with

int MPI_T_pvar_reset(MPI_T_pvar_session session, MPI_T_pvar_handle handle)

The handle value of MPI_T_PVAR_ALL_HANDLES is allowed.

A call to MPI_T_pvar_readreset is an atomic combination of the read and reset calls:

int MPI_T_pvar_readreset
   (MPI_T_pvar_session session,MPI_T_pvar_handle handle,
    void* buf)

13.4 Categories of variables

crumb trail: > mpi-tools > Categories of variables

Variables, both the control and performance kind, can be grouped into categories by the MPI implementation.

The number of categories is queried with MPI_T_category_get_num :

int MPI_T_category_get_num(int *num_cat)

and for each category the information is retrieved with MPI_T_category_get_info :

int MPI_T_category_get_info
   (int cat_index,
    char *name, int *name_len, char *desc, int *desc_len,
    int *num_cvars, int *num_pvars, int *num_categories)

For a given category name the index can be found with MPI_T_category_get_index :

int MPI_T_category_get_index(const char *name, int *cat_index)

The contents of a category are retrieved with MPI_T_category_get_cvars , MPI_T_category_get_pvars , MPI_T_category_get_categories :

int MPI_T_category_get_cvars(int cat_index, int len, int indices[])
int MPI_T_category_get_pvars(int cat_index, int len, int indices[])
int MPI_T_category_get_categories(int cat_index, int len, int indices[])

\begin{raggedlist} These indices can subsequently be used in the calls MPI_T_cvar_get_info , MPI_T_pvar_get_info , MPI_T_category_get_info . \end{raggedlist}

If categories change dynamically, this can be detected with MPI_T_category_changed

int MPI_T_category_changed(int *stamp)

Back to Table of Contents