Commit 09cb4ef7 authored by Spencer Sherwin's avatar Spencer Sherwin

Updated the documentation for the new nparts option and removed the nprocs,procid options

parent 152c065a
......@@ -993,54 +993,81 @@ within the directory.
\section{Processing large files in serial}
When processing large files, it is not always convenient to run in parallel but
process each parallel partition in serial, for example when interpolating a
solution field from one mesh to another.
solution field from one mesh to another or creating an output file for
visualization.
\subsection{Using the \texttt{nprocs} and \texttt{procid} options}
\subsection{Using the \textit{nparts} options}
One option is to use the \inltt{--nprocs} and \inltt{--procid} command line
options. For example, the following command will interpolate partition 2 of a
decomposition into 10 partitions of \inltt{fiile2.xml} from \inltt{file1.fld}
One option is to use the \inltt{nparts} command line
option. For example, the following command will create a
\inltt{.vtu} file using 10 partitions of \inltt{file1.xml}:
\begin{lstlisting}[style=BashInputStyle]
FieldConvert --nprocs 10 --procid 2 \
-m interpfield:fromxml=file1.xml:fromfld=file1.fld \
file2.xml file2.fld
FieldConvert --nparts 10 file1.xml file1.fld file1.vtu
\end{lstlisting}
This call will only therefore consider the interpolation process across one
partition (namely, partition 2). To create the full interpolated field we can
suppress the \inltt{--procid} option, in which case FieldConvert will
loop over each of the partitions:
Note this will create a parallel vtu file as it processes each partition.
Another example is to interpolate \inltt{file1.fld} from one mesh
\inltt{file1.xml} to another \inltt{file2.xml}. If the mesh files are
large we can do this by partitioning \inltt{file2.xml} into 10 (or more)
partitions and interpolating each partition one by one using the
command:
\begin{lstlisting}[style=BashInputStyle]
FieldConvert --nprocs 10 \
-m interpfield:fromxml=file1.xml:fromfld=file1.fld \
file2.xml file2.fld
FieldConvert --nparts 10 -m interpfield:fromxml=file1.xml:fromfld=file1.fld \
file2.xml file2.fld
\end{lstlisting}
Note that internally the routine uses the range option so that it
only has to load the part of \inltt{file1.xml} that overlaps with each
partition of \inltt{file2.xml}.
The resulting output will lie in a directory called \inltt{file2.fld}, with each
of the different parallel partitions in files with names \inltt{P0000000.fld},
\inltt{P0000001.fld}, \dots, \inltt{P0000009.fld}. This is nearly a complete
parallel field file. However, when the output file is in the .fld format,
the \inltt{Info.xml} file, which contains the information about which elements
lie in each partition, is missing. This can be generated by using the command
lie in each partition, is not correct since it will only contain the information for one of the partitions. The correct \inltt{Info.xml} file can be generated by using the command
\begin{lstlisting}[style=BashInputStyle]
FieldConvert file2.xml file2.fld/Info.xml:info:nprocs=10
FieldConvert file2.xml file2.fld/Info.xml:info:nparts=10
\end{lstlisting}
Note the \inltt{:info} extension on the last argument is necessary to tell
FieldConvert that you wish to generate an info file, but with the extension
\inltt{.xml}. This syntax allows the routine not to get confused with the
input/output XML files.
\subsection{Using the --part-only and --part-only-overlapping options}
\subsection{Running in parallel with the \textit{ nparts} option}
The examples above will process each partition serially which may now
take a while for many partitions. You can however run this option in
parallel using a smaller number of cores than the nparts.
For the example of creating a vtu file above you can use 4 processor concurrently wiht the command line:
\begin{lstlisting}[style=BashInputStyle]
mpirun -n 4 FieldConvert --nparts 10 file1.xml file1.fld file1.vtu
\end{lstlisting}
Obviously the executable will have to have been compiled with the MPI option for this to work.
\subsection{Using the \textit{ part-only} and \textit{ part-only-overlapping} options}
Another approach to serially proessing a large file is to initially process the
file into multiple partitions. This can be done with the \inltt{--part-only}
The options above will all load in the full \inltt{file1.xml}, partition
it into \inltt{nparts} files in a director called \inltt{file1\_xml}.
This can be expensive if the \inltt{file1.xml} is already large. So instead you can
pre-partition the file using the using the \inltt{--part-only}
option. So the command
\begin{lstlisting}[style=BashInputStyle]
FieldConvert --part-only 10 file.xml file.fld
\end{lstlisting}
will partition the mesh into 10 paritions and write each partition into a
will partition the mesh into 10 partitions and write each partition into a
directory called \inltt{file\_xml}. If you enter this directory you will find
partitioned XML files \inltt{P0000000.xml}, \inltt{P0000001.xml}, \dots,
\inltt{P0000009.xml} which can then be processed individually as outlined above.
If you have a partitioned directory either from a parallel run or using the \inltt{--part-only} option you can now run the \inltt{FieldConvert} option using the command
\begin{lstlisting}[style=BashInputStyle]
mpirun -n 4 FieldConvert --nparts 10 file1\_xml:xml file1.fld file1.vtu
\end{lstlisting}
Note the form \inltt{file1\_xml:xml} option tells the code it is a parallel partition which should be treated as an \inltt{xml} type file.
There is also a \inltt{--part-only-overlapping} option, which can be run in the
same fashion.
\begin{lstlisting}[style=BashInputStyle]
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment