Commit 09cb4ef7 authored by Spencer Sherwin's avatar Spencer Sherwin
Browse files

Updated the documentation for the new nparts option and removed the nprocs,procid options

parent 152c065a
...@@ -993,54 +993,81 @@ within the directory. ...@@ -993,54 +993,81 @@ within the directory.
\section{Processing large files in serial} \section{Processing large files in serial}
When processing large files, it is not always convenient to run in parallel but When processing large files, it is not always convenient to run in parallel but
process each parallel partition in serial, for example when interpolating a process each parallel partition in serial, for example when interpolating a
solution field from one mesh to another. solution field from one mesh to another or creating an output file for
visualization.
\subsection{Using the \texttt{nprocs} and \texttt{procid} options} \subsection{Using the \textit{nparts} options}
One option is to use the \inltt{--nprocs} and \inltt{--procid} command line One option is to use the \inltt{nparts} command line
options. For example, the following command will interpolate partition 2 of a option. For example, the following command will create a
decomposition into 10 partitions of \inltt{fiile2.xml} from \inltt{file1.fld} \inltt{.vtu} file using 10 partitions of \inltt{file1.xml}:
\begin{lstlisting}[style=BashInputStyle] \begin{lstlisting}[style=BashInputStyle]
FieldConvert --nprocs 10 --procid 2 \ FieldConvert --nparts 10 file1.xml file1.fld file1.vtu
-m interpfield:fromxml=file1.xml:fromfld=file1.fld \
file2.xml file2.fld
\end{lstlisting} \end{lstlisting}
This call will only therefore consider the interpolation process across one Note this will create a parallel vtu file as it processes each partition.
partition (namely, partition 2). To create the full interpolated field we can
suppress the \inltt{--procid} option, in which case FieldConvert will
loop over each of the partitions: Another example is to interpolate \inltt{file1.fld} from one mesh
\inltt{file1.xml} to another \inltt{file2.xml}. If the mesh files are
large we can do this by partitioning \inltt{file2.xml} into 10 (or more)
partitions and interpolating each partition one by one using the
command:
\begin{lstlisting}[style=BashInputStyle] \begin{lstlisting}[style=BashInputStyle]
FieldConvert --nprocs 10 \ FieldConvert --nparts 10 -m interpfield:fromxml=file1.xml:fromfld=file1.fld \
-m interpfield:fromxml=file1.xml:fromfld=file1.fld \ file2.xml file2.fld
file2.xml file2.fld
\end{lstlisting} \end{lstlisting}
Note that internally the routine uses the range option so that it
only has to load the part of \inltt{file1.xml} that overlaps with each
partition of \inltt{file2.xml}.
The resulting output will lie in a directory called \inltt{file2.fld}, with each The resulting output will lie in a directory called \inltt{file2.fld}, with each
of the different parallel partitions in files with names \inltt{P0000000.fld}, of the different parallel partitions in files with names \inltt{P0000000.fld},
\inltt{P0000001.fld}, \dots, \inltt{P0000009.fld}. This is nearly a complete \inltt{P0000001.fld}, \dots, \inltt{P0000009.fld}. This is nearly a complete
parallel field file. However, when the output file is in the .fld format, parallel field file. However, when the output file is in the .fld format,
the \inltt{Info.xml} file, which contains the information about which elements the \inltt{Info.xml} file, which contains the information about which elements
lie in each partition, is missing. This can be generated by using the command lie in each partition, is not correct since it will only contain the information for one of the partitions. The correct \inltt{Info.xml} file can be generated by using the command
\begin{lstlisting}[style=BashInputStyle] \begin{lstlisting}[style=BashInputStyle]
FieldConvert file2.xml file2.fld/Info.xml:info:nprocs=10 FieldConvert file2.xml file2.fld/Info.xml:info:nparts=10
\end{lstlisting} \end{lstlisting}
Note the \inltt{:info} extension on the last argument is necessary to tell Note the \inltt{:info} extension on the last argument is necessary to tell
FieldConvert that you wish to generate an info file, but with the extension FieldConvert that you wish to generate an info file, but with the extension
\inltt{.xml}. This syntax allows the routine not to get confused with the \inltt{.xml}. This syntax allows the routine not to get confused with the
input/output XML files. input/output XML files.
\subsection{Using the --part-only and --part-only-overlapping options} \subsection{Running in parallel with the \textit{ nparts} option}
The examples above will process each partition serially which may now
take a while for many partitions. You can however run this option in
parallel using a smaller number of cores than the nparts.
For the example of creating a vtu file above you can use 4 processor concurrently wiht the command line:
\begin{lstlisting}[style=BashInputStyle]
mpirun -n 4 FieldConvert --nparts 10 file1.xml file1.fld file1.vtu
\end{lstlisting}
Obviously the executable will have to have been compiled with the MPI option for this to work.
\subsection{Using the \textit{ part-only} and \textit{ part-only-overlapping} options}
Another approach to serially proessing a large file is to initially process the The options above will all load in the full \inltt{file1.xml}, partition
file into multiple partitions. This can be done with the \inltt{--part-only} it into \inltt{nparts} files in a director called \inltt{file1\_xml}.
This can be expensive if the \inltt{file1.xml} is already large. So instead you can
pre-partition the file using the using the \inltt{--part-only}
option. So the command option. So the command
\begin{lstlisting}[style=BashInputStyle] \begin{lstlisting}[style=BashInputStyle]
FieldConvert --part-only 10 file.xml file.fld FieldConvert --part-only 10 file.xml file.fld
\end{lstlisting} \end{lstlisting}
will partition the mesh into 10 paritions and write each partition into a will partition the mesh into 10 partitions and write each partition into a
directory called \inltt{file\_xml}. If you enter this directory you will find directory called \inltt{file\_xml}. If you enter this directory you will find
partitioned XML files \inltt{P0000000.xml}, \inltt{P0000001.xml}, \dots, partitioned XML files \inltt{P0000000.xml}, \inltt{P0000001.xml}, \dots,
\inltt{P0000009.xml} which can then be processed individually as outlined above. \inltt{P0000009.xml} which can then be processed individually as outlined above.
If you have a partitioned directory either from a parallel run or using the \inltt{--part-only} option you can now run the \inltt{FieldConvert} option using the command
\begin{lstlisting}[style=BashInputStyle]
mpirun -n 4 FieldConvert --nparts 10 file1\_xml:xml file1.fld file1.vtu
\end{lstlisting}
Note the form \inltt{file1\_xml:xml} option tells the code it is a parallel partition which should be treated as an \inltt{xml} type file.
There is also a \inltt{--part-only-overlapping} option, which can be run in the There is also a \inltt{--part-only-overlapping} option, which can be run in the
same fashion. same fashion.
\begin{lstlisting}[style=BashInputStyle] \begin{lstlisting}[style=BashInputStyle]
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment