Nektar merge requestshttps://gitlab.nektar.info/nektar/nektar/-/merge_requests2024-03-28T20:50:02Zhttps://gitlab.nektar.info/nektar/nektar/-/merge_requests/1784Implement implicit time-discretization in SWE solvers2024-03-28T20:50:02ZJacques XingImplement implicit time-discretization in SWE solvers# Issue/feature addressed
- Implement implicit time-stepping (JFNK) for SWE
- Move `CopyBoundaryTrace` from base class to `LinearSWE`
- Add `const` keyword when appropriate
- Remove unused function `WallBoundary`
- Fix minor inconsiste...# Issue/feature addressed
- Implement implicit time-stepping (JFNK) for SWE
- Move `CopyBoundaryTrace` from base class to `LinearSWE`
- Add `const` keyword when appropriate
- Remove unused function `WallBoundary`
- Fix minor inconsistencies between derived classes
- Tidy-up implementation of the `NonlinearPeregrine` solver
# Proposed solution
# Implementation
## Tests
`solvers/ShallowWaterSolver/Tests/NonlinearSWE_Peregrine_SolitaryWave_DG_P6_implicit.xml`
# Suggested reviewers
*Please suggest any people who would be appropriate to review your code.*
# Notes
*Please add any other information that could be useful for reviewers.*
# Checklist
- [x] Functions and classes, or changes to them, are documented.
- ~~[ ] User guide/documentation is updated.~~
- [x] Changelog is updated.
- [x] Suitable tests added for new functionality.
- [x] Contributed code is correctly formatted. (See the [contributing guidelines](https://gitlab.nektar.info/nektar/nektar/-/blob/master/CONTRIBUTING.md#using-clang-format)).
- ~~[ ] License added to any new files.~~
- [x] No extraneous files have been added (e.g. compiler output or test data files).v5.6.0Dave MoxeyDave Moxeyhttps://gitlab.nektar.info/nektar/nektar/-/merge_requests/1783Remove copy entire input and output to padded aligned arrays in MatrixFree op...2024-03-28T11:08:34ZBOYANG XIARemove copy entire input and output to padded aligned arrays in MatrixFree operators# Issue/feature addressed
_In the previous design, the matrix-free operator `Helmholtz_MatrixFree` copies the input array into the internal padded aligned storage `MatrixFreeOneInOneOut::m_input` before the real task starts. This redund...# Issue/feature addressed
_In the previous design, the matrix-free operator `Helmholtz_MatrixFree` copies the input array into the internal padded aligned storage `MatrixFreeOneInOneOut::m_input` before the real task starts. This redundant large copy significantly slows down the speed._
# Proposed solution
_Remove the copy operation and let operators in MatrixFreeOps support unaligned memory. After this change, the legacy collection matrix-free operators should achieve comparable performance as the redesign matrix-free operators._
# Implementation
The initial design is:
* _Use an aligned local array as an intermediate storage._
* _Always copy data to aligned local storage before `load_interleave` and `deinterleave`_
* _The last sub-block, which may contain paddings, will be processed in a separate code block._
An updated design is:
* Add a new function `load_unalign_interleave`, which can directly access unaligned memory. So we don't need to copy to an aligned local array.
Here are the benchmark results:
![image.png](/uploads/acfc1d9765c517ff47a1bf48b052d1a9/image.png){width="389" height="259"}![image.png](/uploads/920333c71b6fef8c93f055f0ecc783d5/image.png){width=380 height=249}
Although there are noises or errors, we can still get some general conclusions:
* The legacy collection matrix-free is much slower than the redesign, except for hex-7, which requires no padding (no copy) by coincidence.
* The update design is slightly faster than the initial design by further reducing the memory loads.
## Tests
No new tests are required.
# Suggested reviewers
@dmoxey @ccantwel
# Notes
_A complete change for all operators takes a lot of effort. So we first apply changes to `Helmholtz` then to the others._
_Everyone especially the MatrixFreeOps contributors are welcome to join in this MR._
_The`MatrixFreeBase` the class can be removed after all the operators are updated._
# Checklist
- [x] Functions and classes, or changes to them, are documented.
- [ ] ~~User guide/documentation is updated.~~
- [ ] Changelog is updated.
- [ ] ~~Suitable tests added for new functionality.~~
- [x] Contributed code is correctly formatted. (See the [contributing guidelines](https://gitlab.nektar.info/nektar/nektar/-/blob/master/CONTRIBUTING.md#using-clang-format)).
- [ ] ~~License added to any new files.~~
- [x] No extraneous files have been added (e.g. compiler output or test data files).https://gitlab.nektar.info/nektar/nektar/-/merge_requests/1780Draft: Feature/scaling mesh2024-03-21T13:31:25ZKaloyan KirilovDraft: Feature/scaling mesh# Issue/feature addressed
*Scales the coordinates of the mesh (Linear or High-order) with user-defined scaled coefficient. This is useful when users have CAD or have created the high-order/linear mesh with the wrong dimensions (mm vs m)....# Issue/feature addressed
*Scales the coordinates of the mesh (Linear or High-order) with user-defined scaled coefficient. This is useful when users have CAD or have created the high-order/linear mesh with the wrong dimensions (mm vs m). Several users have requested such functionality. *
# Proposed solution
*A very simple module that takes X Y Z coefficients and scales the coordinates (vertices, edgeNodes, FaceNodes). *
# Implementation
*A very simple module that takes scaleX scaleY scaleZ coefficients as an input and scales the coordinates of the (vertices, edgeNodes, FaceNodes). It has 3 loops over vertexSet, edgeSet, faceSet. If no coefficient is given by the user, no scaling is performed in this direction.*
## Tests
# Suggested reviewers
**
# Notes
*Please add any other information that could be useful for reviewers.*
# Checklist
- [X] Functions and classes, or changes to them, are documented.
- [ ] User guide/documentation is updated.
- [ ] Changelog is updated.
- [ ] Suitable tests added for new functionality.
- [x] Contributed code is correctly formatted. (See the [contributing guidelines](https://gitlab.nektar.info/nektar/nektar/-/blob/master/CONTRIBUTING.md#using-clang-format)).
- [x] License added to any new files.
- [x] No extraneous files have been added (e.g. compiler output or test data files).https://gitlab.nektar.info/nektar/nektar/-/merge_requests/1778Separate MeshGraph input/output functions into a new class2024-03-28T15:11:44ZTed StokesSeparate MeshGraph input/output functions into a new class# Issue/feature addressed
Once created, it shouldn't matter which file format a MeshGraph instance was loaded from. However because the format specific subclasses inherit from MeshGraph, it isn't trivial to read from an XML then write t...# Issue/feature addressed
Once created, it shouldn't matter which file format a MeshGraph instance was loaded from. However because the format specific subclasses inherit from MeshGraph, it isn't trivial to read from an XML then write to an HDF5 file.
# Proposed solution
Extract MeshGraph's input/output functions and put them in a new base class MeshGraphIO which has a subclass for each file format, so that MeshGraph is agnostic to the file format it is read from or written to.
# Implementation
The MeshGraph class has had its IO functions trimmed out, and instead of MeshGraphXml, MeshGraphXmlCOmpressed and MeshGraphHDF5 there is MeshGraphIOXml, MeshGraphIOXmlCompressed and MeshGraphIOHDF5, which inherit from MeshGraphIO. A MeshGraph is now read by calling MeshGraph**_IO_**\[format\]::Read.
# Tests
# Suggested reviewers
@dmoxey
# Notes
# Checklist
- [x] Functions and classes, or changes to them, are documented.
- [ ] User guide/documentation is updated.
- [x] Changelog is updated.
- [ ] ~~Suitable tests added for new functionality.~~
- [x] Contributed code is correctly formatted. (See the [contributing guidelines](https://gitlab.nektar.info/nektar/nektar/-/blob/master/CONTRIBUTING.md#using-clang-format)).
- [x] License added to any new files.
- [x] No extraneous files have been added (e.g. compiler output or test data files).https://gitlab.nektar.info/nektar/nektar/-/merge_requests/1774Draft: Addition of general MemoryRegion2024-03-27T00:19:43ZAllen SandersonDraft: Addition of general MemoryRegion# Issue/feature addressed
*Currently the Field class has member variable that is a MemoryRegionCPU with several ancillary methods that work on the variable. This member variable and the associated ancillary methods need to be generalized...# Issue/feature addressed
*Currently the Field class has member variable that is a MemoryRegionCPU with several ancillary methods that work on the variable. This member variable and the associated ancillary methods need to be generalized for general host/device usage.*
# Proposed solution
*A new class MemoryRegion has been introduced which the Field inherits from. At the same time this class is used in the operators so allow host/device usage.
Importantly this class uses the notion of valid host/device data rather than the notion of whether the data is "on" the device which will not work.
At the same time I have introduced the concept of execution and memory spaces which is ubiquitous in Kokkos and I believe should be moved into the base operator class. It allows for a more general way of understanding where an functor is executing and the location of the memory.
When combined with the MemoryRegion one generically asks for a pointer to memory. Depending on the memory space of the operator the appropriate data is returned.
I have utilized and tested the above paradigm in the BwdTransMatFree for the Field and Basis data. One can see it via the USE_MEMORY_REGION macro in library/Operators/BwdTrans/BwdTransMatFree.hpp and library/Operators/BwdTrans/BwdTransMatFreeKernel.hpp
I have utilized but not tested the above paradigm in the BwdTransCUDA for the Field and Basis data.
I have also introduced storing the basis data via the BasisKey which is similar to what is done in the CUDA implementations.
Note the MemoryRegion in BwdTransMatFree uses a vec_t whereas the BwdTransCUDA uses a TData (double).
I have also done some cleanup in the *MatFree* so to remove m_basis which was redundant given the expansion list is also stored.
*
# Implementation
*A more detailed description of the changes. Please include any details necessary for reviewers to understand your work, and point out particular points would like them to pay particular attention to.*
## Tests
# Suggested reviewers
*Please suggest any people who would be appropriate to review your code.*
# Notes
*Please add any other information that could be useful for reviewers.*
# Checklist
- [ ] Functions and classes, or changes to them, are documented.
- [ ] User guide/documentation is updated.
- [ ] Changelog is updated.
- [ ] Suitable tests added for new functionality.
- [ ] Contributed code is correctly formatted. (See the [contributing guidelines](https://gitlab.nektar.info/nektar/nektar/-/blob/master/CONTRIBUTING.md#using-clang-format)).
- [ ] License added to any new files.
- [ ] No extraneous files have been added (e.g. compiler output or test data files).https://gitlab.nektar.info/nektar/nektar/-/merge_requests/1772Feature/redesign helmholtzmatfree2024-03-28T23:23:03ZBOYANG XIAFeature/redesign helmholtzmatfree# Issue/feature addressed
_Add HelmholtzMatFree implementation, unit tests, and also the profiler._
# Proposed solution
# Implementation
## Tests
# Suggested reviewers
# Notes
The previous profiler merge request was closed and the...# Issue/feature addressed
_Add HelmholtzMatFree implementation, unit tests, and also the profiler._
# Proposed solution
# Implementation
## Tests
# Suggested reviewers
# Notes
The previous profiler merge request was closed and the contents are included here.
Sample command to run the profiler:
```plaintext
likwid-mpirun -hostfile ~/hostfile -nperdomain S:18 -m -g MEM_DP $PATH_TO_PROFILER/ProfilerBwdTrans -v -P Ntest=1000 -P order=5 $PATH_TO_MESH/hex-5.xml
```
# Checklist
- [x] Functions and classes, or changes to them, are documented.
- [ ] ~~User guide/documentation is updated.~~
- [ ] ~~Changelog is updated.~~
- [x] Suitable tests added for new functionality.
- [x] Contributed code is correctly formatted. (See the [contributing guidelines](https://gitlab.nektar.info/nektar/nektar/-/blob/master/CONTRIBUTING.md#using-clang-format)).
- [x] License added to any new files.
- [x] No extraneous files have been added (e.g. compiler output or test data files).https://gitlab.nektar.info/nektar/nektar/-/merge_requests/1760Isoparametric h-adaptation in 2D ( high order split and swapping )2024-03-20T08:04:41ZKaloyan KirilovIsoparametric h-adaptation in 2D ( high order split and swapping )# Feature addressed
_This module allows for isoparametric 2D high-order h-adaptation (splitting and swapping). It does split both triangular elements with all possible maps. The quadrilateral elements in a hybrid mesh are also split wit...# Feature addressed
_This module allows for isoparametric 2D high-order h-adaptation (splitting and swapping). It does split both triangular elements with all possible maps. The quadrilateral elements in a hybrid mesh are also split with respect to their adjacent split triangles to ensure conformity._
# Proposed solution
The module uses the concept of Isoparametric splitting based on predefined maps that are defined in standard space (reference space). This way the geometrical accuracy and validity of the element is conserved without the need for the CAD itself or CAD-API queries. If the element is straight-sided, we do perform a standard splitting, hence NO expensive backwards and forward mapping between reference and physical spaces are performed.
# Implementation
The module uses a hierarchical approach where.
1. Process the input options for splitting
* User-defined element IDs
* Uniform (all elements)
* Based on a scale field ( same approach with interpolation as VarOpti r-scaling )
* Finest(use map that split 1:4) for all selected elements to be split
2. Identifies the edges to be splitting
* Longest edge
* User-defined IDs
* Uniform (all edges)
* Once edge is split - add the adjacent element (m_elLink) to the list of elements to be split).
* When the adjacent element is Quad - we propagate the splitting to the opposite edge and until we reach m_elLink= 0 ( boundary)
3. Splits isoparametrically the edges. ( Create Hanging Nodes and New Edges)
* Assign a new member to the Edge-\>m_childEdges Edge-\>m_hangingNode
* Construct the meshgraph pointer of the edge and populate back the edgeNodes
* If linear - split in the middle and do not construct the meshgraph point.
4. Loop over the selected and adjacent elements and split.
1. Identify the relevant map
2. Split according to the map (create the internal edges)
3. If linear - just connect the hanging node with the opposite vertex (Much cheaper - No - Backwards and Forward transform).
4. If the element has an edge that is on the boundary, create m_element\[1\] edge element for the boundary.
5. Swapping (optional, robust improvement)
1. Loop over the adapted elements
2. Check ratios between local edges
3. If large -\> perform swapping and check if the quality is better
1. Select the largest edge and the adjacent element
2. Create new Quad element with the other 4 edges
3. Create the isoparametric representation of this quad element
4. Create the new alternative diagonal.
5. Create the new triangular elements.
## Tests
# Suggested reviewers
@dmoxey , @mgreen
_Please add any other information that could be useful for reviewers._
# Checklist
- [x] Functions and classes, or changes to them, are documented.
- [x] User guide/documentation is updated.
- [x] Changelog is updated.
- [x] Suitable tests added for new functionality.
- [x] Contributed code is correctly formatted. (See the [contributing guidelines](https://gitlab.nektar.info/nektar/nektar/-/blob/master/CONTRIBUTING.md#using-clang-format)).
- [x] License added to any new files.
- [x] No extraneous files have been added (e.g. compiler output or test data files).https://gitlab.nektar.info/nektar/nektar/-/merge_requests/1759Add support for Multiscale Universal Interface (MUI) library2024-02-26T11:26:18ZChris CantwellAdd support for Multiscale Universal Interface (MUI) library# Issue/feature addressed
This adds support for coupling using the Multiscale Universal Interface (MUI) library. This library allows data on grids with different resolutions to be exchanged between two solvers which may be advancing at d...# Issue/feature addressed
This adds support for coupling using the Multiscale Universal Interface (MUI) library. This library allows data on grids with different resolutions to be exchanged between two solvers which may be advancing at different time-steps. The coupling interface handles the interpolation between these grids and the different time-step sizes.
# Proposed solution
In this MR we add support for linking with the MUI library and demonstrate the interface through a new equation system in the DummySolver, extending the earlier work to add the Cwipi coupling library.
# Implementation
At the moment, the MUI interface is demonstrated only in the DummySolver, with the exchange being hard-coded in this solver. Details of how the exchange is managed (which application talks to which other application, which variables are exchanged, etc) are handled through a dedicated `<COUPLING>` section of the XML input file. In particular, this provides details about which variables are to be sent to another application, the names of the variables into which information should be received, as well as the original names of these variables.
## Tests
A test `Dummy_2DTestMUI` is provided which runs two separate instances of the DummySolver (each with their individual input files) and demonstrates the exchange of fields of data between them.
# Suggested reviewers
@dmoxey
# Notes
The majority of the implementation for this work was completed by Stefano Rolfo at STFC.
# Checklist
- [X] Functions and classes, or changes to them, are documented.
- [x] User guide/documentation is updated.
- [x] Changelog is updated.
- [X] Suitable tests added for new functionality.
- [X] Contributed code is correctly formatted. (See the [contributing guidelines](https://gitlab.nektar.info/nektar/nektar/-/blob/master/CONTRIBUTING.md#using-clang-format)).
- [X] License added to any new files.
- [X] No extraneous files have been added (e.g. compiler output or test data files).v5.6.0Chris CantwellChris Cantwellhttps://gitlab.nektar.info/nektar/nektar/-/merge_requests/1758Draft: Variance filter2024-02-11T15:55:34ZChris CantwellDraft: Variance filter# Issue/feature addressed
This adds a variance filter.
# Proposed solution
*A summary of the proposed changes and how they address the issue/feature at hand. Whenever possible, describe alternatives your considered and decisions you mad...# Issue/feature addressed
This adds a variance filter.
# Proposed solution
*A summary of the proposed changes and how they address the issue/feature at hand. Whenever possible, describe alternatives your considered and decisions you made.*
# Implementation
*A more detailed description of the changes. Please include any details necessary for reviewers to understand your work, and point out particular points would like them to pay particular attention to.*
## Tests
# Suggested reviewers
*Please suggest any people who would be appropriate to review your code.*
# Notes
*Please add any other information that could be useful for reviewers.*
# Checklist
- [ ] Functions and classes, or changes to them, are documented.
- [ ] User guide/documentation is updated.
- [ ] Changelog is updated.
- [ ] Suitable tests added for new functionality.
- [ ] Contributed code is correctly formatted. (See the [contributing guidelines](https://gitlab.nektar.info/nektar/nektar/-/blob/master/CONTRIBUTING.md#using-clang-format)).
- [ ] License added to any new files.
- [ ] No extraneous files have been added (e.g. compiler output or test data files).Chris CantwellChris Cantwellhttps://gitlab.nektar.info/nektar/nektar/-/merge_requests/1757Draft: ProcessBL new implementation that retains both face(FaceNodes) and edg...2024-02-23T14:30:12ZKaloyan KirilovDraft: ProcessBL new implementation that retains both face(FaceNodes) and edge (EdgeNodes) curvature.# Issue/feature addressed
The legacy implementation of the ProcessBL directly threw away/deleted the face curvature (faceNodes) and relied on propagating only the edge curvature (EdgeNodes). This significantly deteriorates the geometrica...# Issue/feature addressed
The legacy implementation of the ProcessBL directly threw away/deleted the face curvature (faceNodes) and relied on propagating only the edge curvature (EdgeNodes). This significantly deteriorates the geometrical accuracy of the meshes, especially in regions with high or non-uniform curvature. Additionally, this creates a very high lower bound of the geometrical error of the isoparametric surface, that does not improve even if a very high-order polynomial mesh is chosen, since no face-nodes are present. The legacy implementation will also not warn the user if a pyramid is present in the mesh and will spit out a non-conformal mesh.
# Proposed solution
A completely new implementation of the module was developed, which retains the input same parameters and the fundamental methodology as proposed in Moxey et all (2015). A local split of the isoparametric prism is performed in the reference space of the element and curvature is propagated to the user-chosen settings. Both the EDGENODES, but now also the FACENODES are propagated according to the user-defined number of layers and progression ratio (r).
Summary from user perspective:
- The same user config options are retained (nq,r,surf,layers).
- The user should NOT see any difference between using this or the legacy ProcessBL for prisms.
3D:
- Supports complete O-type BLs prisms meshes for 3D.
- Warns the user if Pyramid is present.
- NO Hex splitting is implemented. I have not tested it but I believe the legacy module have it.
2D:
- NO CHANGE in the 2D implementation.
- Supports quad splitting, but also has some triangular splitting maps present ( no difference in the implementation.)
# Implementation
0. The first part of the legacy code, where the subset of elements relevant to the user chose composites is kept as the legacy code.
1. Create a copy vector of the edges inside the PrismLayer (remember this will be split).
2. Perform isoparametric edge splitting and fill the PrismEdgesCopy with the desired BL distribution and correct directionality
2.1 Create the edge expansion
2.2 Split the macro edge with the bl distribtion and create new child vertices
2.3 creates a copy of the Macro Edge that contains the new(split) vertices (on the curved master edge) as EdgeNodes having edgeType - eBoundaryLayerPoints/Rev
3. Loop over the prism elements on the surface and Split locally on the desired surfaces/composites (as per 0.)
3.0 Create the expansion of the Macro Prism Element
3.1 Identify the orientation of the prism with respect to the surface (face 1 or face 3 point the boundary)
3.2 Create Vector of Vertices for the Copy Edges - always starting from the Boundary/Curved Face !
3.3 Create HO Elements starting from m_n1 !!!
3.3.1. Create the NodeList
3.3.2 Create the Linear Element
3.3.3 For the inner layers create HO EdgeNodes, Populate edge nodes to the correct predefined new local edges OR reuse the existing macro edges if layer 0
3.3.4 Populate the facenodes to the new local faces by using the MACRO element expansion ( physeval) xp[1] comes from the bl settings.
3.3.5 For the first or last layer try to reuse edges and Face directly from el_macro
3.4. For the first layer - create the m_element[2] triag or quad and put in the correcct composite.
3.4 Add the element in m_mesh->m_element[3]
## Tests
All ProcessBL tests look for completion. A more precise test will be creating a composite test with the geometrical accuracy module created for ICOSAHOM, this will be TBD.
New tests c:
- Semi-sphere
- NACA0012
- IFW_extruded50mm
# Suggested reviewers
@mgreen , @dmoxey
# Notes
*Please add any other information that could be useful for reviewers.*
# Checklist
- [x] Functions and classes, or changes to them, are documented.
- [x] User guide/documentation is updated.
- [ ] Changelog is updated.
- [ ] Suitable tests added for new functionality.
- [ ] Contributed code is correctly formatted. (See the [contributing guidelines](https://gitlab.nektar.info/nektar/nektar/-/blob/master/CONTRIBUTING.md#using-clang-format)).
- [x] License added to any new files.
- [x] No extraneous files have been added (e.g. compiler output or test data files).v5.6.0Mohsen LahootiMohsen Lahootihttps://gitlab.nektar.info/nektar/nektar/-/merge_requests/1754Feature/redesign/execution2024-03-27T11:22:57ZJames Edgeleyjames.edgeley@ukaea.ukFeature/redesign/execution# Issue/feature addressed
Addition of Execution library to provide a DAG execution structure for efficient ordering of the redesign operators
# Proposed solution
This MR adds the standalone library but does not connect it to other code....# Issue/feature addressed
Addition of Execution library to provide a DAG execution structure for efficient ordering of the redesign operators
# Proposed solution
This MR adds the standalone library but does not connect it to other code. Currently requires c++20
# Implementation
The Execution library aims to improve performance of Nektar++ operations by reducing them to bare-bones kernels, arranging the kernels into a [directed acyclic graph (DAG)](https://en.wikipedia.org/wiki/Directed_acyclic_graph) which respects their dependencies, and then executing the kernels using the full range of hardware available to the user.
Part of the optimisation comes from avoiding unnecessary copies and allocations that would otherwise be repeated for every time step or iteration in a solver, at the cost of a single graph setup phase.
Further performance improvements can come from spreading the workload over a range of devices, as in some cases it is possible to split one operation into several nodes, which can be executed on different devices, using the optimal algorithm for the particular combination of device, geometry and order.
Currently the library offers a choice of backends of type `ExecutionModel`, which determine the specialisations for a number of key template classes. The chosen backend is held in the `Execution_t` variable. The options are:
- `SerialExecutionModel`: Nodes are executed sequentially, on the CPU
- `ThreadExecutionModel`: Nodes may be executed in parallel if possible, using CPU threads
- `SyclExecutionModel` (WIP): Nodes are executed by a SYCL queue.
- `CudaExecutionModel`: Nodes are executed in parallel on the GPU and CPU. Nodes on the GPU can contain a Cuda kernel, providing multiple levels of parallelism
- `DebugExecutionModel`: There is no graph; kernels are instead executed as they are added. May not provide the correct result if nodes are added out of order
The important template classes for this library are as follows:
- `GraphDetail`: This class contains a list of `NodeDetail` objects, and information about their connectivity.
- `NodeDetail`: This class typically contains a kernel, or another operation such as a copy or allocation. The type of operation (kernel or otherwise) is held in a `std::variant` The operation in the node is performed only when all prior connected nodes have completed execution.
The typedefs `Node_t = NodeDetail<Execution_t>` and `Graph_t = GraphDetail<Execution_t>` are frequently employed.
All `GraphDetail` specialisations have functionality in common:
- A call to `Launch()` commences execution of the graph structure, starting with the root nodes.
- A call to `Wait()` prevents further progression through the program until all of the leaf nodes have completed.
- A call to `Print(std::ostream &os)` writes the structure of the graph to the `os` stream in DOT format.
- `AddDependency(Node_t *pre, Node_t *suc)` adds an edge from `*pre` to `*suc`
`GraphDetail<ThreadExecutionModel>` and `GraphDetail<SerialExecutionModel>` contain an instance of `Executor` template class which handles the specifics of the execution ordering.
`GraphDetail<CudaExecutionModel>` contains a `CUGraph`, which handles the execution using the CUDA driver API's own graph functionality.
Objects referred to in the graph (for example the arguments of any kernels) must remain in scope until completion of execution, otherwise the kernel will not be able to find them when it is executed.
Calls to MPI functions are also possible to allow for communication between graphs on different ranks - there are dedicated `NodeDetail` variants for this purpose.
In `CudaExecutionModel` there is an assigned thread on the CPU that repeatedly checks for a memory flag whose value indicates that whether it should make an MPI call.
CUDA memory operations are used to write and wait on this flag to signal when the CPU should make the call, and when the call has been made (this feature currently necessitates the use of the driver API).
It is also possible to condense a graph into a subgraph node in another graph - this is useful for composite operators. With some restrictions, it is possible to mix execution models.
For instance, a `GraphDetail<ThreadExecutionModel>` can be subsumed into a `NodeDetail<CudaExecutionModel>` in a `GraphDetail<CudaExecutionModel>`. This allows work to be performed simultaneously on CPU and GPU.
The variable `HostExecution_t` determines which `ExecutionModel` (`SerialExecutionModel` or `ThreadExecutionModel`) should be used by `CudaExecutionModel` for host side operations.
The library also provides memory management classes:
- `MemoryRegion` which contains the data, and a pointer to a `MemoryResource`
- `MemoryResource` which governs how memory is allocated, copied and freed. It has several child classes, depending on the purpose of the `MemoryRegion`s or operations that use it.
For instance, users may use
- `CudaPinnedMemory<CudaHostAllocWriter::HostWritten>>` to stage host-side data that will be copied to a GPU,
- then `CudaMemory` for all the intermediate device-side operations
- and then `CudaPinnedMemory<CudaHostAllocWriter::DeviceWritten>>` for host-side data that is copied back from the host after execution.
It is also possible to perform interleaving during copies, in case the user wishes to use vectorisation.
## Tests
Some basic tests have been added to UnitTests
# Suggested reviewers
*Please suggest any people who would be appropriate to review your code.*
# Notes
*Please add any other information that could be useful for reviewers.*
# Checklist
- [ ] Functions and classes, or changes to them, are documented.
- [ ] User guide/documentation is updated.
- [ ] Changelog is updated.
- [x] Suitable tests added for new functionality.
- [x] Contributed code is correctly formatted. (See the [contributing guidelines](https://gitlab.nektar.info/nektar/nektar/-/blob/master/CONTRIBUTING.md#using-clang-format)).
- [x] License added to any new files.
- [x] No extraneous files have been added (e.g. compiler output or test data files).https://gitlab.nektar.info/nektar/nektar/-/merge_requests/1751New feature for ProcessJac in NekMesh. Output Jacobian information on the scr...2024-03-13T13:43:26ZJingtian ZhouNew feature for ProcessJac in NekMesh. Output Jacobian information on the screen and into the log file# Issue/feature addressed
*Histogram now can be plotted internally in ProcessJac of NekMesh. The Jacobian information, element IDs, connected boundary surface/edge IDs, coordinates of vertex are outputted into a text file*
# Proposed so...# Issue/feature addressed
*Histogram now can be plotted internally in ProcessJac of NekMesh. The Jacobian information, element IDs, connected boundary surface/edge IDs, coordinates of vertex are outputted into a text file*
# Proposed solution
*A new option called "histo" is available to the user, which takes three value: The maximum value of Jacobian to be plotted, the number of positive bins in the histogram, the number of negative bins in the histogram. The Jacobians below the threshold will be counted and a histogram will be plotted on the screen. This helps the user to check the quality of the mesh visually. The Jacobian information, element IDs, boundary surface/edge IDs, vertex coordinates, neighbor element IDs will also be outputted into a text file. The text file can also be further used to create a histogram picture by a python script.*
*Another option called "quality" is introduced, which will show the distribution of Jacobians on the screen including the percentage of each bin of on-screen histogram. This helps the user to check the quality of the mesh in detail.*
# Implementation
*The users can now use the following command to plot a histogram of Jacobian on the screen and output a text file with Jacobian information:*
![Command_histo](/uploads/caf33ccfb5a26cda3e25faa8c8514199/Command_histo.png)
*The x-axis of the histogram is the Jaobian, and the y-axis of the histogram is the number of elements. Option "histo" takes three values, separated by a comma. "value1" defines the maximum value of the Jacobian shown on the histogram. In other words, only elements with Jacobian below "value1" will be counted and will be shown on the histogram and will be outputted to a text file. "value2" is the number of bins with positive Jacobian values on the histogram. "value3" is the number of bins with negative Jacobian values on the histogram. The width of each bin is determined by "value1" and "value2": "value1/value2". The maximum value of the histogram is "value1", while the minimum value of the histogram is: "(value1/value2)\*value3". It should be noted that the elements with Jacobian smaller than the minimum value will also be counted and be classified into the left-most bin. By default, the three values are 1,10,1. This means that the default histogram shows the Jacobian between 0.0 and 1.0, with a interval 0.1, and all the other elements with a Jacobian smaller than 1 will be counted and shown on the left-most bin with a name "invalid".*
*"histofile" is an option to set the name of the outputted file with jacobian information. By default, the name of the text file with Jacobian and other elemental information is "jacs.txt"*
*"detail" is another option which will also calculate the composite name and ID of each element in the outputted text file. This process takes long time when the mesh is big and not all the user need this information. And therefore this function is added using another tag.*
*Note that the output.xml is now updated, only the elements with Jacobian smaller than the threshold "value1" will be outputted. This will further help the user to find the elements with a bad quality.*
*The users can now use the following command to show the distribution of jacobians on the screen:*
![Command_quality](/uploads/593d7aed09a6c8bef0d7084eec5b38d6/Command_quality.png)
*This function will list the distribution of the jacobian in the histogram, including the percentage of each bin. This can be regarded as the text description of the histogram, and therefore, the results of "quality" will correspond to the histogram. If "histo" is not set by user, the "quality" will follow the default value of "histo" and show the distribution between 0.0 and 1.0 with an interval 0.1.*
*"quality" also calculate the integration of Jacobians using the following equation:*
$$\frac{\sum_{i=1}^{N}\sum_{j=1}^{Q_i}w_jJ_j*J_{Scaled_i}}{\sum_{i=1}^{N}\sum_{j=1}^{Q_i}w_jJ_j}$$
*$N$ is the number of elements of the mesh and $Q_i$ is the number of quadrature points in each element, $\omega_j$ is the weight function and $J_j$ is the Jacobian of the mapping from reference space to the physical space. The denominator is the area(volume) of the mesh. The $J_{scaled}$ is the scaled Jacobian of each element, calculated by the ratio between the minimum Jacobian of all the quadrature points and the maximum Jacobian of all quadrature points. The $J_{scaled}$ is a variable below 1.0, and will be smaller than zero when tangled, and all the Jacobians on the histogram and the outputted file are also calculated this way. The more $J_{scaled}$ is closer to 1.0, the better the mesh quality is. Therefore the above ratio is a value between 0.0% and 100% which evaluate the quality of the mesh.*
## Tests
# 1. 2D half circle test
*The default results of this module is tested first, using the following command:*
<img src="/uploads/01937f32bf4623ee405d83d842469d4e/Command_ring_default.png">
*The results on the terminal screen are shown in the following picture, which is the default setting of option "histo" and "quality".It is seen that the histogram is shown on the screen, with a default scale between (0.0, 1.0). The elements with negative Jacobians are classified into one bin named as invalid. The information below the histogram is the results of quality, which is a text description of the histogram. It shows the percentage of each bin. At the end of the module, the Worst Jacobian is shown and the integration of the Jacobian over the whole mesh is calculated.*
<div align="center"><img src="/uploads/42313e6d842aed6d5bffb2439e9d46bd/histo_default.png" width="60%"></div>
*Below is the outputted Jacobian information file. The first three columns are ID of elements, Jacobian value of each element, element type. If this element is connected to a boundary, the boundary edge(face for 3D) ID will also be shown, following which is the vertex IDs and coordinates of each edge. When we have a bad element with a small Jacobian, these info can help the user to find the location of this element and find the edge and vertex IDs easily. As we set the "detail" option, the composite ID and name are also shown. The last column is the Id of the neighbor elements of each element. The information of each element is divided by "===" symbols for readability. As we set "histofile" to "output.txt", the name of the text file is modified instead of the default one.*
<div align="center"><img src="/uploads/b921d53f4dd07a5935fdc1f11860fa90/ring_default_output.png"></div>
*Another case with the same mesh using different "histo" values are also tested. The command we use is listed below. *
![Command_ring](/uploads/2e0f4621867883fdae7629d09d7b52c5/Command_ring.png)
*The maximum Jacobian we evaluated is set to 0.2, and the number of positive bins are 3, and the number of negative bins are 2. The screenshot is listed below. It is seen that only elements with a Jacobian below 0.2 is counted. There are 3 positive bins and 2 negative bins. The width of each bin is (0.2/3). The results of quality match that of the histogram.*
<div align="center"><img src="/uploads/aea52a33610939a4fa2fdd3b52a0a8a8/histo_ring.png" width="60%"></div>
*The output file of this command is also listed. Only the elements with Jacobian lower than 0.2 are outputted, which match the "histo" setting.*
![ring_output](/uploads/946a002ee218aea4c8f62f406bb042bc/ring_output.png)
# 2. 3D half sphere test
*The default results using the following command are as follows:*
![Command_sphere_default](/uploads/c71d5f67e477faceb5763f68c88f5001/Command_sphere_default.png)
*The screenshot is listed below. The results of 3D are similar to that of 2D half circle, and are not described in detail here.*
<div align="center"><img src="/uploads/75ab55dac4ae4ab68411a47a73a9a9b2/sphere_default.png" width="60%"></div>
*The output file is listed below. It can be seen that for 3D meshes, one more column with "boundary face ID" is added, for a tetrahedral element, the boundary face is a triangle and has three boundary edges, the boundary edge IDs and the vertex coordinates of them are also shown. In this case the composite name of the mesh is not set during generation, and is shown as NONE instead.*
<div align="center"><img src="/uploads/76b0152a364d0506e4a8dd6faa64d2fb/sphere_Output_default.png" width="80%"></div>
*Another case with the same mesh using different "histo" values are also tested. The command we use is listed below.*
![Command_sphere](/uploads/7bf88b74de4fd64a7e1136a424505f92/Command_sphere.png)
*The results on the screen are below. The histogram is separated with 3 positive bins and 2 negative bins. It is seen that we only have 2 elements which has a Jacobian below 0.5. The results of quality also match the histogram.*
![sphere_histo](/uploads/bdc635a3f9091cf9230323392d8d6ac9/sphere_histo.png)
*The output file is shown below. There are only two elements outputted and one of them is connected to a boundary and the boundary surface , edge, and vertex info are also outputted.*
![sphere_output](/uploads/8a130a65e3ccb9dd832573c1f7b3179c/sphere_output.png)
# Suggested reviewers
*Please suggest any people who would be appropriate to review your code.*
# Notes
*Please add any other information that could be useful for reviewers.*
# Checklist
- [x] Functions and classes, or changes to them, are documented.
- [x] User guide/documentation is updated.
- [x] Changelog is updated.
- [x] Suitable tests added for new functionality.
- [ ] Contributed code is correctly formatted. (See the [contributing guidelines](https://gitlab.nektar.info/nektar/nektar/-/blob/master/CONTRIBUTING.md#using-clang-format)).
- [x] License added to any new files.
- [ ] No extraneous files have been added (e.g. compiler output or test data files).v5.6.0Mashy Greenmashy.green@kcl.ac.ukMashy Greenmashy.green@kcl.ac.ukhttps://gitlab.nektar.info/nektar/nektar/-/merge_requests/1743Draft: Redesign of Nektar++ core data structures and algorithms2024-03-28T08:59:37ZChris CantwellDraft: Redesign of Nektar++ core data structures and algorithms# Issue/feature addressed
The existing Nektar++ design prevented the efficient use of vectorisation, GPUs and other current and future platforms. Assumptions made in the data structures did not allow for the reorganisation of memory to s...# Issue/feature addressed
The existing Nektar++ design prevented the efficient use of vectorisation, GPUs and other current and future platforms. Assumptions made in the data structures did not allow for the reorganisation of memory to support vectorisation, or the use of memory on attached devices. Algorithms were tightly coupled to the `ExpList` class, creating a bottleneck for the development of new solvers.
# Proposed solution
Redesign the core data structures and algorithms of Nektar++ to address the above concerns. A high-level `Field` class encapsulates the storage of solutions, tied to the nature of the representation (physical space, coefficient space, etc), tied to an abstraction of a memory back-end. Mathematical operations are represented as abstract `Operators`, with specific concrete implementations targeted to different platforms or programming models.
# Implementation
New implementation is currently captured in the `Operators` library.
## Tests
A range of unit tests have been added within the `UnitTests` directory.
# Suggested reviewers
Everyone
# Notes
It is anticipated that this MR will be merged by the release of v5.6, but will not be integrated with our solvers until a later release.
# Checklist
- [ ] Functions and classes, or changes to them, are documented.
- [ ] User guide/documentation is updated.
- [ ] Changelog is updated.
- [ ] Suitable tests added for new functionality.
- [ ] Contributed code is correctly formatted. (See the [contributing guidelines](https://gitlab.nektar.info/nektar/nektar/-/blob/master/CONTRIBUTING.md#using-clang-format)).
- [ ] License added to any new files.
- [ ] No extraneous files have been added (e.g. compiler output or test data files).v5.6.0Chris CantwellChris Cantwellhttps://gitlab.nektar.info/nektar/nektar/-/merge_requests/1740Fix windows warnings2024-01-27T16:28:20ZChris CantwellFix windows warnings# Issue/feature addressed
Clean up some compiler warnings from the Windows build.
# Proposed solution
Adjust code to avoid ambiguous statements, or adjust logic to avoid
# Checklist
- [X] Functions and classes, or changes to them, are...# Issue/feature addressed
Clean up some compiler warnings from the Windows build.
# Proposed solution
Adjust code to avoid ambiguous statements, or adjust logic to avoid
# Checklist
- [X] Functions and classes, or changes to them, are documented.
- [X] User guide/documentation is updated.
- [ ] Changelog is updated.
- [X] Suitable tests added for new functionality.
- [X] Contributed code is correctly formatted. (See the [contributing guidelines](https://gitlab.nektar.info/nektar/nektar/-/blob/master/CONTRIBUTING.md#using-clang-format)).
- [X] License added to any new files.
- [X] No extraneous files have been added (e.g. compiler output or test data files).v5.6.0Chris CantwellChris Cantwellhttps://gitlab.nektar.info/nektar/nektar/-/merge_requests/1739Draft: Feature/vorticity stability2024-01-30T13:43:47ZAnkang Gaoankanggao@ustc.edu.cnDraft: Feature/vorticity stability# Issue/feature addressed
*Please provide a description of the problem your changes address. Be sure to link to any related items in the issue tracker.*
# Proposed solution
*A summary of the proposed changes and how they address the iss...# Issue/feature addressed
*Please provide a description of the problem your changes address. Be sure to link to any related items in the issue tracker.*
# Proposed solution
*A summary of the proposed changes and how they address the issue/feature at hand. Whenever possible, describe alternatives your considered and decisions you made.*
# Implementation
*A more detailed description of the changes. Please include any details necessary for reviewers to understand your work, and point out particular points would like them to pay particular attention to.*
## Tests
# Suggested reviewers
*Please suggest any people who would be appropriate to review your code.*
# Notes
*Please add any other information that could be useful for reviewers.*
# Checklist
- [ ] Functions and classes, or changes to them, are documented.
- [ ] User guide/documentation is updated.
- [ ] Changelog is updated.
- [ ] Suitable tests added for new functionality.
- [ ] Contributed code is correctly formatted. (See the [contributing guidelines](https://gitlab.nektar.info/nektar/nektar/-/blob/master/CONTRIBUTING.md#using-clang-format)).
- [ ] License added to any new files.
- [ ] No extraneous files have been added (e.g. compiler output or test data files).https://gitlab.nektar.info/nektar/nektar/-/merge_requests/1737Fix memory leak with updated matrices and Block preconditioner2024-03-22T15:21:00ZHenrik WustenbergFix memory leak with updated matrices and Block preconditioner# Issue/feature addressed
The Block preconditioner initialises a GS mapping for communicating vertex data. This mapping is by default not deallocated when the preconditioner object is destroyed. The mappings do not change in time and can...# Issue/feature addressed
The Block preconditioner initialises a GS mapping for communicating vertex data. This mapping is by default not deallocated when the preconditioner object is destroyed. The mappings do not change in time and can be saved instead of re-initialised every time step.
# Proposed solution
Only initialise the gs mapping upon first creation of a `PreconditionerBlock` and save it in the `AssemblyMap`.
# Implementation
The `AssemblyMapCG` holds a vector with gs mapping(s) which are handed to any `PreconditionerBlock` upon initialisation. The first call to `PreconditionerBlock::BuildPreconditioner()` initialises the mapping and hands a copy to the `AssemblyMapCG`.
Additionally this merge fixes the `PreconditionerBlock` for low polynomial orders where Edges/Faces or the volume element have zero interior DoFs.
## Tests
- We do not have tests with memory-constraint environments. However, I added a test with the `BlockPreconditioner` specifically to make sure this change is not breaking any code.
# Suggested reviewers
@dmoxey
@ssherw
@CFD-Xing
# Notes
- This memory leak also occurs with the `LowEnergyBlock` preconditioner. However, freeing or saving the gs mappings alone does not fix the leak. Still need to work out what causes the memory leak.
# Checklist
- [x] Functions and classes, or changes to them, are documented.
- ~~[ ] User guide/documentation is updated.~~
- [x] Changelog is updated.
- [x] Suitable tests added for new functionality.
- [x] Contributed code is correctly formatted. (See the [contributing guidelines](https://gitlab.nektar.info/nektar/nektar/-/blob/master/CONTRIBUTING.md#using-clang-format)).
- [x] License added to any new files.
- [x] No extraneous files have been added (e.g. compiler output or test data files).v5.6.0Jacques XingJacques Xinghttps://gitlab.nektar.info/nektar/nektar/-/merge_requests/1735Draft: Matrix free ops shape cleanup2024-01-29T19:23:31ZAllen SandersonDraft: Matrix free ops shape cleanup# Draft
I marked this as a draft because my fork shows my merges of the master. I was not expecting that. And not sure we want those commits. But not sure how to get just the last commit.
# Issue/feature addressed
Instead of using the...# Draft
I marked this as a draft because my fork shows my merges of the master. I was not expecting that. And not sure we want those commits. But not sure how to get just the last commit.
# Issue/feature addressed
Instead of using the DIM as a template parameter to the operator class helper, using the shape type to have more flexibility.
# Proposed solution
Change the template parameter
# Implementation
Update the Helper class.
## Tests
All tests pass.
# Suggested reviewers
@ccantwel
# Notes
Minor change
# Checklist
- [ ] Functions and classes, or changes to them, are documented.
- [ ] User guide/documentation is updated.
- [ ] Changelog is updated.
- [ ] Suitable tests added for new functionality.
- [ ] Contributed code is correctly formatted. (See the [contributing guidelines](https://gitlab.nektar.info/nektar/nektar/-/blob/master/CONTRIBUTING.md#using-clang-format)).
- [ ] License added to any new files.
- [ ] No extraneous files have been added (e.g. compiler output or test data files).https://gitlab.nektar.info/nektar/nektar/-/merge_requests/1725Draft: Remove dealiasing in UnsteadyAdvection solver2024-01-12T14:32:45ZJacques XingDraft: Remove dealiasing in UnsteadyAdvection solver# Issue/feature addressed
Dealiasing should not be necessary for linear advection. This MR suggest to remove the dealiasing provision in the UnsteadyAdvection solver.
# Proposed solution
# Implementation
## Tests
# Suggested reviewer...# Issue/feature addressed
Dealiasing should not be necessary for linear advection. This MR suggest to remove the dealiasing provision in the UnsteadyAdvection solver.
# Proposed solution
# Implementation
## Tests
# Suggested reviewers
*Please suggest any people who would be appropriate to review your code.*
# Notes
*Please add any other information that could be useful for reviewers.*
# Checklist
- [x] Functions and classes, or changes to them, are documented.
- ~~[ ] User guide/documentation is updated.~~
- [x] Changelog is updated.
- ~~[ ] Suitable tests added for new functionality.~~
- [x] Contributed code is correctly formatted. (See the [contributing guidelines](https://gitlab.nektar.info/nektar/nektar/-/blob/master/CONTRIBUTING.md#using-clang-format)).
- ~~[ ] License added to any new files.~~
- [x] No extraneous files have been added (e.g. compiler output or test data files).v5.6.0Spencer SherwinSpencer Sherwinhttps://gitlab.nektar.info/nektar/nektar/-/merge_requests/1721Draft: Add AvectionVelocityWvaluator, enable time dependand and time interpol...2024-02-09T14:42:27ZStanislaw GepnerDraft: Add AvectionVelocityWvaluator, enable time dependand and time interpolated advection velocity# Issue/feature addressed
This enables time-dependent Advection Velocity in the ADR solver. Advection Velocity can be either typed in in a session file, marked as `USERDEFINEDTYPE="TimeDependent"`, or it can be read from a sequence of `...# Issue/feature addressed
This enables time-dependent Advection Velocity in the ADR solver. Advection Velocity can be either typed in in a session file, marked as `USERDEFINEDTYPE="TimeDependent"`, or it can be read from a sequence of `/fld` files.
# Proposed solution
Copy @ccantwel solution for the Floquet analysis from the Incompressible solver. Eventually, if the MR approved the same approach should be used there, so time interpolation is performed by the same class.
# Implementation
## Tests
# Suggested reviewers
@ccantwel @CFD-Xing
# Notes
# Checklist
* [x] Functions and classes, or changes to them, are documented.
* [x] User guide/documentation is updated.
* [x] Changelog is updated.
* [ ] Suitable tests added for new functionality.
* [ ] Contributed code is correctly formatted. (See the [contributing guidelines](https://gitlab.nektar.info/nektar/nektar/-/blob/master/CONTRIBUTING.md#using-clang-format)).
* [x] License added to any new files.
* [x] No extraneous files have been added (e.g. compiler output or test data files).v5.6.0https://gitlab.nektar.info/nektar/nektar/-/merge_requests/1702PerAlign extension for all types of meshes2024-03-23T10:25:04ZKaloyan KirilovPerAlign extension for all types of meshes# Issue/feature addressed
This is an extension to the PerAlign module that allows running meshes with Nektar++ that need periodicity in the boundary conditions. The legacy module has some very strict limitations that practically do not ...# Issue/feature addressed
This is an extension to the PerAlign module that allows running meshes with Nektar++ that need periodicity in the boundary conditions. The legacy module has some very strict limitations that practically do not fully support runs with meshes other than Hex. If Tets or Prisms are in the mesh the Incompressible solver will correctly throw away an error on the initialization of solver run, whereas for full hex mesh, it will not stop the user to run, but the results will be noisy on the periodicities.
The legacy state is as follows:
- Fully Hex meshes
* Working fine for fully hex meshes.
* Can be called as two separate calls to the ProcessModule also directly on the HO mesh.
* The reason for that it that it does need the "orient" flag, which calls Module::ReorderPrisms.
* Guglielmo has already ran successfully LES/DNS cases with the module and GMSH.
- Fully Tetrahedral meshes
* Working fine with a single periodic pair only on the linear mesh. The module throws away all HO information of the Tets.
* NOT working if you would like to impose multiple periodic pairs in a single module call.
* NOT possible to give multiple inputs.
* NOT useful to run multiple times the command : you need to pass "orient" flag, hence ReorderPrisms will delete all vertex IDs.
- Hybrid Tet-Prism meshes
* Same as fully Tet +
* NOT working if ran on a wall that has prism quad faces attached to it and PerAlign ran on the linear mesh. After the prisms are split ( both with the legacy and the new ProcessBL) the expansion orientations are not consistent.
* NOT keeping the full HO-information. Keeps only the EdgeNodes ( Face Curvature nodes are thrown away in Module::ReorderPrism).
# Proposed solution
Current functionalities (New Functionalities)
* Added functionality to be able to add multiple surface pairs as an option to the module ProcessPerAlign.
* Added functionality to run with multiple surface pairs ( max 3 at the moment - we can make it a variable) in Module::ReorderPrisms()
* Fully Tetrahedral meshes
* Working fine with a single or multiple periodic pairs only on the linear mesh.
* Working if you would like to impose multiple periodic pairs in a single module call.
* Made it Possible to give multiple inputs.
* NOT useful to run multiple times the command : you need to pass "orient" flag, hence ReorderPrisms will delete all vertex IDs.
* Hybrid Tet-Prism meshes
* Same as fully Tet +
* Working if ran on a wall that has prism quad faces attached to it and PerAlign ran only on the FINAL HO Mesh.
* NOT keeping the full HO-information. Keeps only the EdgeNodes ( Face Curvature nodes are thrown away in Module::ReorderPrism) .
# Implementation
* ProcessPerAlign
* allow m_config to input a vectors of surface pairs in
* EXsurf1 = 2,3 : surf2=5,6 - It will create two periodic pairs than could have completely different periodicity configurations: EX: First pair between surfaces 2 and 5. Second pair between surfaces 3 and 6.
* In order to separate calls for the orient (rotational periodicity) or dir (translational) I have introduced a token separator "!". That is not ideal but the only one I could find working both in Bash (no separate variable created) and C++ since we need to allow the use of both "x,y" as directions and unit vector one "1.0,0.0,1.0" in a single m_config call.
* EX: -m peralign:surf1=3,8:surf2=5,9:dir=y!0.0,0.0,1.0 - This way the first pair(composites 3,5) will take direction "y" and the second pair (8,9) will take "z", but defined as .
* I tried to keep the core part of the module as it is. Hence the algorithm on pairing centroids of the boundary faces is not really changed.
* Splits the pair commands
* Creates a for loop around the periodic Surface Pairs (pIt) to push back in a single perFace, which is then passed as an argument to orient
* **I introduced a final entry to every perFace vector\<int\> list denoting the pairID, which allows me to filter the pairID afterwards in Module::ReorderPrisms().**
* In the end of the every surface pair Boundary composite is rearanged as required in Nektar++ ( no vID are changed here)
* Module::ReorderPrism(perFaces)
* As beforee all vertex IDs are -1 .
* As discussed in the meeting - first we give IDs to the periodic vertices( from perFaces) . However:
* Added an extra loop on top of the elements and running it sequentially based on the pairID. This is where the vertices Since this is a massive unoredered map, the pairID are mixed. Moreover, the corner vertices where the two pairs are done simultanously will become inconsistent if both pairID are giving vertex IDs. This way the expansion are consistent in my test-cases.
* Added a check in the end if there are vId=-1 . This will be useful if there are also to spot pyramids.
## Tests
I have a 3D NACA0012 mesh that I can run with the Incompressible solver for 10 timesteps in 4 (Hex/ Tet/ 2xTet-Prism) different configurations to ensure it is possible. I am not sure how to add these multistage test like:
1. Create the mesh
2. Run the solver
3. Analyze if results continuous
# Suggested reviewers
@mgreen @dmoxey
# Notes
In order to test that PerAlign works properly, a solver running on a produced periodic meshe is required. To accommodate this additional testing requirement, some changes to Tester were introduced that allow to run sequential tests from a .tst file. These changes are not directly related to issue this MR addresses and in hindsight could be seperated to another MR. If there is no objection it will stay here, however if it makes the review simpler to take them out and create a separate MR for Tester we could do that.
# Checklist
* [x] Functions and classes, or changes to them, are documented.
* [ ] User guide/documentation is updated.
* [x] Changelog is updated.
* [x] Suitable tests added for new functionality.
* [ ] Contributed code is correctly formatted. (See the [contributing guidelines](https://gitlab.nektar.info/nektar/nektar/-/blob/master/CONTRIBUTING.md#using-clang-format)).
* [x] License added to any new files.
* [x] No extraneous files have been added (e.g. compiler output or test data files).v5.6.0Spencer SherwinDave MoxeySpencer Sherwin