Refactoring Collections & Introducing the PhysInterp1DScaled function inside collections

Issue/feature addressed

During the profiling of the IncNavierStokesSolver, which was carried out using the Solaris software profiling suite inside the Clapham node, it was found that the PhysInterp1DScaled function is one of the most time-consuming ones. A simplified test-case was used for this purpose, based on the 2D slice of the IFW in the span-wise location of y = -250 mm. This function is called when SPECTRALHPDEALIASING is set to TRUE inside the section for the IncNavierStokesSolver and is responsible for evaluating the advection term using over-intergration, or "practically" more quadrature points than the default NUMMODES+1, or NUMMODES+2 which is specified depending on the polynomial order of the simulation.

Proposed solution

Introduce the PhysInterp1DScaled operator inside Collections, so that more efficient implementations can be developed such as sum-factorization and matrix-free. This introduced an additional challenge because the input size and the output size of this operator are not the same with the other operators.
Currently in Nektar++, it is possible to operate on an expansion list from Physical Space to Coefficient Space and the opposite way around, while the size of the number of quadrature points (Physical Space) and the size of the number of modes (Coefficient Space) is fixed. This is not applicable to PhysInterp1DScaled as it operates from Physical Space (quadrature points of the expansion list) to Physical Space (quadrature points after over-integration). The quadrature points of the output (enhanced space) are calculated based on a scaling factor which 1.5 for the incompressible solver and 2 for the compressible solver. This scaling factor is taken into account through the variable scale = m_factors[StdRegions::eFactorConst], which is taken as an input when the PhysInterp1DScaled function is called. For example inside a 1D domain, if the expansion list includes 10 quadrature points, then the output of the PhysInterp1DScaled function will include 12 quadrature points with a scaling factor of 1.5.

Implementation

1. Simplified/Reduced the if-statements for generating the Collections based on the BasisKeys of each element inside the expansion list.
2. Introduced m_inputSize and m_outputSize members inside the Operator base class.
   m_inputSize stores the number of modes or quadrature points that are given as input to the operator.
   m_outputSize stores the number of modes or quadrature points that are given as output from the operator. \
3. Introduced Helper class for every operator inside collections, which is responsible for storing as a member the size of the collection which is taken as an input by the operator and the size of the collection which is given as an output to allow having a variable size for the output of the operators. Also, it removes the usage of m_coll_coeff_offset and m_coll_phys_offset to access the mesh elements of each collection. The size of each collection can be used to access the elements of each collection. This assumes that each collection is composed of elements that are stored continuously in the expansion list.

In this figure, each continuous "green" block corresponds to a new Collection.

Currently, the helper class has a variable "NekDouble scale{1.5}" for the scaling factor - Need to update this to be modular so that it is 1.5 for the IncNavierStokesSolver, but 2 for the CompressibleFlowSolver and 2 members:

• m_inputSize which stores the input size of the collection for each operator, in the BwdTrans operator this is equal to the number of elements inside the collections multiplied by the number of modes of a single element. It can be accessed inside MultiRegions using the function GetInputSize(Collections::NameOfTheOperator).
• m_outputSize which stores the output size of the collection for each operator, in the BwdTrans operator this is equal to the number of elements inside the collections multiplied by the number of quadrature points of a single element. It can be accessed inside MultiRegions using the function GetOutputSize(Collections::NameOfTheOperator)

It is not necessary anymore to store the m_coll_phys_offset and m_coll_coeff_offset arrays, because the offset from the first element of the expansion list can be calculated using:
input_offset += m_collections[i].GetInputSize(Collections::NameOfTheOperator);
output_offset += m_collections[i].GetOutputSize(Collections::NameOfTheOperator);
where index i corresponds to the index of the collection

4. Introduced PhysInterp1DScaled function inside the Collections to allow for more efficient implementations of this function. Currently, only the NoCollection implementation is available but the code is in place for both sum-factorization, matrix-free etc.

Tests

No additional tests are necessary as this is a refactoring of the existing functionality in Nektar++.

Suggested reviewers

@ssherw @ccantwel @mlahooti

Notes

Checklist

• Functions and classes, or changes to them, are documented.
• [ ] User guide/documentation is updated.
• Changelog is updated.
• Suitable tests added for new functionality.
• Contributed code is correctly formatted. (See the contributing guidelines).
• License added to any new files.
• No extraneous files have been added (e.g. compiler output or test data files).