This MR fixes the issue on latest OpenMPI (as found on Fedora 21) which produces errors when using MPI that MPI_Attr_get() function is called after MPI_Finalize(). This is because we were previously not calling PetscFinalize(). The PetscInitialize and PetscFinalize calls have been moved to CommMpi to ensure they are called at the right time and in the right order.
Dave should probably review this, since he implemented the PETSc stuff.