GEM Considerations
==================

This section discusses some specific considerations for the forthcoming integration of the GEM dynamical core into
CanCPL and CanESM.

The current communication strategy used in the GEM-NEMO coupled model used at MRD differs from the current CanESM
coupling. GEM and NEMO exist in their own ``MPI_COMM_WORLDs`` and communicate over TCP/IP using the GOSSIP communication
layer (as illustrated in the following figure). This differs from the CanESM coupling paradigm which splits a single
``MPI_COMM_WORLD`` into communicators for each model component.

.. figure:: graphics/GEM-NEMO-Communication.PNG

  GEM-NEMO communication topology using TCP/IP based GOSSIP communication layer. Each component exists in its own
  separate ``MPI_COMM_WORLD`` not just separate MPI communicators.

Additionally, GEM uses the ``RPN_COMM_LIBRARY`` which wraps much of the MPI API and deals with most of the MPI
communications within GEM. This includes the splitting of the ``MPI_COMM_WORLD`` into separate communicators for the Yin
and Yang submodels which form the global overset grid.

The MPI initialization of GEM will likely need to be modified to conform to the CanESM communication paradigm which
requires a single ``MPI_COMM_WORLD`` for all components within the coupled model. Two ways forward have been proposed.

1. During the initialization of GEM, GEM is responsible for splitting the ``MPI_COMM_WORLD`` into a GEM and non-GEM set
   of communicators. Both of those are then passed into ``define_groups`` in CanCPL. CanCPL and NEMO during their call to
   ``define_groups`` splits the GEM communicator between the two.
2. GEM calls ``define_groups`` before the Yin-Yang grid
   communicators are split off. This requires relinquishing control from GEM over the entire ``MPI_COMM_WORLD``. This route
   requires less refactoring of CanCPL than the first option, but is slightly more invasive into the GEM/RPN
   initialization.