INSTALL.rst

-*******************
-Build the framework
-*******************
-
-.. todo::
-
-   TBD
-
-.. contents::
-   :local:
-
-Prerequisites
-=============
-
-| The NEMO source code is written in *Fortran 95* and
-  some of its prerequisite tools and libraries are already included in the download.
-| It contains the AGRIF_ preprocessing program ``conv``; the FCM_ build system and
-  the IOIPSL_ library for parts of the output.
-
-System dependencies
--------------------
-
-In the first place the other requirements should be provided natively by your system or
-can be installed from the official repositories of your Unix-like distribution:
-
-- *Perl* interpreter
-- *Fortran* compiler (``ifort``, ``gfortran``, ``pgfortran``, ...),
-- *Message Passing Interface (MPI)* implementation (e.g. |OpenMPI|_ or |MPICH|_).
-- |NetCDF|_ library with its underlying |HDF|_
-
-**NEMO, by default, takes advantage of some MPI features introduced into the MPI-3 standard.**
-
-.. hint::
-
-   The MPI implementation is not strictly essential
-   since it is possible to compile and run NEMO on a single processor.
-   However most realistic configurations will require the parallel capabilities of NEMO and
-   these use the MPI standard.
-
-.. note::
-
-   On older systems, that do not support MPI-3 features,
-   the ``key_mpi2`` preprocessor key should be used at compile time.
-   This will limit MPI features to those defined within the MPI-2 standard
-   (but will lose some performance benefits).
-
-.. |OpenMPI| replace:: *OpenMPI*
-.. _OpenMPI: https://www.open-mpi.org
-.. |MPICH|   replace:: *MPICH*
-.. _MPICH:   https://www.mpich.org
-.. |NetCDF|  replace:: *Network Common Data Form (NetCDF)*
-.. _NetCDF:  https://www.unidata.ucar.edu
-.. |HDF|     replace:: *Hierarchical Data Form (HDF)*
-.. _HDF:     https://www.hdfgroup.org
-
-Specifics for NetCDF and HDF
-----------------------------
-
-NetCDF and HDF versions from official repositories may have not been compiled with MPI support.
-However access to all the options available with the XIOS IO-server will require
-the parallelism of these libraries.
-
-| **To satisfy these requirements, it is common to have to compile from source
-  in this order HDF (C library) then NetCDF (C and Fortran libraries)**
-| It is also necessary to compile these libraries with the same version of the MPI implementation that
-  both NEMO and XIOS (see below) have been compiled and linked with.
-
-.. hint::
-
-   | It is difficult to define the options for the compilation as
-     they differ from one architecture to another according to
-     the hardware used and the software installed.
-   | The following is provided without any warranty
-
-   .. code-block:: console
-
-      $ ./configure [--{enable-fortran,disable-shared,enable-parallel}] ...
-
-   It is recommended to build the tests ``--enable-parallel-tests`` and run them with ``make check``
-
-Particular versions of these libraries may have their own restrictions.
-State the following requirements for netCDF-4 support:
-
-.. caution::
-
-   | When building NetCDF-C library versions older than 4.4.1, use only HDF5 1.8.x versions.
-   | Combining older NetCDF-C versions with newer HDF5 1.10 versions will create superblock 3 files
-     that are not readable by lots of older software.
-
-Extract and install XIOS
-========================
-
-With the sole exception of running NEMO in mono-processor mode
-(in which case output options are limited to those supported by the ``IOIPSL`` library),
-diagnostic outputs from NEMO are handled by the third party ``XIOS`` library.
-It can be used in two different modes:
-
-:*attached*:  Every NEMO process also acts as a XIOS server
-:*detached*:  Every NEMO process runs as a XIOS client.
-  Output is collected and collated by external, stand-alone XIOS server processors.
-
-Instructions on how to install XIOS can be found on its :xios:`wiki<>`.
-
-.. hint::
-
-   It is recommended to use XIOS 2.5 release.
-   This version should be more stable (in terms of future code changes) than the XIOS trunk.
-   It is also the one used by the NEMO system team when testing all developments and new releases.
-
-   This particular version has its own branch and can be checked out with:
-
-   .. code:: console
-
-      $ svn co https://forge.ipsl.jussieu.fr/ioserver/svn/XIOS/branchs/xios-2.5
-
-Download and install the NEMO code
-==================================
-
-Checkout the NEMO sources
--------------------------
-
-.. code:: console
-
-   $ svn co https://forge.ipsl.jussieu.fr/nemo/svn/NEMO/trunk
-
-Description of 1\ :sup:`st` level tree structure
-------------------------------------------------
-
-+---------------+----------------------------------------+
-| :file:`arch`  | Compilation settings                   |
-+---------------+----------------------------------------+
-| :file:`cfgs`  | :doc:`Reference configurations <cfgs>` |
-+---------------+----------------------------------------+
-| :file:`doc`   | :doc:`Documentation <doc>`             |
-+---------------+----------------------------------------+
-| :file:`ext`   | Dependencies included                  |
-|               | (``AGRIF``, ``FCM`` & ``IOIPSL``)      |
-+---------------+----------------------------------------+
-| :file:`mk`    | Compilation scripts                    |
-+---------------+----------------------------------------+
-| :file:`src`   | :doc:`Modelling routines <src>`        |
-+---------------+----------------------------------------+
-| :file:`tests` | :doc:`Test cases <tests>`              |
-|               | (unsupported)                          |
-+---------------+----------------------------------------+
-| :file:`tools` | :doc:`Utilities <tools>`               |
-|               | to {pre,post}process data              |
-+---------------+----------------------------------------+
-
-Setup your architecture configuration file
-------------------------------------------
-
-All compiler options in NEMO are controlled using files in :file:`./arch/arch-'my_arch'.fcm` where
-``my_arch`` is the name of the computing architecture
-(generally following the pattern ``HPCC-compiler`` or ``OS-compiler``).
-It is recommended to copy and rename an configuration file from an architecture similar to your owns.
-You will need to set appropriate values for all of the variables in the file.
-In particular the FCM variables:
-``%NCDF_HOME``; ``%HDF5_HOME`` and ``%XIOS_HOME`` should be set to
-the installation directories used for XIOS installation
-
-.. code-block:: sh
-
-   %NCDF_HOME    /usr/local/path/to/netcdf
-   %HDF5_HOME    /usr/local/path/to/hdf5
-   %XIOS_HOME    /home/$( whoami )/path/to/xios-2.5
-   %OASIS_HOME   /home/$( whoami )/path/to/oasis
-
-Create and compile a new configuration
-======================================
-
-The main script to {re}compile and create executable is called :file:`makenemo` located at
-the root of the working copy.
-It is used to identify the routines you need from the source code, to build the makefile and run it.
-As an example, compile a :file:`MY_GYRE` configuration from GYRE with 'my_arch':
-
-.. code-block:: sh
-
-   ./makenemo –m 'my_arch' –r GYRE -n 'MY_GYRE'
-
-Then at the end of the configuration compilation,
-:file:`MY_GYRE` directory will have the following structure.
-
-+------------+----------------------------------------------------------------------------+
-| Directory  | Purpose                                                                    |
-+============+============================================================================+
-| ``BLD``    | BuiLD folder: target executable, headers, libs, preprocessed routines, ... |
-+------------+----------------------------------------------------------------------------+
-| ``EXP00``  | Run   folder: link to executable, namelists, ``*.xml`` and IOs             |
-+------------+----------------------------------------------------------------------------+
-| ``EXPREF`` | Files under version control only for :doc:`official configurations <cfgs>` |
-+------------+----------------------------------------------------------------------------+
-| ``MY_SRC`` | New routines or modified copies of NEMO sources                            |
-+------------+----------------------------------------------------------------------------+
-| ``WORK``   | Links to all raw routines from :file:`./src` considered                    |
-+------------+----------------------------------------------------------------------------+
-
-After successful execution of :file:`makenemo` command,
-the executable called `nemo` is available in the :file:`EXP00` directory
-
-More :file:`makenemo` options
------------------------------
-
-``makenemo`` has several other options that can control which source files are selected and
-the operation of the build process itself.
-
-.. literalinclude:: ../../../makenemo
-   :language: text
-   :lines: 119-143
-   :caption: Output of ``makenemo -h``
-
-These options can be useful for maintaining several code versions with only minor differences but
-they should be used sparingly.
-Note however the ``-j`` option which should be used more routinely to speed up the build process.
-For example:
-
-.. code-block:: sh
-
-        ./makenemo –m 'my_arch' –r GYRE -n 'MY_GYRE' -j 8
-
-will compile up to 8 processes simultaneously.
-
-Default behaviour
------------------
-
-At the first use,
-you need the ``-m`` option to specify the architecture configuration file
-(compiler and its options, routines and libraries to include),
-then for next compilation, it is assumed you will be using the same compiler.
-If the ``-n`` option is not specified the last compiled configuration will be used.
-
-Tools used during the process
------------------------------
-
-* :file:`functions.sh`: bash functions used by ``makenemo``, for instance to create the WORK directory
-* :file:`cfg.txt`     : text list of configurations and source directories
-* :file:`bld.cfg`     : FCM rules for compilation
-
-Examples
---------
-
-.. literalinclude:: ../../../makenemo
-   :language: text
-   :lines: 146-153
-
-Running the model
-=================
-
-Once :file:`makenemo` has run successfully,
-the ``nemo`` executable is available in :file:`./cfgs/MY_CONFIG/EXP00`.
-For the reference configurations, the :file:`EXP00` folder also contains the initial input files
-(namelists, ``*.xml`` files for the IOs, ...).
-If the configuration needs other input files, they have to be placed here.
-
-.. code-block:: sh
-
-   cd 'MY_CONFIG'/EXP00
-   mpirun -n $NPROCS ./nemo   # $NPROCS is the number of processes
-                              # mpirun is your MPI wrapper
-
-Viewing and changing list of active CPP keys
-============================================
-
-For a given configuration (here called ``MY_CONFIG``),
-the list of active CPP keys can be found in :file:`./cfgs/'MYCONFIG'/cpp_MY_CONFIG.fcm`
-
-This text file can be edited by hand or with :file:`makenemo` to change the list of active CPP keys.
-Once changed, one needs to recompile ``nemo`` in order for this change to be taken in account.
-Note that most NEMO configurations will need to specify the following CPP keys:
-``key_xios`` for IOs. MPI parallelism is activated by default. Use ``key_mpi_off`` to compile without MPI.

README.rst

-.. todo::
+NEMO
+====
 
+*Nucleus for European Modelling of the Ocean* (NEMO) is a state-of-the-art modelling
+framework for research activities and forecasting services in ocean and climate sciences,
+developed in a sustainable way by the NEMO European consortium since 2008.
 
+For all information please refer to the `NEMO wiki <https://forge.nemo-ocean.eu/nemo/nemo/-/wikis/home>`_
 
-NEMO_ for *Nucleus for European Modelling of the Ocean* is a state-of-the-art modelling framework for
-research activities and forecasting services in ocean and climate sciences,
-developed in a sustainable way by a European consortium since 2008.
+How to cite
+===========
 
-.. contents::
-   :local:
-
-Overview
-========
-
-The NEMO ocean model has 3 major components:
-
-- |OCE| models the ocean {thermo}dynamics and solves the primitive equations
-  (:file:`./src/OCE`)
-- |ICE| simulates sea-ice {thermo}dynamics, brine inclusions and
-  subgrid-scale thickness variations (:file:`./src/ICE`)
-- |MBG| models the {on,off}line oceanic tracers transport and biogeochemical processes
-  (:file:`./src/TOP`)
-
-These physical core engines are described in
-their respective `reference publications <#project-documentation>`_ that
-must be cited for any work related to their use (see :doc:`cite`).
-
-Assets and solutions
-====================
-
-Not only does the NEMO framework model the ocean circulation,
-it offers various features to enable
-
-- Create :doc:`embedded zooms<zooms>` seamlessly thanks to 2-way nesting package AGRIF_.
-- Opportunity to integrate an :doc:`external biogeochemistry model<tracers>`
-- Versatile :doc:`data assimilation<da>`
-- Generation of :doc:`diagnostics<diags>` through effective XIOS_ system
-- Roll-out Earth system modeling with :doc:`coupling interface<cplg>` based on OASIS_
-
-Several :doc:`built-in configurations<cfgs>` are provided to
-evaluate the skills and performances of the model which
-can be used as templates for setting up a new configurations (:file:`./cfgs`).
-
-The user can also checkout available :doc:`idealized test cases<tests>` that
-address specific physical processes (:file:`./tests`).
-
-A set of :doc:`utilities <tools>` is also provided to {pre,post}process your data (:file:`./tools`).
-
-Project documentation
-=====================
-
-A walkthrough tutorial illustrates how to get code dependencies, compile and execute NEMO
-(:file:`./INSTALL.rst`).
-
-Reference manuals and quick start guide can be build from source and
-exported to HTML or PDF formats (:file:`./doc`) or
-downloaded directly from the :forge:`development platform<wiki/Documentations>`.
-
-============ ================== ===================
- Component    Reference Manual   Quick Start Guide
-============ ================== ===================
- |NEMO-OCE|   |DOI man OCE|_     |DOI qsg|
- |NEMO-ICE|   |DOI man ICE|
- |NEMO-MBG|   |DOI man MBG|
-============ ================== ===================
-
-Since 2014 the project has a `Special Issue`_ in the open-access journal
-Geoscientific Model Development (GMD) from the European Geosciences Union (EGU_).
-The main scope is to collect relevant manuscripts covering various topics and
-to provide a single portal to assess the model potential and evolution.
-
-Used by a wide audience,
-numerous :website:`associated projects<projects>` have been carried out and
-extensive :website:`bibliography<bibliography/publications>` published.
-
-Development board
-=================
-
-The NEMO Consortium pulling together 5 European institutes
-(CMCC_, CNRS_, MOI_, `Met Office`_ and NERC_) plans the sustainable development in order to
-keep a reliable evolving framework since 2008.
-
-It defines the |DOI dev stgy|_ that is implemented by the System Team on a yearly basis
-in order to release a new version almost every four years.
-
-When the need arises, :forge:`working groups<wiki/WorkingGroups>` are created or resumed to
-gather the community expertise for advising on the development activities.
-
-.. |DOI dev stgy| replace:: multi-year development strategy
+To acknowledge the sustainable development efforts of the NEMO Consortium, please quote `these references <https://www.nemo-ocean.eu/bibliography/how-to-cite/>`_ in your publications and presentations using NEMO.
 
 Disclaimer
 ==========
 
 The NEMO source code is freely available and distributed under
-:download:`CeCILL v2.0 license <../../../LICENSE>` (GNU GPL compatible).
+`CeCILL v2.0 license <./LICENSE>`_ (GNU GPL compatible).
 
 You can use, modify and/or redistribute the software under its terms,
 but users are provided only with a limited warranty and the software's authors and

REFERENCES.bib

-
-@manual{          NEMO_man,
-  title         = "NEMO ocean engine",
-  series        = "Scientific Notes of Climate Modelling Center",
-  number        = "27",
-  author        = "Gurvan Madec and NEMO System Team",
-  institution   = "Institut Pierre-Simon Laplace (IPSL)",
-  publisher     = "Zenodo",
-  issn          = "1288-1619",
-  doi           = "10.5281/zenodo.1464816"
-}
-%   edition="",
-%   year=""
-
-@manual{          SI3_man,
-  title         = "Sea Ice modelling Integrated Initiative (SI$^3$) -- The
-                  NEMO sea ice engine",
-  series        = "Scientific Notes of Climate Modelling Center",
-  number        = "31",
-  author        = "NEMO Sea Ice Working Group",
-  institution   = "Institut Pierre-Simon Laplace (IPSL)",
-  publisher     = "Zenodo",
-  issn          = "1288-1619",
-  doi           = "10.5281/zenodo.1471689"
-}
-%   edition="",
-%   year=""
-
-@manual{          TOP_man,
-  title         = "Tracers in Ocean Paradigm (TOP) -- The NEMO passive
-                  tracers engine",
-  series        = "Scientific Notes of Climate Modelling Center",
-  number        = "28",
-  author        = "NEMO TOP Working Group",
-  institution   = "Institut Pierre-Simon Laplace (IPSL)",
-  publisher     = "Zenodo",
-  issn          = "1288-1619",
-  doi           = "10.5281/zenodo.1471700"
-}
-%   edition="",
-%   year=""
-
-@article{         TAM_pub,
-  title         = "NEMOTAM: Tangent and Adjoint Models for the ocean
-                  modelling platform NEMO",
-  pages         = "1245--1257",
-  journal       = "Geoscientific Model Development",
-  volume        = "8",
-  number        = "4",
-  author        = "Vidard, A. and Bouttier, P.-A. and Vigilant, F.",
-  year          = "2015",
-  doi           = "10.5194/gmd-8-1245-2015"
-}

arch/CMCC/arch-X64_JUNO.env

+
+module purge
+
+module load intel-2021.6.0/cmake/3.25.1-7wfsx
+module load oneapi-2022.1.0/compiler-rt/2022.1.0
+module load intel-2021.6.0/2021.6.0
+module load impi-2021.6.0/2021.6.0
+module load intel-2021.6.0/impi-2021.6.0/hdf5-threadsafe/1.13.3-zbgha
+module load intel-2021.6.0/impi-2021.6.0/netcdf-c-threadsafe/4.9.0-wpe4t
+module load intel-2021.6.0/impi-2021.6.0/netcdf-fortran-threadsafe/4.6.0-75oow
+module load intel-2021.6.0/impi-2021.6.0/parallel-netcdf/1.12.3-eshb5
+module load intel-2021.6.0/perl/5.36.0-jj4hw
+module load intel-2021.6.0/perl-uri/1.72-6at2i
+module load intel-2021.6.0/impi-2021.6.0/xios/2.5-36kwn
+
+# set linker path to 64-bit libraries
+export LD_LIBRARY_PATH="/lib64/":$LD_LIBRARY_PATH

arch/CMCC/arch-X64_JUNO.fcm

+# mpi ifort compiler options for ZEUS cluster + XIOS
+#
+#
+# NCDF_INC    netcdf4 include file
+# NCDF_LIB    netcdf4 library
+# XIOS_INC    xios include file    (taken into accound only if key_xios is activated)
+# XIOS_LIB    xios library         (taken into accound only if key_xios is activated)
+#
+# CPP         Pre-processor
+# FC          Fortran compiler command
+# FCFLAGS     Fortran compiler flags
+# FFLAGS      Fortran 77 compiler flags
+# LD          linker
+# FPPFLAGS    pre-processing flags
+# LDFLAGS     linker flags, e.g. -L<lib dir> if you have libraries
+# AR          assembler
+# ARFLAGS     assembler flags
+# MK          make (usually GNU make)
+# USER_INC    complete list of include files
+# USER_LIB    complete list of libraries to pass to the linker
+# CC          C compiler used to compile conv for AGRIF
+# CFLAGS      compiler flags used with CC
+#
+# Note that:
+#  - unix variables "$..." are accpeted and will be evaluated before calling fcm.
+#  - fcm variables are starting with a % (and not a $)
+#
+
+# Environment variables are set automatically when loading modules on JUNO cluster (see arch-X64_JUNO.env)
+
+%NCDF_INC            -I${NETCDF_FORTRAN}/include -I${NETCDF_C}/include -I${PARALLEL_NETCDF}/include
+%NCDF_LIB            -L${NETCDF_FORTRAN}/lib -lnetcdff -L${NETCDF_C}/lib -lnetcdf -L${PARALLEL_NETCDF}/lib -lpnetcdf
+%HDF5_INC            -I${HDF5}/include
+%HDF5_LIB            -L${HDF5}/lib -lhdf5_hl -lhdf5
+%XIOS_INC            -I${XIOS}/inc
+%XIOS_LIB            -L${XIOS}/lib -lxios
+
+%USER_INC            %XIOS_INC %NCDF_INC %HDF5_INC
+%USER_LIB            %XIOS_LIB %NCDF_LIB %HDF5_LIB
+
+%FC                  mpiifort
+%PROD_FCFLAGS        -r8 -O3 -fp-model source -traceback -qmkl=cluster -march=icelake-client -mtune=icelake-client -qopt-zmm-usage=low -no-fma
+%DEBUG_FCFLAGS       -r8 -g -O0 -fp-model source -traceback -qmkl=cluster -march=icelake-client -mtune=icelake-client -qopt-zmm-usage=low -no-fma
+%FFLAGS              %FCFLAGS
+%CC                  mpiicc
+%CFLAGS              -O0
+
+%LD                  mpiifort
+%LDFLAGS             -lstdc++ -lgpfs
+%AR                  ar 
+%ARFLAGS             -r
+
+%CPP                 icc
+%FPPFLAGS            -E -P -traditional
+%MK                  gmake

arch/CMCC/arch-X64_ZEUS.fcm

@@ -3,8 +3,8 @@
 #
 # NCDF_INC    netcdf4 include file
 # NCDF_LIB    netcdf4 library
-# XIOS_INC    xios include file    (taken into accound only if key_iomput is activated)
-# XIOS_LIB    xios library         (taken into accound only if key_iomput is activated)
+# XIOS_INC    xios include file    (taken into accound only if key_xios is activated)
+# XIOS_LIB    xios library         (taken into accound only if key_xios is activated)
 #
 # CPP         Pre-processor
 # FC          Fortran compiler command

arch/CNRS/arch-X64_IRENE_DEBUG.fcm → arch/CNRS/arch-X64_ADASTRA.fcm

@@ -30,11 +30,12 @@
 #  - unix variables "$..." are accpeted and will be evaluated before calling fcm.
 #  - fcm variables are starting with a % (and not a $)
 #
-%XIOS_HOME           $CCCWORKDIR/xios-2.5
+%PROD_XIOS_HOME      /lus/work/CT1/egi6035/jpetit/XIOS/XIOS_2629_ADASTRA_INTEL/
+%DEBUG_XIOS_HOME     %{PROD_XIOS_HOME}
 %OASIS_HOME          $CCCWORKDIR/now/models/oa3mct
 
-%NCDF_INC            -I$NETCDFFORTRAN_INCDIR -I$NETCDF_INCDIR
-%NCDF_LIB            -L$NETCDFFORTRAN_LIBDIR -lnetcdff -L$NETCDF_LIBDIR -lnetcdf -L$HDF5_LIBDIR -lhdf5_hl -lhdf5 -lz -lcurl
+%NCDF_INC            -I ${NETCDF_DIR}/include
+%NCDF_LIB            -L ${NETCDF_DIR}/lib -lnetcdff -lnetcdf -L${HDF5_DIR}/lib -lhdf5_hl -lhdf5 -lz -lcurl
 
 %XIOS_INC            -I%XIOS_HOME/inc 
 %XIOS_LIB            -L%XIOS_HOME/lib -lxios -lstdc++
@@ -43,7 +44,8 @@
 
 %CPP	             cpp
 %FC                  mpif90 -c -cpp
-%FCFLAGS             -i4 -r8 -g -O0 -debug all -traceback -fp-model strict -ftrapuv -check all,noarg_temp_created -fpe-all0 -ftz -init=arrays,snan,huge
+%PROD_FCFLAGS        -i4 -r8 -O2 -fp-model strict -fno-alias
+%DEBUG_FCFLAGS       -i4 -r8 -g -O0 -debug all -traceback -fp-model strict -ftrapuv -check all,noarg_temp_created -fpe-all0 -ftz -init=arrays,snan,huge
 %FFLAGS              %FCFLAGS
 %LD                  mpif90
 %LDFLAGS             
@@ -54,5 +56,5 @@
 %USER_INC            %XIOS_INC %OASIS_INC %NCDF_INC
 %USER_LIB            %XIOS_LIB %OASIS_LIB %NCDF_LIB
 
-%CC                  cc
+%CC                  icc
 %CFLAGS              -O0

arch/CNRS/arch-X64_IRENE.fcm

@@ -30,7 +30,8 @@
 #  - unix variables "$..." are accpeted and will be evaluated before calling fcm.
 #  - fcm variables are starting with a % (and not a $)
 #
-%XIOS_HOME           $CCCWORKDIR/xios-2.5
+%PROD_XIOS_HOME      $CCCWORKDIR/xios-trunk
+%DEBUG_XIOS_HOME     %{PROD_XIOS_HOME}_debug
 %OASIS_HOME          $CCCWORKDIR/now/models/oa3mct
 
 %NCDF_INC            -I$NETCDFFORTRAN_INCDIR -I$NETCDF_INCDIR
@@ -43,7 +44,8 @@
 
 %CPP	             cpp
 %FC                  mpif90 -c -cpp
-%FCFLAGS             -i4 -r8 -O3 -fp-model strict -xCORE-AVX512 -fno-alias
+%PROD_FCFLAGS        -i4 -r8 -O3 -fp-model strict -xCORE-AVX512 -fno-alias
+%DEBUG_FCFLAGS       -i4 -r8 -g -O0 -debug all -traceback -fp-model strict -ftrapuv -check all,noarg_temp_created -fpe-all0 -ftz -init=arrays,snan,huge
 %FFLAGS              %FCFLAGS
 %LD                  mpif90
 %LDFLAGS             
@@ -54,5 +56,5 @@
 %USER_INC            %XIOS_INC %OASIS_INC %NCDF_INC
 %USER_LIB            %XIOS_LIB %OASIS_LIB %NCDF_LIB
 
-%CC                  cc
+%CC                  icc
 %CFLAGS              -O0

arch/CNRS/arch-X64_IRENE_GCC.fcm

@@ -5,9 +5,9 @@
 #   module purge
 #   module load gnu/8.3.0
 #   module load flavor/buildcompiler/gcc/8
-#   module load flavor/buildmpi/openmpi/2.0
+#   module load flavor/buildmpi/openmpi/4.0
 #   module load flavor/hdf5/parallel
-#   module load mpi/openmpi/2.0.4
+#   module load mpi/openmpi/4.0.5.3
 #   module load hdf5/1.8.20
 #   module load netcdf-c/4.6.0
 #   module load netcdf-fortran/4.4.4
@@ -16,7 +16,7 @@
 #  - unix variables "$..." are accpeted and will be evaluated before calling fcm.
 #  - fcm variables are starting with a % (and not a $)
 #
-%XIOS_HOME           $CCCWORKDIR/xios-2.5_gcc
+%XIOS_HOME           $CCCWORKDIR/xios-trunk_gcc
 %OASIS_HOME          $CCCWORKDIR/now/models/oa3mct
 
 %NCDF_INC            -I$NETCDFFORTRAN_INCDIR -I$NETCDF_INCDIR
@@ -29,7 +29,8 @@
 
 %CPP	             cpp -Dkey_nosignedzero 
 %FC                  mpif90
-%FCFLAGS             -fdefault-real-8 -O3 -funroll-all-loops -fcray-pointer -ffree-line-length-none -Wno-missing-include-dirs 
+%PROD_FCFLAGS        -fdefault-real-8 -O3 -funroll-all-loops -fcray-pointer -ffree-line-length-none -Wno-missing-include-dirs 
+%DEBUG_FCFLAGS       -fdefault-real-8 -O0 -g -fbacktrace -funroll-all-loops -fcray-pointer -ffree-line-length-none -Wno-missing-include-dirs -fcheck=all -finit-real=nan
 %FFLAGS              %FCFLAGS
 %LD                  mpif90
 %LDFLAGS             

arch/CNRS/arch-X64_IRENE_GCC_DEBUG.fcm

-# Irene BULL at TGCC, http://www-hpc.cea.fr/en/complexe/tgcc-Irene.htm
-#
-# list of modules used
-#
-#   module purge
-#   module load gnu/8.3.0
-#   module load flavor/buildcompiler/gcc/8
-#   module load flavor/buildmpi/openmpi/2.0
-#   module load flavor/hdf5/parallel
-#   module load mpi/openmpi/2.0.4
-#   module load hdf5/1.8.20
-#   module load netcdf-c/4.6.0
-#   module load netcdf-fortran/4.4.4
-#
-# Note that:
-#  - unix variables "$..." are accpeted and will be evaluated before calling fcm.
-#  - fcm variables are starting with a % (and not a $)
-#
-%XIOS_HOME           $CCCWORKDIR/xios-2.5_gcc
-%OASIS_HOME          $CCCWORKDIR/now/models/oa3mct
-
-%NCDF_INC            -I$NETCDFFORTRAN_INCDIR -I$NETCDF_INCDIR
-%NCDF_LIB            -L$NETCDFFORTRAN_LIBDIR -lnetcdff -L$NETCDF_LIBDIR -lnetcdf -L$HDF5_LIBDIR -lhdf5_hl -lhdf5 -lz -lcurl
-
-%XIOS_INC            -I%XIOS_HOME/inc 
-%XIOS_LIB            -L%XIOS_HOME/lib -lxios -lstdc++
-%OASIS_INC           -I%OASIS_HOME/build/lib/mct -I%OASIS_HOME/build/lib/psmile.MPI1
-%OASIS_LIB           -L%OASIS_HOME/lib -lpsmile.MPI1 -lmct -lmpeu -lscrip
-
-%CPP	             cpp -Dkey_nosignedzero 
-%FC                  mpif90
-%FCFLAGS             -fdefault-real-8 -O0 -g -fbacktrace -funroll-all-loops -fcray-pointer -ffree-line-length-none -fcheck=all -finit-real=nan
-%FFLAGS              %FCFLAGS
-%LD                  mpif90
-%LDFLAGS             
-%FPPFLAGS            -P -traditional
-%AR                  ar
-%ARFLAGS             rs
-%MK                  make
-%USER_INC            %XIOS_INC %OASIS_INC %NCDF_INC
-%USER_LIB            %XIOS_LIB %OASIS_LIB %NCDF_LIB
-
-%CC                  cc
-%CFLAGS              -O0

arch/CNRS/arch-X64_JEANZAY.fcm

@@ -36,7 +36,7 @@
 #---------------------------------------------------------------------------------------------
 #
 #
-%XIOS_HOME           $WORK/xios-2.5
+%XIOS_HOME           $WORK/xios-trunk
 %OASIS_HOME          
 
 %NCDF_INC            
@@ -48,7 +48,8 @@
 
 %CPP	             cpp
 %FC                  mpiifort -c -cpp
-%FCFLAGS             -i4 -r8 -O3 -fp-model strict -xCORE-AVX512 -fno-alias
+%PROD_FCFLAGS        -i4 -r8 -O3 -fp-model strict -xCORE-AVX512 -fno-alias
+%DEBUG_FCFLAGS       -i4 -r8 -g -O0 -debug all -traceback -fp-model strict -ftrapuv -check all,noarg_temp_created -fpe-all0 -ftz -init=arrays,snan,huge
 %FFLAGS              %FCFLAGS
 %LD                  mpiifort
 %LDFLAGS             

arch/CNRS/arch-X64_JEANZAY_DEBUG.fcm → arch/CNRS/arch-X64_JEANZAY_GCC.fcm

@@ -28,6 +28,14 @@
 #  - unix variables "$..." are accpeted and will be evaluated before calling fcm.
 #  - fcm variables are starting with a % (and not a $)
 #
+# Module we used:
+#    module purge
+#    module load gcc/8.3.1
+#    module load openmpi/4.1.1
+#    module load hdf5/1.12.0-mpi
+#    module load netcdf-c/4.7.4-mpi
+#    module load netcdf-fortran/4.5.3-mpi
+#
 #---------------------------------------------------------------------------------------------
 #---------------------------------------------------------------------------------------------
 # All NETCDF and HDF paths are empty as they are automatically defined through environment
@@ -36,21 +44,24 @@
 #---------------------------------------------------------------------------------------------
 #
 #
-%XIOS_HOME           $WORK/xios-2.5
+%XIOS_HOME           $WORK/xios-trunk_gcc
 %OASIS_HOME          
+%NETCDF_C_HOME       $( echo $PATH | xargs -d ':' -n 1 | grep netcdf-c )/..
+%NETCDF_F_HOME       $( echo $PATH | xargs -d ':' -n 1 | grep netcdf-fortran )/..
 
-%NCDF_INC            
-%NCDF_LIB            -lnetcdff -lnetcdf
+%NCDF_INC            -I%NETCDF_F_HOME/include -I%NETCDF_C_HOME/include
+%NCDF_LIB            -L%NETCDF_F_HOME/lib -lnetcdff -L%NETCDF_C_HOME/lib -lnetcdf
 %XIOS_INC            -I%XIOS_HOME/inc 
 %XIOS_LIB            -L%XIOS_HOME/lib -lxios -lstdc++
 %OASIS_INC           -I%OASIS_HOME/build/lib/mct -I%OASIS_HOME/build/lib/psmile.MPI1
 %OASIS_LIB           -L%OASIS_HOME/lib -lpsmile.MPI1 -lmct -lmpeu -lscrip
 
-%CPP	             cpp
-%FC                  mpiifort -c -cpp
-%FCFLAGS             -i4 -r8 -g -O0 -debug all -traceback -fp-model strict -ftrapuv -check all,noarg_temp_created -fpe-all0 -ftz -init=arrays,snan,huge
+%CPP	             cpp -Dkey_nosignedzero 
+%FC                  mpif90
+%PROD_FCFLAGS        -fdefault-real-8 -O3 -funroll-all-loops -fcray-pointer -ffree-line-length-none -Wno-missing-include-dirs 
+%DEBUG_FCFLAGS       -fdefault-real-8 -O0 -g -fbacktrace -funroll-all-loops -fcray-pointer -ffree-line-length-none  -Wno-missing-include-dirs -fcheck=all -finit-real=nan
 %FFLAGS              %FCFLAGS
-%LD                  mpiifort
+%LD                  mpif90
 %LDFLAGS             
 %FPPFLAGS            -P -traditional
 %AR                  ar

arch/CNRS/arch-ifort_SPIRIT.fcm

+# Ada  IBM x3750 at french IDRIS, http://www.idris.fr/ada/ada-hw-ada.html
+#
+# NCDF_HOME   root directory containing lib and include subdirectories for netcdf4
+# HDF5_HOME   root directory containing lib and include subdirectories for HDF5
+# XIOS_HOME   root directory containing lib for XIOS
+# OASIS_HOME  root directory containing lib for OASIS
+#
+# NCDF_INC    netcdf4 include file
+# NCDF_LIB    netcdf4 library
+# XIOS_INC    xios include file    (taken into accound only if key_iomput is activated)
+# XIOS_LIB    xios library         (taken into accound only if key_iomput is activated)
+# OASIS_INC   oasis include file   (taken into accound only if key_oasis3 is activated)
+# OASIS_LIB   oasis library        (taken into accound only if key_oasis3 is activated)
+#
+# FC          Fortran compiler command
+# FCFLAGS     Fortran compiler flags
+# FFLAGS      Fortran 77 compiler flags
+# LD          linker
+# LDFLAGS     linker flags, e.g. -L<lib dir> if you have libraries
+# FPPFLAGS    pre-processing flags
+# AR          assembler
+# ARFLAGS     assembler flags
+# MK          make
+# USER_INC    complete list of include files
+# USER_LIB    complete list of libraries to pass to the linker
+# CC          C compiler used to compile conv for AGRIF
+# CFLAGS      compiler flags used with CC
+#
+# Note that:
+#  - unix variables "$..." are accpeted and will be evaluated before calling fcm.
+#  - fcm variables are starting with a % (and not a $)
+#
+#
+%XIOS_HOME           $WORK/xios-trunk
+%OASIS_HOME          /not/defined
+
+%NCDF_INC            -I${NETCDF_FORTRAN_ROOT}/include -I${NETCDF_C_ROOT}/include
+%NCDF_LIB            -L${NETCDF_FORTRAN_ROOT}/lib -lnetcdff -L${NETCDF_C_ROOT}/lib -lnetcdf -L${HDF5_ROOT}/lib -lhdf5_hl -lhdf5 -lz -lcurl
+%XIOS_INC            -I%XIOS_HOME/inc
+%XIOS_LIB            -L%XIOS_HOME/lib -lxios -lstdc++
+%OASIS_INC           -I%OASIS_HOME/build/lib/mct -I%OASIS_HOME/build/lib/psmile.MPI1
+%OASIS_LIB           -L%OASIS_HOME/lib -lpsmile.MPI1 -lmct -lmpeu -lscrip
+
+%CPP	             cpp
+%FC                  mpif90 -c -cpp
+%PROD_FCFLAGS        -DCPP_PARA -i4 -r8 -O3 -fp-model strict -fno-alias
+%DEBUG_FCFLAGS       -i4 -r8 -g -O0 -debug all -traceback -fp-model strict -ftrapuv -check all,noarg_temp_created -fpe-all0 -ftz -init=arrays,snan,huge
+%FFLAGS              %FCFLAGS
+%LD                  mpif90
+%LDFLAGS             
+%FPPFLAGS            -P -traditional
+%AR                  ar
+%ARFLAGS             rs
+%MK                  make
+%USER_INC            %XIOS_INC %OASIS_INC %NCDF_INC
+%USER_LIB            %XIOS_LIB %OASIS_LIB %NCDF_LIB
+
+%CC                  cc
+%CFLAGS              -O0

arch/MERCATOR/arch-X64_KARA_GCC_OMPI_DEBUG.fcm → arch/MERCATOR/arch-X64_AA_GCC.fcm

-# compiler options for KARA (using GNU compiler)
-#
-# GNU_OMPI (NEMO-4.0 + XIOS-2.5)
-# gcc/9.3.0 openmpi/4.0.5_gcc9.3.0 hdf5/1.8.18_gcc9.3.0 netcdf/4.7.1_gcc9.3.0 xios/2.5_rev1903_gcc9.3.0
-#
-# GNU_OMPI (NEMO-4.2 + XIOS-TRUNK)
-# gcc/9.3.0 openmpi/4.0.5_gcc9.3.0 hdf5/1.8.18_gcc9.3.0 netcdf/4.7.1_gcc9.3.0 xios/trunk_rev2136_gcc9.3.0
+# compiler options for AA (using GCC compiler)
 #
 #
 # NCDF_INC    netcdf4 include file
@@ -40,25 +34,27 @@
 #---------------------------------------------------------------------------------------------
 #
 #
-%NCDF_INC            -I${NETCDF_INC}
-%NCDF_LIB            -L${NETCDF_LIB} -lnetcdff -lnetcdf -L${HDF5_LIB} -lhdf5_hl -lhdf5
+%NCDF_INC            ${NETCDF4_INCLUDE}
+%NCDF_LIB            -L${NETCDF4_DIR}/lib -lnetcdff -lnetcdf -L${HDF5_DIR}/lib -lhdf5_hl -lm
 %XIOS_INC            -I${XIOS_INC}
-%XIOS_LIB            -L${XIOS_LIB} -lxios
+%XIOS_LIB            -L${XIOS_LIB} -lxios -lstdc++
 %OASIS_INC           -I${OASIS_DIR}/build/lib/mct -I${OASIS_DIR}/build/lib/psmile.MPI1
 %OASIS_LIB           -L${OASIS_DIR}/lib -lpsmile.MPI1 -lmct -lmpeu -lscrip
 
 %CPP                 cpp -Dkey_nosignedzero
 %FC                  mpif90 -c -cpp
-%FCFLAGS             -fdefault-real-8 -O0 -g -fbacktrace -ftree-vectorize -funroll-all-loops -march=skylake-avx512 -ffree-line-length-none -fcheck=all -finit-real=nan -Wno-missing-include-dirs
+# O3 breaks reproduci/restartabi-lity with gcc/12.2.0
+%PROD_FCFLAGS        -fdefault-real-8 -O2 -funroll-all-loops -fcray-pointer -ffree-line-length-none -fallow-argument-mismatch -Wno-missing-include-dirs
+%DEBUG_FCFLAGS       -fdefault-real-8 -Og -g -fbacktrace -funroll-all-loops -fcray-pointer -ffree-line-length-none -fcheck=all,no-array-temps -finit-real=nan -ffpe-trap=invalid,zero,overflow -ffpe-summary=invalid,zero,overflow -fallow-argument-mismatch -Wno-missing-include-dirs
 %FFLAGS              %FCFLAGS
 %LD                  mpif90
-%LDFLAGS             -lstdc++
+%LDFLAGS             -Wl,-rpath,${HDF5_DIR}/lib -Wl,-rpath=${NETCDF4_DIR}/lib -Wl,-rpath=${XIOS_DIR}/lib
 %FPPFLAGS            -P -traditional
 %AR                  ar
 %ARFLAGS             rs
-%MK                  gmake
+%MK                  make
 %USER_INC            %XIOS_INC %OASIS_INC %NCDF_INC
 %USER_LIB            %XIOS_LIB %OASIS_LIB %NCDF_LIB
 
 %CC                  gcc
-%CFLAGS              -O0
+%CFLAGS              -O0 -fcommon

arch/MERCATOR/arch-X64_AA_GCC_OMPI.env

+if ! grep -qi xios <<< $(module list); then
+  module purge
+  module use /home/ar0s/modules
+  # may23
+  module load prgenv/gnu gcc/12.2.0 openmpi/4.1.1.1 hdf5-parallel/1.12.2 netcdf4-parallel/4.9.1 xios/trunk/r2515_gcc-ompi.may23 &> /dev/null
+  if ! grep -qi xios <<< $(module list); then echo 'module error'; exit 1; fi
+else
+  echo 'xios module already loaded'
+fi
+module list

arch/MERCATOR/arch-X64_AA_GCC_OMPI.fcm

+arch-X64_AA_GCC.fcm
\ No newline at end of file

arch/MERCATOR/arch-X64_AA_GCC_XMPI.env

+if ! grep -qi xios <<< $(module list); then
+  module purge
+  module use /home/ar0s/modules
+  # may23
+  module load prgenv/gnu gcc/12.2.0 hpcx-openmpi/may23 hdf5-parallel/may23 netcdf4-parallel/may23 xios/trunk/r2515_gcc-xmpi.may23 &> /dev/null
+  # may24
+  #module load prgenv/gnu gcc/may24 hpcx-openmpi/2.17.1 hdf5-parallel/may24 netcdf4-parallel/may24 xios/trunk/r2515_gcc-xmpi.may24 &> /dev/null
+  if ! grep -qi xios <<< $(module list); then echo 'module error'; exit 1; fi
+else
+  echo 'xios module already loaded'
+fi
+module list
\ No newline at end of file

arch/MERCATOR/arch-X64_AA_GCC_XMPI.fcm

+arch-X64_AA_GCC.fcm
\ No newline at end of file

arch/MERCATOR/arch-X64_BELENOS_INTEL_IMPI_DEBUG.fcm → arch/MERCATOR/arch-X64_AA_INTEL.fcm

-# compiler options for BELENOS/TARANIS (using INTEL compiler)
-#
-# INTEL_IMPI (XIOS-2.5)
-# intel/2018.5.274 intelmpi/2018.5.274 phdf5/1.8.18 netcdf_par/4.7.1_V2 xios-2.5_rev1903
-#
-# INTEL_IMPI (XIOS-TRUNK must be compiled)
-# gcc/9.2.0 intel/2018.5.274 intelmpi/2018.5.274 phdf5/1.8.18 netcdf_par/4.7.1_V2
+# compiler options for AA (using INTEL compiler)
 #
 #
 # NCDF_INC    netcdf4 include file
@@ -40,25 +34,26 @@
 #---------------------------------------------------------------------------------------------
 #
 #
-%NCDF_INC            -I${NETCDF_INC}
-%NCDF_LIB            -L${NETCDF_LIB} -lnetcdff -lnetcdf -L${PHDF5_LIB_DIR} -lhdf5_hl -lhdf5
+%NCDF_INC            ${NETCDF4_INCLUDE}
+%NCDF_LIB            ${NETCDF4_LIB} ${HDF5_LIB}
 %XIOS_INC            -I${XIOS_INC}
 %XIOS_LIB            -L${XIOS_LIB} -lxios -lstdc++
-%OASIS_INC           -I${OASIS_DIR}/build/lib/mct -I${OASIS_DIR}/build/lib/psmile.MPI1
+%OASIS_INC           -I${OASIS_DIR}/include
 %OASIS_LIB           -L${OASIS_DIR}/lib -lpsmile.MPI1 -lmct -lmpeu -lscrip
 
 %CPP                 cpp
-%FC                  mpiifort -c -cpp
-%FCFLAGS             -march=core-avx2 -i4 -r8 -g -O0 -debug all -traceback -fp-model strict -ftrapuv -check all,noarg_temp_created -fpe-all0 -ftz -init=arrays,snan,huge
+%FC                  mpifort -c -cpp
+%PROD_FCFLAGS        -march=core-avx2 -i4 -r8 -O3 -fp-model strict -fno-alias -align array64byte
+%DEBUG_FCFLAGS       -march=core-avx2 -i4 -r8 -g -O0 -debug all -traceback -fp-model strict -ftrapuv -check all,noarg_temp_created -fpe-all0 -ftz -init=arrays,snan,huge
 %FFLAGS              %FCFLAGS
-%LD                  mpiifort
+%LD                  mpifort
 %LDFLAGS             
 %FPPFLAGS            -P -traditional
 %AR                  ar
 %ARFLAGS             rs
-%MK                  gmake
+%MK                  make
 %USER_INC            %XIOS_INC %OASIS_INC %NCDF_INC
 %USER_LIB            %XIOS_LIB %OASIS_LIB %NCDF_LIB
 
-%CC                  mpiicc
+%CC                  icc
 %CFLAGS              -O0

arch/MERCATOR/arch-X64_AA_INTEL_OMPI.env

+if ! grep -qi xios <<< $(module list); then
+  module purge
+  module use /home/ar0s/modules
+  # may21
+  #module load prgenv/intel intel/2021.4.0 openmpi/4.1.1.1 hdf5-parallel/may21  netcdf4-parallel/may21 xios/trunk/r2515_intel-ompi.may21 &> /dev/null
+  # may23
+  module load prgenv/intel intel/2021.4.0 openmpi/4.1.1.1 hdf5-parallel/may23  netcdf4-parallel/may23 xios/trunk/r2515_intel-ompi.may23 &> /dev/null
+  if ! grep -qi xios <<< $(module list); then echo 'module error'; exit 1; fi
+else
+  echo 'xios module already loaded'
+fi
+module list