Newer
Older
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
.. todo::
TBD
.. contents::
:local:
Prerequisites
-------------
| The NEMO source code is written in *Fortran 95* and
some of its prerequisite tools and libraries are already included in the download.
| It contains the AGRIF_ mesh refinement library; the FCM_ build system ; the PPR_ polynomial reconstruction library and
the IOIPSL_ library for parts of the output.
System dependencies
-------------------
Other requirements should be provided natively by your system or
can be installed from the official repositories of your Unix-like distribution:
- *subversion (svn)* for version control of XIOS sources
- *git* for version control of NEMO sources
- *Perl* interpreter
- *Fortran* compiler (``ifort``, ``gfortran``, ``pgfortran``, ``ftn``, ...),
- *Message Passing Interface (MPI)* implementation (e.g. |OpenMPI|_ or |MPICH|_).
- |NetCDF|_ library with its underlying |HDF|_
**NEMO, by default, takes advantage of some MPI features introduced into the MPI-3 standard.**
.. hint::
The MPI implementation is not strictly essential
since it is possible to compile and run NEMO on a single processor.
However most realistic configurations will require the parallel capabilities of NEMO and
these use the MPI standard.
.. note::
On older systems, that do not support MPI-3 features,
the ``key_mpi2`` preprocessor key should be used at compile time.
This will limit MPI features to those defined within the MPI-2 standard
(but will lose some performance benefits).
.. |OpenMPI| replace:: *OpenMPI*
.. _OpenMPI: https://www.open-mpi.org
.. |MPICH| replace:: *MPICH*
.. _MPICH: https://www.mpich.org
.. |NetCDF| replace:: *Network Common Data Form (NetCDF)*
.. _NetCDF: https://www.unidata.ucar.edu
.. |HDF| replace:: *Hierarchical Data Form (HDF)*
.. _HDF: https://www.hdfgroup.org
Specifics for NetCDF and HDF
----------------------------
NetCDF and HDF versions from official repositories may have not been compiled with MPI support.
However access to all the options available with the XIOS IO-server will require
the parallelism of these libraries.
| **To satisfy these requirements, it is common to have to compile from source
in this order HDF (C library) then NetCDF (C and Fortran libraries)**
| It is also necessary to compile these libraries with the same version of the MPI implementation that
both NEMO and XIOS (see below) have been compiled and linked with.
.. hint::
| It is difficult to define the options for the compilation as
they differ from one architecture to another according to
the hardware used and the software installed.
| The following is provided without any warranty
.. code-block:: console
$ ./configure [--{enable-fortran,disable-shared,enable-parallel}] ...
It is recommended to build the tests ``--enable-parallel-tests`` and run them with ``make check``
Particular versions of these libraries may have their own restrictions.
Note the following requirements for netCDF-4 support:
.. caution::
| When building NetCDF-C library versions older than 4.4.1, use only HDF5 1.8.x versions.
| Combining older NetCDF-C versions with newer HDF5 1.10 versions will create superblock 3 files
that are not readable by lots of older software.
Install XIOS library
--------------------
With the sole exception of running NEMO in mono-processor mode
(in which case output options are limited to those supported by the internal ``IOIPSL`` library),
diagnostic outputs in NEMO are handled by the third party ``XIOS`` library (:xios:`XIOS homepage<>`).
1. Install the NetCDF4 library.
If you want to use single file output you will need to compile the HDF & NetCDF libraries to
allow parallel IO.
2. Download the version of XIOS that you wish to use.
The recommended version is now XIOS2:
$ svn co http://forge.ipsl.jussieu.fr/ioserver/svn/XIOS2/trunk
and follow the instructions in :xios:`XIOS documentation <wiki/documentation>` to compile it.
If you find problems at this stage, support can be found by subscribing to
the :xios:`XIOS mailing list <../mailman/listinfo.cgi/xios-users>` and sending a mail message to it.
Use of XIOS for NEMO IOs is activated using the pre-compiler key ``key_xios``.
.. hint::
Since version 4.2.1 it is also supported the use of XIOS3 by means of the pre-compiler key ``key_xios3``.
This can be checked out with:
.. code:: console
svn co http://forge.ipsl.jussieu.fr/ioserver/svn/XIOS3/trunk
**********************************
Download and install the NEMO code
**********************************

Tomas Lovato
committed
Checkout the NEMO source
------------------------
There are several ways to obtain the NEMO source code. Users who are not familiar with ``git``

Tomas Lovato
committed
and simply want a fixed code to compile and run, can download a tarball from the :tarrepo:`4.2.0 release site<>`
Users who are familiar with ``git`` and likely to use it to manage their own local branches and
modifications, can clone the repository at the release tag:
.. code:: console
git clone --branch 4.2.1 https://forge.nemo-ocean.eu/nemo/nemo.git nemo_4.2.1
Experienced developers who may wish to experiment with other branches or code more recent than the
release -perhaps with a view to returning developements to the system, can clone the main repository
and then switch to the tagged release:
.. code:: console
git clone https://forge.nemo-ocean.eu/nemo/nemo.git
cd nemo
git switch --detach 4.2.1
Description of 1\ :sup:`st` level tree structure
------------------------------------------------
+---------------+-------------------------------------------------+
| :file:`arch` | Compilation settings |
+---------------+-------------------------------------------------+
| :file:`cfgs` | :doc:`Reference configurations <cfgs>` |
+---------------+-------------------------------------------------+
| :file:`ext` | Dependencies included |
| | (``AGRIF``, ``FCM``, ``PPR`` & ``IOIPSL``) |
+---------------+-------------------------------------------------+
| :file:`mk` | Compilation scripts |
+---------------+-------------------------------------------------+
| :file:`src` | NEMO codebase |
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
+---------------+-------------------------------------------------+
| :file:`tests` | :doc:`Test cases <tests>` |
| | (unsupported) |
+---------------+-------------------------------------------------+
| :file:`tools` | :doc:`Utilities <tools>` |
| | to {pre,post}process data |
+---------------+-------------------------------------------------+
| :file:`sette` | :doc:`SETTE <sette>` |
| | a code testing framework |
+---------------+-------------------------------------------------+
Setup your architecture configuration file
------------------------------------------
All compiler options in NEMO are controlled using files in :file:`./arch/arch-'my_arch'.fcm` where
``my_arch`` is the name of the computing architecture
(generally following the pattern ``HPCC-compiler`` or ``OS-compiler``).
It is recommended to copy and rename an configuration file from an architecture similar to your own.
You will need to set appropriate values for all of the variables in the file.
In particular the FCM variables:
``%NCDF_HOME``; ``%HDF5_HOME`` and ``%XIOS_HOME`` should be set to
the installation directories used for XIOS installation
.. code-block:: sh
%NCDF_HOME /usr/local/path/to/netcdf
%HDF5_HOME /usr/local/path/to/hdf5
%XIOS_HOME /home/$( whoami )/path/to/xios-trunk
%OASIS_HOME /home/$( whoami )/path/to/oasis
It is also possible to use the script :file:`./arch/build_arch-auto.sh`
to create a machine specific architecture file by setting the required environment variables.
See the script usage to retrieve detailed information.
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
***********************
Preparing an experiment
***********************
Create and compile a new configuration
--------------------------------------
The main script to {re}compile and create executable is called :file:`makenemo` located at
the root of the working copy.
It is used to identify the routines you need from the source code, to build the makefile and run it.
As an example, compile a :file:`MY_GYRE` configuration from GYRE with 'my_arch':
.. code-block:: sh
./makenemo –m 'my_arch' –r GYRE -n 'MY_GYRE'
Then at the end of the configuration compilation,
:file:`MY_GYRE` directory will have the following structure.
+------------+----------------------------------------------------------------------------+
| Directory | Purpose |
+============+============================================================================+
| ``BLD`` | BuiLD folder: target executable, headers, libs, preprocessed routines, ... |
+------------+----------------------------------------------------------------------------+
| ``EXP00`` | Run folder: link to executable, namelists, ``*.xml`` and IOs |
+------------+----------------------------------------------------------------------------+
| ``EXPREF`` | Files under version control only for :doc:`official configurations <cfgs>` |
+------------+----------------------------------------------------------------------------+
| ``MY_SRC`` | New routines or modified copies of NEMO sources |
+------------+----------------------------------------------------------------------------+
| ``WORK`` | Links to all raw routines from :file:`./src` considered |
+------------+----------------------------------------------------------------------------+
After successful execution of :file:`makenemo` command,
the executable called `nemo` is available in the :file:`EXP00` directory
Viewing and changing list of active CPP keys
--------------------------------------------
For a given configuration (here called ``MY_CONFIG``),
the list of active CPP keys can be found in :file:`./cfgs/'MYCONFIG'/cpp_MY_CONFIG.fcm`
This text file can be edited by hand or with :file:`makenemo` to change the list of active CPP keys.
Once changed, one needs to recompile ``nemo`` in order for this change to be taken in account.
Note that most NEMO configurations will need to specify the following CPP keys:
``key_xios`` for IOs. MPI parallelism is activated by default. Use ``key_mpi_off`` to compile without MPI.
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
Configure XIOS outputs
----------------------
XIOS allows for an efficient management of diagnostic outputs as
the time averaging, file writing and even some simple arithmetic or regridding are carried out in
parallel to the NEMO model run.
This page gives a basic introduction to using XIOS with NEMO.
Additional information are available at the XIOS :xios:`wiki<>` and in the NEMO reference manual.
Use of XIOS for NEMO IOs is activated using the pre-compiler key ``key_xios``.
XIOS is controlled by means of XML input files that should be copied to your model run directory before running the model.
Examples of these files can be found in the reference configurations (:file:`./cfgs`).
The XIOS executable expects to find a file called :file:`iodef.xml` in the model run directory.
In NEMO we have made the decision to use include statements in the :file:`iodef.xml` file to include:
- :file:`field_def_nemo-oce.xml` (for physics),
- :file:`field_def_nemo-ice.xml` (for ice),
- :file:`field_def_nemo-pisces.xml` (for biogeochemistry) and
- :file:`domain_def.xml` from the :file:`./cfgs/SHARED` directory.
Most users will not need to modify :file:`domain_def.xml` or :file:`field_def_nemo-???.xml` unless
they want to add new diagnostics to the NEMO code.
The definition of the output files is organized into separate :file:`file_definition.xml` files which
are included in the :file:`iodef.xml` file.
XIOS be used along with NEMO with two different modes:
Detached Mode
^^^^^^^^^^^^^
In detached mode the XIOS executable is executed on separate cores from the NEMO model.
This is the recommended method for using XIOS for realistic model runs.
To use this mode set ``using_server`` to ``true`` at the bottom of the :file:`iodef.xml` file:
.. code-block:: xml
<variable id="using_server" type="boolean">true</variable>
Make sure there is a copy (or link to) your XIOS executable in the working directory and
in your job submission script allocate processors to XIOS.
Attached Mode
^^^^^^^^^^^^^
In attached mode XIOS runs on each of the cores used by NEMO.
This method is less efficient than the detached mode but can be more convenient for testing or
with small configurations.
To activate this mode simply set ``using_server`` to false in the :file:`iodef.xml` file
More :file:`makenemo` options
-----------------------------
``makenemo`` has several other options that can control which source files are selected and
the operation of the build process itself.

Tomas Lovato
committed
.. rli:: https://forge.nemo-ocean.eu/nemo/nemo/-/raw/4.2.0/makenemo

Tomas Lovato
committed
:lines: 119-151
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
:caption: Output of ``makenemo -h``
These options can be useful for maintaining several code versions with only minor differences but
they should be used sparingly.
Note however the ``-j`` option which should be used more routinely to speed up the build process.
For example:
.. code-block:: sh
./makenemo –m 'my_arch' –r GYRE -n 'MY_GYRE' -j 8
will compile up to 8 processes simultaneously.
Default behaviour
-----------------
At the first use,
you need the ``-m`` option to specify the architecture configuration file
(compiler and its options, routines and libraries to include),
then for next compilation, it is assumed you will be using the same compiler.
If the ``-n`` option is not specified the last compiled configuration will be used.
Tools used during the process
-----------------------------
The various bash scripts used by ``makenemo`` (for instance, to create the
``WORK`` directory) are located in the ``mk`` subdirectory. In most cases, there
should be no need for user intervention with these scripts. Occasionally,
incomplete builds can leave the environment in a indeterminate state. If
problems are experienced with subsequent attempts then running:
.. code-block:: sh
./makenemo –m 'my_arch' –r GYRE -n 'MY_GYRE' clean
will prepare the directories for a fresh attempt and remove any intermediate files
that may be causing issues.
The reference configurations that may be provided to the ``-r`` argument of ``makenemo``
are listed in the :file:`cfgs/ref_cfgs.txt` file:
.. code-block:: sh
AGRIF_DEMO OCE ICE TOP NST
AMM12 OCE
C1D_PAPA OCE
GYRE_BFM OCE TOP
GYRE_PISCES OCE TOP
ORCA2_OFF_PISCES OCE TOP OFF
ORCA2_OFF_TRC OCE TOP OFF
ORCA2_SAS_ICE OCE ICE NST SAS
ORCA2_ICE_PISCES OCE TOP ICE NST ABL
ORCA2_ICE_ABL OCE ICE ABL
SPITZ12 OCE ICE
WED025 OCE ICE
User added configurations will be listed in :file:`cfgs/work_cfgs.txt`
*******************
Running the model
*******************
Once :file:`makenemo` has run successfully, a symbolic link to
the ``nemo`` executable is available in :file:`./cfgs/MY_CONFIG/EXP00`.
For the reference configurations, the :file:`EXP00` folder also contains the initial input files
(namelists, ``*.xml`` files for the IOs, ...).
If the configuration needs other input files, they have to be placed here.
.. code-block:: sh
cd 'MY_CONFIG'/EXP00
mpirun -n $NPROCS ./nemo # $NPROCS is the number of processes
# mpirun is your MPI wrapper
If you are completely new to NEMO or this is your first experience of running MPI-based
ocean models, then you may wish to consider activating the ``SETTE`` testing framework. If
your HPC environment matches or is similar to one of the supported environments then you
may find it simpler to activate one of the reference configurations this way since SETTE
includes example submission scripts for common batch environments and scripts to fetch
all the input files for the more realistic reference configurations. See the
:doc:`SETTE <sette>` section of this guide for more details.