From cfd38ac38d0d226137a7807f7c5293583f50c9e7 Mon Sep 17 00:00:00 2001
From: Andrew Coward <acc@noc.ac.uk>
Date: Fri, 6 Dec 2024 17:00:37 +0000
Subject: [PATCH] Updated a bit more of the sette guide to match latest version
---
source/sette.rst | 176 ++++++++++++++++++++++++++++-------------------
1 file changed, 104 insertions(+), 72 deletions(-)
diff --git a/source/sette.rst b/source/sette.rst
index d08c9f9..e20b356 100644
--- a/source/sette.rst
+++ b/source/sette.rst
@@ -348,110 +348,142 @@ a basic report is presented to the user which lists the various successes or fai
./sette.sh
- <lots of progress information and compilation stages followed by:>
+ <lots of progress information and compilation stages>
+ ./sette_rpt.sh -V different_branch
SETTE validation report generated for :
- trunk @ c1604aac (with local changes)
-
- on X86_ARCHER2-Cray arch file
+ branch_5.0 @ 5ac534cc (with local changes)
+ on ANEMONE-ifort23 arch file
!!---------------1st pass------------------!!
!----restart----!
- GYRE_PISCES run.stat restartability passed : 24335_c1604aac+
- GYRE_PISCES tracer.stat restartability passed : 24335_c1604aac+
- ORCA2_ICE_PISCES run.stat restartability passed : 24335_c1604aac+
- ORCA2_ICE_PISCES tracer.stat restartability passed : 24335_c1604aac+
- ORCA2_OFF_PISCES tracer.stat restartability passed : 24335_c1604aac+
- AMM12 run.stat restartability passed : 24335_c1604aac+
- ORCA2_SAS_ICE run.stat restartability passed : 24335_c1604aac+
- AGRIF_DEMO run.stat restartability passed : 24335_c1604aac+
- AGRIF_DEMO tracer.stat restartability passed : 24335_c1604aac+
- WED025 run.stat restartability passed : 24335_c1604aac+
- ISOMIP+ run.stat restartability passed : 24335_c1604aac+
- OVERFLOW run.stat restartability passed : 24335_c1604aac+
- LOCK_EXCHANGE run.stat restartability passed : 24335_c1604aac+
- VORTEX run.stat restartability passed : 24335_c1604aac+
- ICE_AGRIF run.stat restartability passed : 24335_c1604aac+
- SWG run.stat restartability passed : 24335_c1604aac+
+ ORCA2_ICE_PISCES run.stat restartability passed : 24341_5ac534cc+
+ ORCA2_ICE_PISCES tracer.stat restartability passed : 24341_5ac534cc+
+ ORCA2_OFF_PISCES tracer.stat restartability passed : 24341_5ac534cc+
+ AMM12 run.stat restartability passed : 24341_5ac534cc+
+ AGRIF_DEMO run.stat restartability passed : 24341_5ac534cc+
+ AGRIF_DEMO tracer.stat restartability passed : 24341_5ac534cc+
+ WED025 run.stat restartability passed : 24341_5ac534cc+
+ GYRE_PISCES run.stat restartability passed : 24341_5ac534cc+
+ GYRE_PISCES tracer.stat restartability passed : 24341_5ac534cc+
+ ORCA2_SAS_ICE run.stat restartability passed : 24341_5ac534cc+
+ C1D_PAPA run.stat restartability passed : 24341_5ac534cc+
+ SWG run.stat restartability passed : 24341_5ac534cc+
+ ICE_AGRIF run.stat restartability passed : 24341_5ac534cc+
+ OVERFLOW run.stat restartability passed : 24341_5ac534cc+
+ LOCK_EXCHANGE run.stat restartability passed : 24341_5ac534cc+
+ VORTEX run.stat restartability passed : 24341_5ac534cc+
+ ISOMIP+ run.stat restartability passed : 24341_5ac534cc+
+ IWAVE run.stat restartability passed : 24341_5ac534cc+
!----repro----!
- GYRE_PISCES run.stat reproducibility passed : 24335_c1604aac+
- GYRE_PISCES tracer.stat reproducibility passed : 24335_c1604aac+
- ORCA2_ICE_PISCES run.stat reproducibility passed : 24335_c1604aac+
- ORCA2_ICE_PISCES tracer.stat reproducibility passed : 24335_c1604aac+
- ORCA2_OFF_PISCES tracer.stat reproducibility passed : 24335_c1604aac+
- AMM12 run.stat reproducibility passed : 24335_c1604aac+
- ORCA2_SAS_ICE run.stat reproducibility passed : 24335_c1604aac+
- ORCA2_ICE_OBS run.stat reproducibility passed : 24335_c1604aac+
- AGRIF_DEMO run.stat reproducibility passed : 24335_c1604aac+
- AGRIF_DEMO tracer.stat reproducibility passed : 24335_c1604aac+
- WED025 run.stat reproducibility passed : 24335_c1604aac+
- ISOMIP+ run.stat reproducibility passed : 24335_c1604aac+
- VORTEX run.stat reproducibility passed : 24335_c1604aac+
- ICE_AGRIF run.stat reproducibility passed : 24335_c1604aac+
- SWG run.stat reproducibility passed : 24335_c1604aac+
+ ORCA2_ICE_PISCES run.stat reproducibility passed : 24341_5ac534cc+
+ ORCA2_ICE_PISCES tracer.stat reproducibility passed : 24341_5ac534cc+
+ ORCA2_OFF_PISCES tracer.stat reproducibility passed : 24341_5ac534cc+
+ AMM12 run.stat reproducibility passed : 24341_5ac534cc+
+ AGRIF_DEMO run.stat reproducibility passed : 24341_5ac534cc+
+ AGRIF_DEMO tracer.stat reproducibility passed : 24341_5ac534cc+
+ WED025 run.stat reproducibility passed : 24341_5ac534cc+
+ GYRE_PISCES run.stat reproducibility passed : 24341_5ac534cc+
+ GYRE_PISCES tracer.stat reproducibility passed : 24341_5ac534cc+
+ ORCA2_SAS_ICE run.stat reproducibility passed : 24341_5ac534cc+
+ ORCA2_ICE_OBS run.stat reproducibility passed : 24341_5ac534cc+
+ ORCA2_ICE_OBS obs.stat reproducibility passed : 24341_5ac534cc+
+ SWG run.stat reproducibility passed : 24341_5ac534cc+
+ ICE_AGRIF run.stat reproducibility passed : 24341_5ac534cc+
+ VORTEX run.stat reproducibility passed : 24341_5ac534cc+
+ ISOMIP+ run.stat reproducibility passed : 24341_5ac534cc+
+
+ !----transform----!
+ ORCA2_ICE_PISCES transformed variants MISSING : 24341_5ac534cc+
+ ORCA2_OFF_PISCES transformed variants MISSING : 24341_5ac534cc+
+ AMM12 transformed variants MISSING : 24341_5ac534cc+
+ AGRIF_DEMO transformed variants MISSING : 24341_5ac534cc+
+ WED025 transformed variants MISSING : 24341_5ac534cc+
+ GYRE_PISCES transformed variants MISSING : 24341_5ac534cc+
+ ORCA2_SAS_ICE transformed variants MISSING : 24341_5ac534cc+
+ ORCA2_ICE_OBS transformed variants MISSING : 24341_5ac534cc+
+ C1D_PAPA transformed variants MISSING : 24341_5ac534cc+
+ SWG transformed variants MISSING : 24341_5ac534cc+
+ ICE_AGRIF transformed variants MISSING : 24341_5ac534cc+
+ OVERFLOW transformed variants MISSING : 24341_5ac534cc+
+ LOCK_EXCHANGE transformed variants MISSING : 24341_5ac534cc+
+ VORTEX transformed variants MISSING : 24341_5ac534cc+
+ ISOMIP+ transformed variants MISSING : 24341_5ac534cc+
+ IWAVE transformed variants MISSING : 24341_5ac534cc+
+
+ !----phyopt----!
+ OVERFLOW/sco_FCT2_flux_cen-ahm1000 ocean.output phyopts passed : 24341_5ac534cc+
+ OVERFLOW/sco_FCT2_flux_ubs ocean.output phyopts passed : 24341_5ac534cc+
+ OVERFLOW/zps_FCT2_flux_ubs ocean.output phyopts passed : 24341_5ac534cc+
+ OVERFLOW/sco_FCT4_flux_cen-ahm1000 ocean.output phyopts passed : 24341_5ac534cc+
+ OVERFLOW/sco_FCT4_flux_ubs ocean.output phyopts passed : 24341_5ac534cc+
+ OVERFLOW/zps_FCT4_flux_ubs ocean.output phyopts passed : 24341_5ac534cc+
+ OVERFLOW/zps_FCT4_vect_een ocean.output phyopts passed : 24341_5ac534cc+
+ LOCK_EXCHANGE/FCT2_flux_cen2 ocean.output phyopts passed : 24341_5ac534cc+
+ LOCK_EXCHANGE/FCT2_vect_ene ocean.output phyopts passed : 24341_5ac534cc+
+ LOCK_EXCHANGE/FCT2_vect_ens ocean.output phyopts passed : 24341_5ac534cc+
+ LOCK_EXCHANGE/FCT2_vect_een ocean.output phyopts passed : 24341_5ac534cc+
+ LOCK_EXCHANGE/FCT2_flux_ubs ocean.output phyopts passed : 24341_5ac534cc+
+ LOCK_EXCHANGE/FCT2_vect_eenH ocean.output phyopts passed : 24341_5ac534cc+
+ LOCK_EXCHANGE/FCT4_flux_cen2 ocean.output phyopts passed : 24341_5ac534cc+
+ LOCK_EXCHANGE/FCT4_vect_ene ocean.output phyopts passed : 24341_5ac534cc+
+ LOCK_EXCHANGE/FCT4_vect_een ocean.output phyopts passed : 24341_5ac534cc+
+ LOCK_EXCHANGE/FCT4_flux_ubs ocean.output phyopts passed : 24341_5ac534cc+
+ LOCK_EXCHANGE/FCT4_vect_eenH ocean.output phyopts passed : 24341_5ac534cc+
+ LOCK_EXCHANGE/FCT4_vect_ens ocean.output phyopts passed : 24341_5ac534cc+
!----agrif check----!
- ORCA2 AGRIF vs ORCA2 NOAGRIF run.stat unchanged - passed : 24335_c1604aac+ 15541
+ ORCA2 AGRIF vs ORCA2 NOAGRIF run.stat unchanged passed : 24341_5ac534cc+ 24341_5ac534cc+
!----result comparison check----!
-
check result differences between :
- VALID directory : /work/n01/n01/acc/NEMO/4.2.0/sette/NEMO_VALIDATION/MAIN at rev 24335_c1604aac+
+ VALID directory : /dssgfs01/scratch/acc/branches/pre_release_5.0/sette/NEMO_VALIDATION/branch_5.0 at rev 24341_5ac534cc+
and
- REFERENCE directory : /work/n01/n01/acc/NEMO/4.2.0/sette/NEMO_VALIDATION/MAIN at rev 15150
+ REFERENCE directory : /dssgfs01/scratch/acc/branches/pre_release_5.0/sette/NEMO_VALIDATION/different_branch at rev 24332_16c60ec5+
- GYRE_PISCES run.stat files are identical
- GYRE_PISCES tracer.stat files are identical
- ORCA2_ICE_PISCES run.stat files are DIFFERENT (results are different after 1 time steps)
- ORCA2_ICE_PISCES tracer.stat files are DIFFERENT (results are different after 1 time steps)
- ORCA2_OFF_PISCES tracer.stat files are DIFFERENT (results are different after 1 time steps)
- AMM12 run.stat files are DIFFERENT (results are different after 1 time steps)
- ORCA2_SAS_ICE run.stat files are DIFFERENT (results are different after 3 time steps)
- AGRIF_DEMO run.stat files are DIFFERENT (results are different after 1 time steps)
- AGRIF_DEMO tracer.stat files are DIFFERENT (results are different after 1 time steps)
- WED025 run.stat files are DIFFERENT (results are different after 1 time steps)
- ISOMIP+ run.stat files are DIFFERENT (results are different after 1 time steps)
- VORTEX run.stat files are DIFFERENT (results are different after 1 time steps)
- ICE_AGRIF run.stat files are DIFFERENT (results are different after 2 time steps)
- OVERFLOW run.stat files are identical
- LOCK_EXCHANGE run.stat files are identical
- SWG run.stat files are identical
+ ORCA2_ICE_PISCES run.stat files are DIFFERENT (results are different after 1 time steps) (LONG)
+ ORCA2_ICE_PISCES tracer.stat files are DIFFERENT (results are different after 1 time steps) (LONG)
Report timing differences between REFERENCE and VALID (if available) :
- GYRE_PISCES ref. time: 22.805 cur. time: 40.126 diff.: 17.321
- ORCA2_ICE_PISCES ref. time: 133.614 cur. time: 63.484 diff.: -70.13
- ORCA2_OFF_PISCES ref. time: 172.469 cur. time: 471.569 diff.: 299.1
- AMM12 ref. time: 139.546 cur. time: 222.412 diff.: 82.866
- WED025 ref. time: 462.350 cur. time: 913.722 diff.: 451.372
- ISOMIP+ ref. time: 33.319 cur. time: 69.091 diff.: 35.772
- OVERFLOW ref. time: 16.864 cur. time: 35.474 diff.: 18.61
- LOCK_EXCHANGE ref. time: 11.912 cur. time: 13.802 diff.: 1.89
-
+ ORCA2_ICE_PISCES ref. time: 263.349203 cur. time: 211.858526 diff.: -51.4907
+
+
The report shows the result of restartability and reproducibility tests on the whole range of test
configurations. Passing these tests is a necessary and mandatory requirement for any official release
of NEMO. Note these tests are not sufficient to guarantee restartability and reproducibility in all
user-defined configrations and anyone running configurations, which are not close variants of the
reference or test configurations, should conduct their own tests.
-This report ends by comparing the latest results against a reference set (as defined in ``param.cfg``).
-In this case the comparison is between revisions that were known to introduce numerical differences and
-between runs with different levels of compiler optimisation. This is confined by the comparsion but the
-report is most useful when numerical results are not expected to change between revisions and when changes
-are expected to provide a performance benefit. It is not shown here but, on many terminals, test failures or
-performance drops are presented in red to highlight areas of concern.
+``SETTE`` will also carry out tests on the PSyclone code transformation (if enabled) and tests with
+alternative physics options that are defined for some configurations. The range of tests carried out
+can be varied via the ``-x`` command-line option to ``sette.sh``. The default behaviour is equivalent to
+
+.. code-block:: bash
+
+ sette.sh -x "COMPILE RESTART REPRO CORRUPT PHYOPTS TRANSFORM"
+
+This report ends by comparing the latest results against a reference set (as defined in
+``param.cfg``). Only ``ORCA2_ICE_PISCES`` had been run on the alternative branch and the
+missing comparisons have been omitted here. In this case the comparison is between
+branches that were known to introduce numerical differences. This is confined by the
+comparsion but the report is most useful when numerical results are not expected to change
+between revisions and when changes are expected to provide a performance benefit. It is
+not shown here but, on many terminals, test failures or performance drops are presented in
+red to highlight areas of concern.
The set of tests executed by default are set in ``param.cfg`` in the ``TEST_CONFIGS``
environment variable:
.. code-block:: bash
- grep TEST_CONFIGS= param.cfg
- export TEST_CONFIGS=(${SETTE_TEST_CONFIGS[@]:-"ORCA2_ICE_PISCES ORCA2_OFF_PISCES AMM12 AGRIF WED025 GYRE_PISCES SAS ORCA2_ICE_OBS SWG ICE_AGRIF OVERFLOW LOCK_EXCHANGE VORTEX ISOMIP+"})
+ grep TEST_CONFIG_AVAIL param.default
+ TEST_CONFIG_AVAILABLE=(ORCA2_ICE_PISCES ORCA2_OFF_PISCES AMM12 AGRIF_DEMO WED025 GYRE_PISCES ORCA2_SAS_ICE ORCA2_ICE_OBS C1D_PAPA SWG ICE_AGRIF OVERFLOW LOCK_EXCHANGE VORTEX ISOMIP+ IWAVE)
+ export TEST_CONFIGS=(${SETTE_TEST_CONFIGS[@]:-${TEST_CONFIG_AVAILABLE[@]}})
Note this set can be overridden by externally setting the ``SETTE_TEST_CONFIGS``
environment variable but individual or sub-sets of tests can also be selected by arguments
--
GitLab