Skip to content
Snippets Groups Projects
Commit cfd38ac3 authored by Andrew Coward's avatar Andrew Coward
Browse files

Updated a bit more of the sette guide to match latest version

parent c2199ed7
No related branches found
No related tags found
No related merge requests found
Hide whitespace changes
Inline Side-by-side
Showing
with 104 additions and 72 deletions
source/sette.rst
+ 104
− 72
View file @ cfd38ac3
......@@ -348,110 +348,142 @@ a basic report is presented to the user which lists the various successes or fai
./sette.sh
<lots of progress information and compilation stages followed by:>
<lots of progress information and compilation stages>
./sette_rpt.sh -V different_branch
SETTE validation report generated for :
trunk @ c1604aac (with local changes)
on X86_ARCHER2-Cray arch file
branch_5.0 @ 5ac534cc (with local changes)
on ANEMONE-ifort23 arch file
!!---------------1st pass------------------!!
!----restart----!
GYRE_PISCES run.stat restartability passed : 24335_c1604aac+
GYRE_PISCES tracer.stat restartability passed : 24335_c1604aac+
ORCA2_ICE_PISCES run.stat restartability passed : 24335_c1604aac+
ORCA2_ICE_PISCES tracer.stat restartability passed : 24335_c1604aac+
ORCA2_OFF_PISCES tracer.stat restartability passed : 24335_c1604aac+
AMM12 run.stat restartability passed : 24335_c1604aac+
ORCA2_SAS_ICE run.stat restartability passed : 24335_c1604aac+
AGRIF_DEMO run.stat restartability passed : 24335_c1604aac+
AGRIF_DEMO tracer.stat restartability passed : 24335_c1604aac+
WED025 run.stat restartability passed : 24335_c1604aac+
ISOMIP+ run.stat restartability passed : 24335_c1604aac+
OVERFLOW run.stat restartability passed : 24335_c1604aac+
LOCK_EXCHANGE run.stat restartability passed : 24335_c1604aac+
VORTEX run.stat restartability passed : 24335_c1604aac+
ICE_AGRIF run.stat restartability passed : 24335_c1604aac+
SWG run.stat restartability passed : 24335_c1604aac+
ORCA2_ICE_PISCES run.stat restartability passed : 24341_5ac534cc+
ORCA2_ICE_PISCES tracer.stat restartability passed : 24341_5ac534cc+
ORCA2_OFF_PISCES tracer.stat restartability passed : 24341_5ac534cc+
AMM12 run.stat restartability passed : 24341_5ac534cc+
AGRIF_DEMO run.stat restartability passed : 24341_5ac534cc+
AGRIF_DEMO tracer.stat restartability passed : 24341_5ac534cc+
WED025 run.stat restartability passed : 24341_5ac534cc+
GYRE_PISCES run.stat restartability passed : 24341_5ac534cc+
GYRE_PISCES tracer.stat restartability passed : 24341_5ac534cc+
ORCA2_SAS_ICE run.stat restartability passed : 24341_5ac534cc+
C1D_PAPA run.stat restartability passed : 24341_5ac534cc+
SWG run.stat restartability passed : 24341_5ac534cc+
ICE_AGRIF run.stat restartability passed : 24341_5ac534cc+
OVERFLOW run.stat restartability passed : 24341_5ac534cc+
LOCK_EXCHANGE run.stat restartability passed : 24341_5ac534cc+
VORTEX run.stat restartability passed : 24341_5ac534cc+
ISOMIP+ run.stat restartability passed : 24341_5ac534cc+
IWAVE run.stat restartability passed : 24341_5ac534cc+
!----repro----!
GYRE_PISCES run.stat reproducibility passed : 24335_c1604aac+
GYRE_PISCES tracer.stat reproducibility passed : 24335_c1604aac+
ORCA2_ICE_PISCES run.stat reproducibility passed : 24335_c1604aac+
ORCA2_ICE_PISCES tracer.stat reproducibility passed : 24335_c1604aac+
ORCA2_OFF_PISCES tracer.stat reproducibility passed : 24335_c1604aac+
AMM12 run.stat reproducibility passed : 24335_c1604aac+
ORCA2_SAS_ICE run.stat reproducibility passed : 24335_c1604aac+
ORCA2_ICE_OBS run.stat reproducibility passed : 24335_c1604aac+
AGRIF_DEMO run.stat reproducibility passed : 24335_c1604aac+
AGRIF_DEMO tracer.stat reproducibility passed : 24335_c1604aac+
WED025 run.stat reproducibility passed : 24335_c1604aac+
ISOMIP+ run.stat reproducibility passed : 24335_c1604aac+
VORTEX run.stat reproducibility passed : 24335_c1604aac+
ICE_AGRIF run.stat reproducibility passed : 24335_c1604aac+
SWG run.stat reproducibility passed : 24335_c1604aac+
ORCA2_ICE_PISCES run.stat reproducibility passed : 24341_5ac534cc+
ORCA2_ICE_PISCES tracer.stat reproducibility passed : 24341_5ac534cc+
ORCA2_OFF_PISCES tracer.stat reproducibility passed : 24341_5ac534cc+
AMM12 run.stat reproducibility passed : 24341_5ac534cc+
AGRIF_DEMO run.stat reproducibility passed : 24341_5ac534cc+
AGRIF_DEMO tracer.stat reproducibility passed : 24341_5ac534cc+
WED025 run.stat reproducibility passed : 24341_5ac534cc+
GYRE_PISCES run.stat reproducibility passed : 24341_5ac534cc+
GYRE_PISCES tracer.stat reproducibility passed : 24341_5ac534cc+
ORCA2_SAS_ICE run.stat reproducibility passed : 24341_5ac534cc+
ORCA2_ICE_OBS run.stat reproducibility passed : 24341_5ac534cc+
ORCA2_ICE_OBS obs.stat reproducibility passed : 24341_5ac534cc+
SWG run.stat reproducibility passed : 24341_5ac534cc+
ICE_AGRIF run.stat reproducibility passed : 24341_5ac534cc+
VORTEX run.stat reproducibility passed : 24341_5ac534cc+
ISOMIP+ run.stat reproducibility passed : 24341_5ac534cc+
!----transform----!
ORCA2_ICE_PISCES transformed variants MISSING : 24341_5ac534cc+
ORCA2_OFF_PISCES transformed variants MISSING : 24341_5ac534cc+
AMM12 transformed variants MISSING : 24341_5ac534cc+
AGRIF_DEMO transformed variants MISSING : 24341_5ac534cc+
WED025 transformed variants MISSING : 24341_5ac534cc+
GYRE_PISCES transformed variants MISSING : 24341_5ac534cc+
ORCA2_SAS_ICE transformed variants MISSING : 24341_5ac534cc+
ORCA2_ICE_OBS transformed variants MISSING : 24341_5ac534cc+
C1D_PAPA transformed variants MISSING : 24341_5ac534cc+
SWG transformed variants MISSING : 24341_5ac534cc+
ICE_AGRIF transformed variants MISSING : 24341_5ac534cc+
OVERFLOW transformed variants MISSING : 24341_5ac534cc+
LOCK_EXCHANGE transformed variants MISSING : 24341_5ac534cc+
VORTEX transformed variants MISSING : 24341_5ac534cc+
ISOMIP+ transformed variants MISSING : 24341_5ac534cc+
IWAVE transformed variants MISSING : 24341_5ac534cc+
!----phyopt----!
OVERFLOW/sco_FCT2_flux_cen-ahm1000 ocean.output phyopts passed : 24341_5ac534cc+
OVERFLOW/sco_FCT2_flux_ubs ocean.output phyopts passed : 24341_5ac534cc+
OVERFLOW/zps_FCT2_flux_ubs ocean.output phyopts passed : 24341_5ac534cc+
OVERFLOW/sco_FCT4_flux_cen-ahm1000 ocean.output phyopts passed : 24341_5ac534cc+
OVERFLOW/sco_FCT4_flux_ubs ocean.output phyopts passed : 24341_5ac534cc+
OVERFLOW/zps_FCT4_flux_ubs ocean.output phyopts passed : 24341_5ac534cc+
OVERFLOW/zps_FCT4_vect_een ocean.output phyopts passed : 24341_5ac534cc+
LOCK_EXCHANGE/FCT2_flux_cen2 ocean.output phyopts passed : 24341_5ac534cc+
LOCK_EXCHANGE/FCT2_vect_ene ocean.output phyopts passed : 24341_5ac534cc+
LOCK_EXCHANGE/FCT2_vect_ens ocean.output phyopts passed : 24341_5ac534cc+
LOCK_EXCHANGE/FCT2_vect_een ocean.output phyopts passed : 24341_5ac534cc+
LOCK_EXCHANGE/FCT2_flux_ubs ocean.output phyopts passed : 24341_5ac534cc+
LOCK_EXCHANGE/FCT2_vect_eenH ocean.output phyopts passed : 24341_5ac534cc+
LOCK_EXCHANGE/FCT4_flux_cen2 ocean.output phyopts passed : 24341_5ac534cc+
LOCK_EXCHANGE/FCT4_vect_ene ocean.output phyopts passed : 24341_5ac534cc+
LOCK_EXCHANGE/FCT4_vect_een ocean.output phyopts passed : 24341_5ac534cc+
LOCK_EXCHANGE/FCT4_flux_ubs ocean.output phyopts passed : 24341_5ac534cc+
LOCK_EXCHANGE/FCT4_vect_eenH ocean.output phyopts passed : 24341_5ac534cc+
LOCK_EXCHANGE/FCT4_vect_ens ocean.output phyopts passed : 24341_5ac534cc+
!----agrif check----!
ORCA2 AGRIF vs ORCA2 NOAGRIF run.stat unchanged - passed : 24335_c1604aac+ 15541
ORCA2 AGRIF vs ORCA2 NOAGRIF run.stat unchanged passed : 24341_5ac534cc+ 24341_5ac534cc+
!----result comparison check----!
check result differences between :
VALID directory : /work/n01/n01/acc/NEMO/4.2.0/sette/NEMO_VALIDATION/MAIN at rev 24335_c1604aac+
VALID directory : /dssgfs01/scratch/acc/branches/pre_release_5.0/sette/NEMO_VALIDATION/branch_5.0 at rev 24341_5ac534cc+
and
REFERENCE directory : /work/n01/n01/acc/NEMO/4.2.0/sette/NEMO_VALIDATION/MAIN at rev 15150
REFERENCE directory : /dssgfs01/scratch/acc/branches/pre_release_5.0/sette/NEMO_VALIDATION/different_branch at rev 24332_16c60ec5+
GYRE_PISCES run.stat files are identical
GYRE_PISCES tracer.stat files are identical
ORCA2_ICE_PISCES run.stat files are DIFFERENT (results are different after 1 time steps)
ORCA2_ICE_PISCES tracer.stat files are DIFFERENT (results are different after 1 time steps)
ORCA2_OFF_PISCES tracer.stat files are DIFFERENT (results are different after 1 time steps)
AMM12 run.stat files are DIFFERENT (results are different after 1 time steps)
ORCA2_SAS_ICE run.stat files are DIFFERENT (results are different after 3 time steps)
AGRIF_DEMO run.stat files are DIFFERENT (results are different after 1 time steps)
AGRIF_DEMO tracer.stat files are DIFFERENT (results are different after 1 time steps)
WED025 run.stat files are DIFFERENT (results are different after 1 time steps)
ISOMIP+ run.stat files are DIFFERENT (results are different after 1 time steps)
VORTEX run.stat files are DIFFERENT (results are different after 1 time steps)
ICE_AGRIF run.stat files are DIFFERENT (results are different after 2 time steps)
OVERFLOW run.stat files are identical
LOCK_EXCHANGE run.stat files are identical
SWG run.stat files are identical
ORCA2_ICE_PISCES run.stat files are DIFFERENT (results are different after 1 time steps) (LONG)
ORCA2_ICE_PISCES tracer.stat files are DIFFERENT (results are different after 1 time steps) (LONG)
Report timing differences between REFERENCE and VALID (if available) :
GYRE_PISCES ref. time: 22.805 cur. time: 40.126 diff.: 17.321
ORCA2_ICE_PISCES ref. time: 133.614 cur. time: 63.484 diff.: -70.13
ORCA2_OFF_PISCES ref. time: 172.469 cur. time: 471.569 diff.: 299.1
AMM12 ref. time: 139.546 cur. time: 222.412 diff.: 82.866
WED025 ref. time: 462.350 cur. time: 913.722 diff.: 451.372
ISOMIP+ ref. time: 33.319 cur. time: 69.091 diff.: 35.772
OVERFLOW ref. time: 16.864 cur. time: 35.474 diff.: 18.61
LOCK_EXCHANGE ref. time: 11.912 cur. time: 13.802 diff.: 1.89
ORCA2_ICE_PISCES ref. time: 263.349203 cur. time: 211.858526 diff.: -51.4907
The report shows the result of restartability and reproducibility tests on the whole range of test
configurations. Passing these tests is a necessary and mandatory requirement for any official release
of NEMO. Note these tests are not sufficient to guarantee restartability and reproducibility in all
user-defined configrations and anyone running configurations, which are not close variants of the
reference or test configurations, should conduct their own tests.
This report ends by comparing the latest results against a reference set (as defined in ``param.cfg``).
In this case the comparison is between revisions that were known to introduce numerical differences and
between runs with different levels of compiler optimisation. This is confined by the comparsion but the
report is most useful when numerical results are not expected to change between revisions and when changes
are expected to provide a performance benefit. It is not shown here but, on many terminals, test failures or
performance drops are presented in red to highlight areas of concern.
``SETTE`` will also carry out tests on the PSyclone code transformation (if enabled) and tests with
alternative physics options that are defined for some configurations. The range of tests carried out
can be varied via the ``-x`` command-line option to ``sette.sh``. The default behaviour is equivalent to
.. code-block:: bash
sette.sh -x "COMPILE RESTART REPRO CORRUPT PHYOPTS TRANSFORM"
This report ends by comparing the latest results against a reference set (as defined in
``param.cfg``). Only ``ORCA2_ICE_PISCES`` had been run on the alternative branch and the
missing comparisons have been omitted here. In this case the comparison is between
branches that were known to introduce numerical differences. This is confined by the
comparsion but the report is most useful when numerical results are not expected to change
between revisions and when changes are expected to provide a performance benefit. It is
not shown here but, on many terminals, test failures or performance drops are presented in
red to highlight areas of concern.
The set of tests executed by default are set in ``param.cfg`` in the ``TEST_CONFIGS``
environment variable:
.. code-block:: bash
grep TEST_CONFIGS= param.cfg
export TEST_CONFIGS=(${SETTE_TEST_CONFIGS[@]:-"ORCA2_ICE_PISCES ORCA2_OFF_PISCES AMM12 AGRIF WED025 GYRE_PISCES SAS ORCA2_ICE_OBS SWG ICE_AGRIF OVERFLOW LOCK_EXCHANGE VORTEX ISOMIP+"})
grep TEST_CONFIG_AVAIL param.default
TEST_CONFIG_AVAILABLE=(ORCA2_ICE_PISCES ORCA2_OFF_PISCES AMM12 AGRIF_DEMO WED025 GYRE_PISCES ORCA2_SAS_ICE ORCA2_ICE_OBS C1D_PAPA SWG ICE_AGRIF OVERFLOW LOCK_EXCHANGE VORTEX ISOMIP+ IWAVE)
export TEST_CONFIGS=(${SETTE_TEST_CONFIGS[@]:-${TEST_CONFIG_AVAILABLE[@]}})
Note this set can be overridden by externally setting the ``SETTE_TEST_CONFIGS``
environment variable but individual or sub-sets of tests can also be selected by arguments
......
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment