Skip to content
GitLab
Explore
Sign in
Register
Primary navigation
Search or go to…
Project
User guide
Manage
Activity
Members
Labels
Plan
Issues
2
Issue boards
Milestones
Iterations
Requirements
External wiki
Code
Merge requests
0
Repository
Branches
Commits
Tags
Repository graph
Compare revisions
Locked files
Build
Pipelines
Jobs
Pipeline schedules
Test cases
Artifacts
Deploy
Releases
Model registry
Analyze
Model experiments
Help
Help
Support
GitLab documentation
Compare GitLab plans
Community forum
Contribute to GitLab
Provide feedback
Terms and privacy
Keyboard shortcuts
?
Snippets
Groups
Projects
Show more breadcrumbs
NEMO Workspace
Documentation
User guide
Commits
cfd38ac3
Commit
cfd38ac3
authored
3 months ago
by
Andrew Coward
Browse files
Options
Downloads
Patches
Plain Diff
Updated a bit more of the sette guide to match latest version
parent
c2199ed7
No related branches found
No related tags found
No related merge requests found
Changes
1
Hide whitespace changes
Inline
Side-by-side
Showing
1 changed file
source/sette.rst
+104
-72
104 additions, 72 deletions
source/sette.rst
with
104 additions
and
72 deletions
source/sette.rst
+
104
−
72
View file @
cfd38ac3
...
...
@@ -348,110 +348,142 @@ a basic report is presented to the user which lists the various successes or fai
./sette.sh
<lots of progress information and compilation stages
followed by:
>
<lots of progress information and compilation stages>
./sette_rpt.sh -V different_branch
SETTE validation report generated for :
trunk @ c1604aac (with local changes)
on X86_ARCHER2-Cray arch file
branch_5.0 @ 5ac534cc (with local changes)
on ANEMONE-ifort23 arch file
!!---------------1st pass------------------!!
!----restart----!
GYRE_PISCES run.stat restartability passed : 24335_c1604aac+
GYRE_PISCES tracer.stat restartability passed : 24335_c1604aac+
ORCA2_ICE_PISCES run.stat restartability passed : 24335_c1604aac+
ORCA2_ICE_PISCES tracer.stat restartability passed : 24335_c1604aac+
ORCA2_OFF_PISCES tracer.stat restartability passed : 24335_c1604aac+
AMM12 run.stat restartability passed : 24335_c1604aac+
ORCA2_SAS_ICE run.stat restartability passed : 24335_c1604aac+
AGRIF_DEMO run.stat restartability passed : 24335_c1604aac+
AGRIF_DEMO tracer.stat restartability passed : 24335_c1604aac+
WED025 run.stat restartability passed : 24335_c1604aac+
ISOMIP+ run.stat restartability passed : 24335_c1604aac+
OVERFLOW run.stat restartability passed : 24335_c1604aac+
LOCK_EXCHANGE run.stat restartability passed : 24335_c1604aac+
VORTEX run.stat restartability passed : 24335_c1604aac+
ICE_AGRIF run.stat restartability passed : 24335_c1604aac+
SWG run.stat restartability passed : 24335_c1604aac+
ORCA2_ICE_PISCES run.stat restartability passed : 24341_5ac534cc+
ORCA2_ICE_PISCES tracer.stat restartability passed : 24341_5ac534cc+
ORCA2_OFF_PISCES tracer.stat restartability passed : 24341_5ac534cc+
AMM12 run.stat restartability passed : 24341_5ac534cc+
AGRIF_DEMO run.stat restartability passed : 24341_5ac534cc+
AGRIF_DEMO tracer.stat restartability passed : 24341_5ac534cc+
WED025 run.stat restartability passed : 24341_5ac534cc+
GYRE_PISCES run.stat restartability passed : 24341_5ac534cc+
GYRE_PISCES tracer.stat restartability passed : 24341_5ac534cc+
ORCA2_SAS_ICE run.stat restartability passed : 24341_5ac534cc+
C1D_PAPA run.stat restartability passed : 24341_5ac534cc+
SWG run.stat restartability passed : 24341_5ac534cc+
ICE_AGRIF run.stat restartability passed : 24341_5ac534cc+
OVERFLOW run.stat restartability passed : 24341_5ac534cc+
LOCK_EXCHANGE run.stat restartability passed : 24341_5ac534cc+
VORTEX run.stat restartability passed : 24341_5ac534cc+
ISOMIP+ run.stat restartability passed : 24341_5ac534cc+
IWAVE run.stat restartability passed : 24341_5ac534cc+
!----repro----!
GYRE_PISCES run.stat reproducibility passed : 24335_c1604aac+
GYRE_PISCES tracer.stat reproducibility passed : 24335_c1604aac+
ORCA2_ICE_PISCES run.stat reproducibility passed : 24335_c1604aac+
ORCA2_ICE_PISCES tracer.stat reproducibility passed : 24335_c1604aac+
ORCA2_OFF_PISCES tracer.stat reproducibility passed : 24335_c1604aac+
AMM12 run.stat reproducibility passed : 24335_c1604aac+
ORCA2_SAS_ICE run.stat reproducibility passed : 24335_c1604aac+
ORCA2_ICE_OBS run.stat reproducibility passed : 24335_c1604aac+
AGRIF_DEMO run.stat reproducibility passed : 24335_c1604aac+
AGRIF_DEMO tracer.stat reproducibility passed : 24335_c1604aac+
WED025 run.stat reproducibility passed : 24335_c1604aac+
ISOMIP+ run.stat reproducibility passed : 24335_c1604aac+
VORTEX run.stat reproducibility passed : 24335_c1604aac+
ICE_AGRIF run.stat reproducibility passed : 24335_c1604aac+
SWG run.stat reproducibility passed : 24335_c1604aac+
ORCA2_ICE_PISCES run.stat reproducibility passed : 24341_5ac534cc+
ORCA2_ICE_PISCES tracer.stat reproducibility passed : 24341_5ac534cc+
ORCA2_OFF_PISCES tracer.stat reproducibility passed : 24341_5ac534cc+
AMM12 run.stat reproducibility passed : 24341_5ac534cc+
AGRIF_DEMO run.stat reproducibility passed : 24341_5ac534cc+
AGRIF_DEMO tracer.stat reproducibility passed : 24341_5ac534cc+
WED025 run.stat reproducibility passed : 24341_5ac534cc+
GYRE_PISCES run.stat reproducibility passed : 24341_5ac534cc+
GYRE_PISCES tracer.stat reproducibility passed : 24341_5ac534cc+
ORCA2_SAS_ICE run.stat reproducibility passed : 24341_5ac534cc+
ORCA2_ICE_OBS run.stat reproducibility passed : 24341_5ac534cc+
ORCA2_ICE_OBS obs.stat reproducibility passed : 24341_5ac534cc+
SWG run.stat reproducibility passed : 24341_5ac534cc+
ICE_AGRIF run.stat reproducibility passed : 24341_5ac534cc+
VORTEX run.stat reproducibility passed : 24341_5ac534cc+
ISOMIP+ run.stat reproducibility passed : 24341_5ac534cc+
!----transform----!
ORCA2_ICE_PISCES transformed variants MISSING : 24341_5ac534cc+
ORCA2_OFF_PISCES transformed variants MISSING : 24341_5ac534cc+
AMM12 transformed variants MISSING : 24341_5ac534cc+
AGRIF_DEMO transformed variants MISSING : 24341_5ac534cc+
WED025 transformed variants MISSING : 24341_5ac534cc+
GYRE_PISCES transformed variants MISSING : 24341_5ac534cc+
ORCA2_SAS_ICE transformed variants MISSING : 24341_5ac534cc+
ORCA2_ICE_OBS transformed variants MISSING : 24341_5ac534cc+
C1D_PAPA transformed variants MISSING : 24341_5ac534cc+
SWG transformed variants MISSING : 24341_5ac534cc+
ICE_AGRIF transformed variants MISSING : 24341_5ac534cc+
OVERFLOW transformed variants MISSING : 24341_5ac534cc+
LOCK_EXCHANGE transformed variants MISSING : 24341_5ac534cc+
VORTEX transformed variants MISSING : 24341_5ac534cc+
ISOMIP+ transformed variants MISSING : 24341_5ac534cc+
IWAVE transformed variants MISSING : 24341_5ac534cc+
!----phyopt----!
OVERFLOW/sco_FCT2_flux_cen-ahm1000 ocean.output phyopts passed : 24341_5ac534cc+
OVERFLOW/sco_FCT2_flux_ubs ocean.output phyopts passed : 24341_5ac534cc+
OVERFLOW/zps_FCT2_flux_ubs ocean.output phyopts passed : 24341_5ac534cc+
OVERFLOW/sco_FCT4_flux_cen-ahm1000 ocean.output phyopts passed : 24341_5ac534cc+
OVERFLOW/sco_FCT4_flux_ubs ocean.output phyopts passed : 24341_5ac534cc+
OVERFLOW/zps_FCT4_flux_ubs ocean.output phyopts passed : 24341_5ac534cc+
OVERFLOW/zps_FCT4_vect_een ocean.output phyopts passed : 24341_5ac534cc+
LOCK_EXCHANGE/FCT2_flux_cen2 ocean.output phyopts passed : 24341_5ac534cc+
LOCK_EXCHANGE/FCT2_vect_ene ocean.output phyopts passed : 24341_5ac534cc+
LOCK_EXCHANGE/FCT2_vect_ens ocean.output phyopts passed : 24341_5ac534cc+
LOCK_EXCHANGE/FCT2_vect_een ocean.output phyopts passed : 24341_5ac534cc+
LOCK_EXCHANGE/FCT2_flux_ubs ocean.output phyopts passed : 24341_5ac534cc+
LOCK_EXCHANGE/FCT2_vect_eenH ocean.output phyopts passed : 24341_5ac534cc+
LOCK_EXCHANGE/FCT4_flux_cen2 ocean.output phyopts passed : 24341_5ac534cc+
LOCK_EXCHANGE/FCT4_vect_ene ocean.output phyopts passed : 24341_5ac534cc+
LOCK_EXCHANGE/FCT4_vect_een ocean.output phyopts passed : 24341_5ac534cc+
LOCK_EXCHANGE/FCT4_flux_ubs ocean.output phyopts passed : 24341_5ac534cc+
LOCK_EXCHANGE/FCT4_vect_eenH ocean.output phyopts passed : 24341_5ac534cc+
LOCK_EXCHANGE/FCT4_vect_ens ocean.output phyopts passed : 24341_5ac534cc+
!----agrif check----!
ORCA2 AGRIF vs ORCA2 NOAGRIF run.stat unchanged
-
passed
:
243
35_c1604aac+ 15541
ORCA2 AGRIF vs ORCA2 NOAGRIF
run.stat unchanged
passed
:
243
41_5ac534cc+ 24341_5ac534cc+
!----result comparison check----!
check result differences between :
VALID directory : /
work/n01/n01/acc/NEMO/4.2
.0/sette/NEMO_VALIDATION/
MAIN
at rev 243
35_c1604aa
c+
VALID directory : /
dssgfs01/scratch/acc/branches/pre_release_5
.0/sette/NEMO_VALIDATION/
branch_5.0
at rev 243
41_5ac534c
c+
and
REFERENCE directory : /
work/n01/n01/acc/NEMO/4.2.0/sette/NEMO_VALIDATION/MAIN at rev 15150
REFERENCE directory : /
dssgfs01/scratch/acc/branches/pre_release_5.0/sette/NEMO_VALIDATION/different_branch at rev 24332_16c60ec5+
GYRE_PISCES run.stat files are identical
GYRE_PISCES tracer.stat files are identical
ORCA2_ICE_PISCES run.stat files are DIFFERENT (results are different after 1 time steps)
ORCA2_ICE_PISCES tracer.stat files are DIFFERENT (results are different after 1 time steps)
ORCA2_OFF_PISCES tracer.stat files are DIFFERENT (results are different after 1 time steps)
AMM12 run.stat files are DIFFERENT (results are different after 1 time steps)
ORCA2_SAS_ICE run.stat files are DIFFERENT (results are different after 3 time steps)
AGRIF_DEMO run.stat files are DIFFERENT (results are different after 1 time steps)
AGRIF_DEMO tracer.stat files are DIFFERENT (results are different after 1 time steps)
WED025 run.stat files are DIFFERENT (results are different after 1 time steps)
ISOMIP+ run.stat files are DIFFERENT (results are different after 1 time steps)
VORTEX run.stat files are DIFFERENT (results are different after 1 time steps)
ICE_AGRIF run.stat files are DIFFERENT (results are different after 2 time steps)
OVERFLOW run.stat files are identical
LOCK_EXCHANGE run.stat files are identical
SWG run.stat files are identical
ORCA2_ICE_PISCES run.stat files are DIFFERENT (results are different after 1 time steps) (LONG)
ORCA2_ICE_PISCES tracer.stat files are DIFFERENT (results are different after 1 time steps) (LONG)
Report timing differences between REFERENCE and VALID (if available) :
GYRE_PISCES ref. time: 22.805 cur. time: 40.126 diff.: 17.321
ORCA2_ICE_PISCES ref. time: 133.614 cur. time: 63.484 diff.: -70.13
ORCA2_OFF_PISCES ref. time: 172.469 cur. time: 471.569 diff.: 299.1
AMM12 ref. time: 139.546 cur. time: 222.412 diff.: 82.866
WED025 ref. time: 462.350 cur. time: 913.722 diff.: 451.372
ISOMIP+ ref. time: 33.319 cur. time: 69.091 diff.: 35.772
OVERFLOW ref. time: 16.864 cur. time: 35.474 diff.: 18.61
LOCK_EXCHANGE ref. time: 11.912 cur. time: 13.802 diff.: 1.89
ORCA2_ICE_PISCES ref. time: 263.349203 cur. time: 211.858526 diff.: -51.4907
The report shows the result of restartability and reproducibility tests on the whole range of test
configurations. Passing these tests is a necessary and mandatory requirement for any official release
of NEMO. Note these tests are not sufficient to guarantee restartability and reproducibility in all
user-defined configrations and anyone running configurations, which are not close variants of the
reference or test configurations, should conduct their own tests.
This report ends by comparing the latest results against a reference set (as defined in ``param.cfg``).
In this case the comparison is between revisions that were known to introduce numerical differences and
between runs with different levels of compiler optimisation. This is confined by the comparsion but the
report is most useful when numerical results are not expected to change between revisions and when changes
are expected to provide a performance benefit. It is not shown here but, on many terminals, test failures or
performance drops are presented in red to highlight areas of concern.
``SETTE`` will also carry out tests on the PSyclone code transformation (if enabled) and tests with
alternative physics options that are defined for some configurations. The range of tests carried out
can be varied via the ``-x`` command-line option to ``sette.sh``. The default behaviour is equivalent to
.. code-block:: bash
sette.sh -x "COMPILE RESTART REPRO CORRUPT PHYOPTS TRANSFORM"
This report ends by comparing the latest results against a reference set (as defined in
``param.cfg``). Only ``ORCA2_ICE_PISCES`` had been run on the alternative branch and the
missing comparisons have been omitted here. In this case the comparison is between
branches that were known to introduce numerical differences. This is confined by the
comparsion but the report is most useful when numerical results are not expected to change
between revisions and when changes are expected to provide a performance benefit. It is
not shown here but, on many terminals, test failures or performance drops are presented in
red to highlight areas of concern.
The set of tests executed by default are set in ``param.cfg`` in the ``TEST_CONFIGS``
environment variable:
.. code-block:: bash
grep TEST_CONFIGS= param.cfg
export TEST_CONFIGS=(${SETTE_TEST_CONFIGS[@]:-"ORCA2_ICE_PISCES ORCA2_OFF_PISCES AMM12 AGRIF WED025 GYRE_PISCES SAS ORCA2_ICE_OBS SWG ICE_AGRIF OVERFLOW LOCK_EXCHANGE VORTEX ISOMIP+"})
grep TEST_CONFIG_AVAIL param.default
TEST_CONFIG_AVAILABLE=(ORCA2_ICE_PISCES ORCA2_OFF_PISCES AMM12 AGRIF_DEMO WED025 GYRE_PISCES ORCA2_SAS_ICE ORCA2_ICE_OBS C1D_PAPA SWG ICE_AGRIF OVERFLOW LOCK_EXCHANGE VORTEX ISOMIP+ IWAVE)
export TEST_CONFIGS=(${SETTE_TEST_CONFIGS[@]:-${TEST_CONFIG_AVAILABLE[@]}})
Note this set can be overridden by externally setting the ``SETTE_TEST_CONFIGS``
environment variable but individual or sub-sets of tests can also be selected by arguments
...
...
This diff is collapsed.
Click to expand it.
Simon Mueller
@smueller
mentioned in issue
#22 (closed)
·
3 months ago
mentioned in issue
#22 (closed)
mentioned in issue #22
Toggle commit list
Preview
0%
Loading
Try again
or
attach a new file
.
Cancel
You are about to add
0
people
to the discussion. Proceed with caution.
Finish editing this message first!
Save comment
Cancel
Please
register
or
sign in
to comment