# WRF

> [!WARNING]
>
> WRF is [user software](<https://docs.cscs.ch/policies/support/index.md#user-applications> "User Applications") on Alps. This guide is provided based on our experiences helping users – however we can’t provide the same level of support as we do for [supported software](<https://docs.cscs.ch/policies/support/index.md#officially-supported-applications> "Officially Supported Applications"). See the [main applications page](<https://docs.cscs.ch/software/index.md#applications-and-frameworks> "Applications and frameworks") for more information.

[The Weather Research &amp; Forecasting Model](<https://www.mmm.ucar.edu/models/wrf>) (WRF) is a numerical weather prediction system designed for both atmospheric research and weather forecasting.

It is used for CPU-only simulation on [Eiger](<https://docs.cscs.ch/clusters/eiger/index.md#eiger>), for which this guide applies.

## Using Spack

Spack provides the [wrf](<https://packages.spack.io/package.html?name=wrf>) package, which we can install using the [uenv-spack](<https://docs.cscs.ch/build-install/uenv/index.md#building-software-using-spack> "Building software using Spack") tool.

First create a working directory where you will install the software. Here, we create it in your project’s [Store](<https://docs.cscs.ch/storage/filesystems/index.md#store>) path, where the package can be accessed by all users in your project.

```bash
mkdir $STORE/wrf
cd $STORE/wrf
```

Then follow the steps in the [uenv-spack](<https://docs.cscs.ch/build-install/uenv/index.md#building-software-using-spack> "Building software using Spack") guide to install `uenv-spack`

```bash
git clone https://github.com/eth-cscs/uenv-spack.git
# add the uenv-spack executable to your path
export PATH=$PWD/uenv-spack:$PATH
```

The [`prgenv-gnu`](<https://docs.cscs.ch/software/prgenv/prgenv-gnu/index.md#prgenv-gnu>) uenv is suitable for building WRF.

```text
uenv start prgenv-gnu/24.11:v2 --view=spack
```

In this example we use the latest version of `prgenv-gnu` on Eiger at the time of writing—check the `prgenv-gnu` [guide](<https://docs.cscs.ch/software/prgenv/prgenv-gnu/index.md#prgenv-gnu> "prgenv-gnu") for the latest version.

```bash
# build the latest version provided by the version of Spack used by prgenv-gnu
$ uenv-spack $PWD/build --uarch=zen2 --specs=wrf

# build a specific version
$ uenv-spack $PWD/build --uarch=zen2 --specs=wrf@4.5.2

# build a specific version with WRF-Chem enabled
$ uenv-spack $PWD/build --uarch=zen2 --specs=wrf@4.5.2 +chem
```

> [!NOTE]
>
> See the wrf Spack [package documentation](<https://packages.spack.io/package.html?name=wrf>) for information about options that can be enabled or disabled.

Finally, it is time to build the software:

```text
cd build
./build
```

This will take 30-60 minutes, while Spack builds some dependencies then WRF.

### Using the Spack installation

The installation creates a module file in the `wrf/build` path, that you created. Assuming you have installed it in the `$STORE` path for your project, add the following to the top of your sbatch script:

```bash
#SBATCH --uenv=prgenv-gnu/24.11:v2

module use $STORE/wrf/build/modules
module load wrf
```

> [!NOTE]
>
> **Modules installed by Spack**
>
> Spack creates a module for every installed package:
>
> ```console
> $ module use $STORE/wrf/build/modules
> $ module avail
>
> ------------------ /capstor/store/cscs/cscs/csstaff/wrf/build/modules ------------------
>    boost/1.86.0             kokkos-tools/develop    netlib-scalapack/2.2.0
>    cmake/3.30.5             kokkos/4.4.01           ninja/1.12.1
>    cray-mpich/8.1.30        libtree/3.1.1           openblas/0.3.28
>    fftw/3.3.10              lua/5.4.6               osu-micro-benchmarks/5.9
>    fmt/11.0.2               lz4/1.10.0              python/3.12.5
>    gcc/13.3.0               meson/1.5.1             superlu/5.3.0
>    gsl/2.8                  netcdf-c/4.9.2          wrf/4.6.1
>    hdf5/1.14.5              netcdf-cxx/4.2          zlib-ng/2.2.1
>    kokkos-kernels/4.4.01    netcdf-fortran/4.6.1
>
> $ module load wrf
> $ which wrf.exe
> /capstor/store/cscs/cscs/csstaff/wrf/build/store/linux-sles15-zen2/gcc-13.3.0/wrf-4.6.1-owj2dsfeslzkulaobdqbad4kh6ojh6n5/main/wrf.exe
> ```

## CRYOWRF

[CRYOWRF](<https://gitlabext.wsl.ch/atmospheric-models/CRYOWRF>) is a coupled atmosphere-snow cover model with WRF acting as the atmospheric core and SNOWPACK acting as snow cover model.

Building CRYOWRF is a three step process:

1. install the dependencies like [`parallel-netcdf`](<https://packages.spack.io/package.html?name=parallel-netcdf>)
2. build the SNOWPACK extension
3. Build the bundled WRF

> [!NOTE]
>
> This workflow was developed in July 2025 using the most recent commit `8f83858f` of [CRYOWRF](<https://gitlabext.wsl.ch/atmospheric-models/CRYOWRF>) (committed in August 2023).
>
> It isn’t very easy to install, and we have tried to streamline the process as much as possible, so take your time and follow the instructions closely.

> [!WARNING]
>
> **Eiger only**
>
> This guide is for building on [Eiger](<https://docs.cscs.ch/clusters/eiger/index.md#eiger>), which is an x86-based system.
>
> Building on the Grace-Hopper clusters like [Daint](<https://docs.cscs.ch/clusters/daint/index.md#daint>) is

We use [`prgenv-gnu/24.11:v2`](<https://docs.cscs.ch/software/prgenv/prgenv-gnu/index.md#prgenv-gnu>) [uenv](<https://docs.cscs.ch/software/uenv/index.md#uenv>).

### Step 1: install required packages

> [!NOTE]
>
> **Full script**
>
> Here is the full script for the steps described in this step.
>
> ```bash
> # the root directory under which all dependencies and WRF/CRYOWRF will be installed
> export WRFROOT=$STORE/wrf
> # the directory where we will install the dependencies of WRF/CRYOWRF/WPS
> export WRFDEPS=$WRFROOT/dependencies
>
> # assume that we are crating WRFROOT for the first time
> mkdir $WRFROOT
>
> # download and install uenv-spack
> git clone https://github.com/eth-cscs/uenv-spack.git $WRFROOT/uenv-spack
> (cd $WRFROOT/uenv-spack/uenv-spack && ./bootstrap)
>
> # start the uenv with the spack view enabled
> # warning: bootstrap the uenv-spack tool _before_ starting the uenv
> uenv start prgenv-gnu/24.11:v2 --view=spack
>
> # configure the environment and build it
> $WRFROOT/uenv-spack/uenv-spack $WRFDEPS --uarch=zen2 --specs='parallel-netcdf,jasper~shared,libpng,zlib-ng'
> cd $WRFDEPS
> ./build
>
> # finish the session
> exit
> ```

The first step is to create an empty directory where everything will be installed. Here, we create it in your project’s [Store](<https://docs.cscs.ch/storage/filesystems/index.md#store>) path, where the package can be accessed by all users in your project.

```bash
# the root directory under which all dependencies and WRF/CRYOWRF will be installed
export WRFROOT=$STORE/wrf
# the directory where we will install the dependencies of WRF/CRYOWRF/WPS
export WRFDEPS=$WRFROOT/dependencies
# assume that we are crating WRFROOT for the first time
mkdir $WRFROOT

cd $WRFROOT
```

The following dependencies that are not provided by `prgenv-gnu` are required:

- `parallel-netcdf`: used by WRF.
- `jasper~shared`: used by WPS (`~shared` will build static libraries, required by WPS).
- `zlib-ng` and `libpng`: used by WPS.

Then follow the steps in the [uenv-spack](<https://docs.cscs.ch/build-install/uenv/index.md#building-software-using-spack> "Building software using Spack") guide to install `uenv-spack`, which will be used to install the dependencies

```bash
# download and install uenv-spack
git clone https://github.com/eth-cscs/uenv-spack.git $WRFROOT/uenv-spack
export PATH=$PWD/uenv-spack:$PATH
```

> [!WARNING]
>
> Run the step of downloading and running the bootstrap script for `uenv-spack` *before* starting a uenv.

Now we configure and build the environment, note that the final “build” phase will take a around 5-10 minutes.

```bash
# start the uenv with the spack view enabled
uenv start prgenv-gnu/24.11:v2 --view=spack

$WRFROOT/uenv-spack/uenv-spack $WRFDEPS --uarch=zen2 --specs='parallel-netcdf,jasper~shared,libpng,zlib-ng'
cd $WRFDEPS
./build
```

Now the dependencies are installed, finish the uenv spack session:

```bash
exit
```

> [!NOTE]
>
> This step is performed once, and will install the software in `$WRFDEPS`, where they can be used to build and run WRF.

### Step 2: build SNOWPACK

> [!NOTE]
>
> **Full script**
>
> Here is the full script, that is described in detail in this step.
>
> ```bash
> uenv start --view-default prgenv-gnu/24.11:v2
>
> # set the paths to match those used in Step 1
> export WRFROOT=$STORE/wrf
> export WRFDEPS=$WRFROOT/dependencies
>
> git clone https://gitlabext.wsl.ch/atmospheric-models/CRYOWRF.git $WRFROOT/CRYOWRF
> cd $WRFROOT/CRYOWRF
>
> export NETCDF=/user-environment/env/default
> export HDF5=/user-environment/env/default
> export PNETCDF=$WRFDEPS/view
>
> export WRF_EM_CORE=1
> export WRF_NMM_CORE=0
> export WRF_DA_CORE=0
>
> export WRF_CHEM=0
> export WRF_KPP=0
>
> export NETCDF4=1
> export WRFIO_NCD_LARGE_FILE_SUPPORT=1
> export WRFIO_NCD_NO_LARGE_FILE_SUPPORT=0
>
> export CC=mpicc
> export FC=mpifort
> export CXX=mpic++
>
> ./clean.sh
> ./compiler_snow_libs.sh
>
> # finish the session
> exit
> ```

Use the `default` view of `prgenv-gnu` to build SNOWPACK, WRF and WPS:

```text
# set the paths to match those used in Step 1
export WRFROOT=$STORE/wrf
export WRFDEPS=$WRFROOT/dependencies

uenv start prgenv-gnu/24.11:v2 --view=default
```

> [!NOTE]
>
> You don’t need to load any modules: the `default` view will add everything to your environment.

First download the CRYOWRF software:

```bash
git clone https://gitlabext.wsl.ch/atmospheric-models/CRYOWRF.git $WRFROOT/CRYOWRF
cd $WRFROOT/CRYOWRF
```

Set the following environment variables:

```bash
export NETCDF=/user-environment/env/default
export HDF5=/user-environment/env/default
export PNETCDF=$WRFDEPS/view

export WRF_EM_CORE=1
export WRF_NMM_CORE=0
export WRF_DA_CORE=0

export WRF_CHEM=0
export WRF_KPP=0

export NETCDF4=1
export WRFIO_NCD_LARGE_FILE_SUPPORT=1
export WRFIO_NCD_NO_LARGE_FILE_SUPPORT=0

export CC=mpicc
export FC=mpifort
export CXX=mpic++
```

Then compile SNOWPACK:

```text
./clean.sh
./compiler_snow_libs.sh
```

### Step 3: build WRF

> [!NOTE]
>
> **Full script**
>
> Here is the full script, that is described in detail in this step.
>
> ```bash
> uenv start --view-default prgenv-gnu/24.11:v2
>
> # set the paths to match those used in Step 1
> export WRFROOT=$STORE/wrf
> export WRFDEPS=$WRFROOT/dependencies
>
> # required for the CRYOWRF build to find SNOWPACK built in step 1
> export SNOWLIBS=$WRFROOT/CRYOWRF/snpack_for_wrf
>
> # set variables used by the WRF build tool
> export NETCDF=/user-environment/env/default
> export HDF5=/user-environment/env/default
> export PNETCDF=$WRFDEPS/view
>
> export WRF_EM_CORE=1
> export WRF_NMM_CORE=0
> export WRF_DA_CORE=0
>
> export WRF_CHEM=0
> export WRF_KPP=0
>
> export NETCDF4=1
> export WRFIO_NCD_LARGE_FILE_SUPPORT=1
> export WRFIO_NCD_NO_LARGE_FILE_SUPPORT=0
>
> export CC=mpicc
> export FC=mpifort
> export CXX=mpic++
>
> git clone https://gitlabext.wsl.ch/atmospheric-models/CRYOWRF.git $WRFROOT/CRYOWRF
> cd $WRFROOT/CRYOWRF/WRF
>
> ./clean -a
> # [choose option 34][nesting: choose option 1] when prompted by configure
> ./configure
>
> # edit configure.wrf
> vim configure.wrf
> # SFC             =    gfortran
> # SCC             =    gcc
> # CCOMP           =    gcc
> # DM_FC           =    mpif90
> # DM_CC           =    mpicc
> # FC              =    mpif90
> # FCBASEOPTS      =    $(FCBASEOPTS_NO_G) $(FCDEBUG) -fallow-argument-mismatch -fallow-invalid-boz -g
> # NETCDFPATH      =    /user-environment/env/default
>
> sed -i 's|hdf5hl|hdf5_hl|g' configure.wrf
> ./compile em_real -j 64 &> log_compile
> ```

The CRYOWRF repository includes a copy of WRF v4.2.1, that has been modified to integrate the SNOWPACK extension build in the previous step.

```bash
export SNOWLIBS=$WRFROOT/CRYOWRF/snpack_for_wrf
cd $WRFROOT/CRYOWRF/WRF
./clean -a
# [choose option 34][nesting: choose option 1] when prompted by configure
./configure
```

> [!NOTE]
>
> **Set `SNOWLIBS`**
>
> The `SNOWLIBS` environment variable needs to be set so that WRF can find the extension we compiled earlier.

Open the configure.wrf file that was generated by calling `./configure`, and update the following lines:

```bash
SFC             =    gfortran
SCC             =    gcc
CCOMP           =    gcc
DM_FC           =    mpif90
DM_CC           =    mpicc
FC              =    mpif90
FCBASEOPTS      =    $(FCBASEOPTS_NO_G) $(FCDEBUG) -fallow-argument-mismatch -fallow-invalid-boz -g
NETCDFPATH      =    /user-environment/env/default
```

And apply the following “patch”:

```bash
sed -i 's|hdf5hl|hdf5_hl|g' configure.wrf
```

Now compile WRF, which will take a while:

```text
./compile em_real -j 64 &> log_compile
```

The compilation output is captured in `log_compile`. On success, the log should have the message `Executables successfully built`:

```console
$ tail -n14 log_compile

==========================================================================
build started:   Thu 10 Jul 2025 04:54:53 PM CEST
build completed: Thu 10 Jul 2025 05:17:41 PM CEST

--->                  Executables successfully built                  <---

-rwxr-xr-x 1 bcumming csstaff 121952104 Jul 10 17:16 main/ndown.exe
-rwxr-xr-x 1 bcumming csstaff 121728120 Jul 10 17:17 main/real.exe
-rwxr-xr-x 1 bcumming csstaff 120519144 Jul 10 17:17 main/tc.exe
-rwxr-xr-x 1 bcumming csstaff 141159472 Jul 10 17:14 main/wrf.exe

==========================================================================
```

### Step 4: build WPS

> [!NOTE]
>
> **Full script**
>
> Here is the full script, that is described in detail in this step.
>
> ```bash
> uenv start --view-default prgenv-gnu/24.11:v2
>
> # set the paths to match those used in Step 1
> export WRFROOT=$STORE/wrf
> export WRFDEPS=$WRFROOT/dependencies
>
> # set variables used by the WRF build tool
> export NETCDF=/user-environment/env/default
> export HDF5=/user-environment/env/default
> export PNETCDF=$WRFDEPS/view
> export JASPERLIB=$WRFDEPS/view/lib64
> export JASPERINC=$WRFDEPS/view/include
>
> export CC=mpicc
> export FC=mpifort
> export CXX=mpic++
>
> cd $WRFROOT/CRYOWRF/WPS-4.2
> ./configure # choose option 1
>
> # edit configure.wrf to ensure the following variables are set
> vim ./configure.wrf
>
> # SFC                 = gfortran
> # SCC                 = gcc
> # DM_FC               = mpif90
> # DM_CC               = mpicc
> # FC                  = gfortran
> # CC                  = gcc
> # LD                  = $(FC)
> # FFLAGS              = -ffree-form -O -fconvert=big-endian -frecord-marker=4 -fallow-argument-mismatch -fallow-invalid-boz
> # F77FLAGS            = -ffixed-form -O -fconvert=big-endian -frecord-marker=4 -fallow-argument-mismatch -fallow-invalid-boz
>
> ./compile &> log_compile
> ```

Using the same environment as above

```bash
export WRFDEPS=$WRFROOT/dependencies

export NETCDF=/user-environment/env/default
export HDF5=/user-environment/env/default
export PNETCDF=$WRFDEPS/view
export JASPERLIB=$WRFDEPS/view/lib64
export JASPERINC=$WRFDEPS/view/include

cd $WRFROOT/CRYOWRF/WPS-4.2
./configure # choose option 1
```

Update `configure.wps` as follows:

```text
SFC                 = gfortran
SCC                 = gcc
DM_FC               = mpif90
DM_CC               = mpicc
FC                  = gfortran
CC                  = gcc
LD                  = $(FC)
FFLAGS              = -ffree-form -O -fconvert=big-endian -frecord-marker=4 -fallow-argument-mismatch -fallow-invalid-boz
F77FLAGS            = -ffixed-form -O -fconvert=big-endian -frecord-marker=4 -fallow-argument-mismatch -fallow-invalid-boz
```

Note the arguments `-fallow-argument-mismatch -fallow-invalid-boz` added to `FFLAGS` and `F77FLAGS`.

Then compile:

```text
./compile &> log_compile
```

### Running CRYOWRF

Add the following to your SBATCH job script:

```bash
#SBATCH --uenv=prgenv-gnu/24.11:v2
#SBATCH --view=default

# set LD_LIBRARY_PATH to find the dependencies installed in step 1
export WRFROOT=$STORE/wrf
export WRFDEPS=$WRFROOT/dependencies
export LD_LIBRARY_PATH=$WRFDEPS/view/lib:$WRFDEPS/view/lib64:$LD_LIBRARY_PATH

# set WRF variables
export WRF_EM_CORE=1
export WRF_NMM_CORE=0
export WRF_DA_CORE=0

export WRF_CHEM=0
export WRF_KPP=0

export NETCDF4=1
export WRFIO_NCD_LARGE_FILE_SUPPORT=1
export WRFIO_NCD_NO_LARGE_FILE_SUPPORT=0

# set other environment variables

# then run wrf.exe
wrf.exe
```
