Setup & Workflow

Recommended Project Structure

It is recommended to create a separate project folder for each simulation or set of simulations and set it up as a Julia environment. In practice, this means launching Julia with julia --project=path/to/project both in interactive mode and when launching simulations.

It is further recommended to create a Julia script that defines and runs the simulation, as explained below. The same approach can be used for post-processing of results, although it may be more convenient to use a Pluto.jl notebook.

Adding ABL.jl to Project

To set up and run a simulation, the ABL package needs to be in the Julia load path. This is best achieved by adding the package to the project environment.

First, a copy of the ABL.jl repository should be obtained with either

1. a git clone to a single directory that will be shared between projects using the ABL package,
2. a git clone to a subdirectory of the current project,
3. a git clone directly into the directory of the project, using a fork of the ABL.jl repository as basis for the project,
4. git submodule add after setting up a new Git repository for the project.

In all cases except for the third option, the ABL package needs to be added to the project environment. This is achieved by running Julia with --project=path/to/project and adding ABL as a development dependency:

julia> import Pkg
julia> Pkg.develop(path="path/to/ABL.jl")

Any changes that are made to the ABL code are reflected immediately and you do not have to run Pkg.update(), unless you want to update other dependencies.

Interactive Use

Once ABL.jl is part of the project environment, it can be loaded with using ABL as usual. This is only recommended for exploration and to access the built-in documentation. For running simulations and post-processing, it is better to write a script or create a notebook so the work is reproducible.

Notebook Use

Notebooks are mainly useful for code development and post-processing, as they enable rapid, interactive iteration. Jupyter is probably the most well-known software for this, but for Julia code it is worth considering Pluto.jl instead. You can add Pluto to the project dependencies with Pkg.add("Pluto") and launch it with julia --project=path/to/project --eval 'using Pluto; Pluto.run()'. On a remote system, you probably want to run with Pluto.run(launch_browser=false) instead.

By default, Pluto creates a separate, internal project environment for each notebook. When working with ABL.jl simulations, it is usually better to deactivate this functionality and use a single, shared project environment – mainly because Pluto does not have the equivalent of Pkg.develop, but also to ensure that simulations and post-processing use identical package versions. You can use the following code as the first notebook cell:

begin
    LOAD_PATH = ["@", "@stdlib"] # to avoid loading globally-installed packages
    using Pkg; Pkg.activate(".") # disables the built-in package management
    using ABL # add additional dependencies here (can also use `import`)
end

Simulation Scripts

To run simulations, it is recommended to write a script that configures the model and runs the time integration.

An example is given here:

using ABL, MPI

function run_dns()

    outdir = abspath("$(dirname(PROGRAM_FILE))/data")

    ν = 1e-4
    grid_size = (256,192,128)
    u0(x,y,z) = (1 - (z-1)^2)*3/2

    domain = Domain((4π, 2π, 2), SmoothWall(), SmoothWall())
    processes = incompressible_flow(ν, constant_flux = 1)
    abl = DiscretizedABL(grid_size, domain, processes)
    initialize!(abl, vel1 = u0)

    # manually add noise with appropriate intensity
    ABL.State.add_noise!(abl.state.vel1, 1e-2)

    turnovers = 25
    dt = 1e-4
    steps_per_turnover = 5_000
    nt = turnovers * steps_per_turnover
    sfreq = div(steps_per_turnover, 10) * dt
    snapshots = Snapshots(path = joinpath(outdir, "snapshots"),
                          frequency = sfreq, precision = Float32)

    evolve!(abl, dt * nt, dt = dt, output = snapshots, verbose = true)
end

MPI.Init()
run_dns()

If this file is saved as simulation.jl, you can run the simulation as follows:

$ julia --project=path/to/project simulation.jl

High-Performance Computing

TODO