Command-line interface

netcdf-scm

NetCDF-SCM’s command-line interface

netcdf-scm [OPTIONS] COMMAND [ARGS]...

Options

--log-level <log_level>

Options

DEBUG|INFO|WARNING|ERROR|EXCEPTION|CRITICAL

crunch

Crunch data in src to netCDF-SCM .nc files in dst.

src is searched recursively and netcdf-scm will attempt to crunch all the files found. The directory structure in src will be mirrored in dst.

Failures and warnings are recorded and written into a text file in dst. We recommend examining this file using a file analysis tool such as grep. We often use the command grep "\|WARNING\|INFO\|ERROR <log-file>.

crunch_contact is written into the output .nc files’ crunch_contact attribute.

netcdf-scm crunch [OPTIONS] SRC DST CRUNCH_CONTACT

Options

--drs <drs>

Data reference syntax to use for crunching.

Default

Scm

Options

Scm|MarbleCMIP5|CMIP6Input4MIPs|CMIP6Output

--regexp <regexp>

Regular expression to apply to file directory (only crunches matches). Be careful, if you use a very copmlex regexp directory sorting can be extremely slow (see e.g. discussion at https://stackoverflow.com/a/5428712)!

Default

^(?!.*(fx)).*$

--regions <regions>

Comma-separated regions to crunch.

Default

World,World|Northern Hemisphere,World|Southern Hemisphere,World|Land,World|Ocean,World|Northern Hemisphere|Land,World|Southern Hemisphere|Land,World|Northern Hemisphere|Ocean,World|Southern Hemisphere|Ocean

--data-sub-dir <data_sub_dir>

Sub-directory of dst to save data in.

Default

netcdf-scm-crunched

-f, --force, --do-not-force

Overwrite any existing files.

Default

False

--small-number-workers <small_number_workers>

Maximum number of workers to use when crunching files.

Default

10

--small-threshold <small_threshold>

Maximum number of data points (in millions) in a file for it to be processed in parallel with small-number-workers

Default

50.0

--medium-number-workers <medium_number_workers>

Maximum number of workers to use when crunching files.

Default

3

--medium-threshold <medium_threshold>

Maximum number of data points (in millions) in a file for it to be processed in parallel with medium-number-workers

Default

120.0

--force-lazy-threshold <force_lazy_threshold>

Maximum number of data points (in millions) in a file for it to be processed in memory

Default

1000.0

--cell-weights <cell_weights>

How to weight cells when calculating aggregates. If ‘area-surface-fraction’, land surface fraction weights will be included when taking cell means. If ‘area-only’, land surface fraction weights will not be included when taking cell means, hence cells will only be weighted by their area. If nothing is provided, netCDF-SCM will guess whether land surface fraction weights should be included or not based on the data being processed. See netcdf_scm.iris_cube_wrappers.ScmCube.get_scm_timeseries_weights() for more details.

Options

area-only|area-surface-fraction

Arguments

SRC

Required argument

DST

Required argument

CRUNCH_CONTACT

Required argument

stitch

Stitch netCDF-SCM .nc files together and write out in the specified format.

SRC is searched recursively and netcdf-scm will attempt to stitch all the files found. Output is written in DST.

STITCH_CONTACT is written into the header of the output files.

netcdf-scm stitch [OPTIONS] SRC DST STITCH_CONTACT

Options

--regexp <regexp>

Regular expression to apply to file directory (only stitches matches). Be careful, if you use a very copmlex regexp directory sorting can be extremely slow (see e.g. discussion at https://stackoverflow.com/a/5428712)!

Default

^(?!.*(fx)).*$

--prefix <prefix>

Prefix to apply to output file names (not paths).

--out-format <out_format>

Format to re-write crunched data into. The time operation conventions follow those in Pymagicc .

Default

mag-files

Options

mag-files|mag-files-average-year-start-year|mag-files-average-year-mid-year|mag-files-average-year-end-year|mag-files-point-start-year|mag-files-point-mid-year|mag-files-point-end-year|magicc-input-files|magicc-input-files-average-year-start-year|magicc-input-files-average-year-mid-year|magicc-input-files-average-year-end-year|magicc-input-files-point-start-year|magicc-input-files-point-mid-year|magicc-input-files-point-end-year|tuningstrucs-blend-model

--drs <drs>

Data reference syntax to use to decipher paths. This is required to ensure the output folders match the input data reference syntax.

Default

None

Options

None|MarbleCMIP5|CMIP6Input4MIPs|CMIP6Output

-f, --force, --do-not-force

Overwrite any existing files.

Default

False

--number-workers <number_workers>

Number of worker (threads) to use when stitching.

Default

4

--target-units-specs <target_units_specs>

csv containing target units for stitched variables.

--normalise <normalise>

How to normalise the data relative to piControl (if not provided, no normalisation is performed).

Options

31-yr-mean-after-branch-time|21-yr-running-mean|21-yr-running-mean-dedrift|30-yr-running-mean|30-yr-running-mean-dedrift

Arguments

SRC

Required argument

DST

Required argument

STITCH_CONTACT

Required argument

wrangle

Wrangle netCDF-SCM .nc files into other formats and directory structures.

src is searched recursively and netcdf-scm will attempt to wrangle all the files found.

wrangle_contact is written into the header of the output files.

netcdf-scm wrangle [OPTIONS] SRC DST WRANGLE_CONTACT

Options

--regexp <regexp>

Regular expression to apply to file directory (only wrangles matches). Be careful, if you use a very copmlex regexp directory sorting can be extremely slow (see e.g. discussion at https://stackoverflow.com/a/5428712)!

Default

^(?!.*(fx)).*$

--prefix <prefix>

Prefix to apply to output file names (not paths).

--out-format <out_format>

Format to re-write crunched data into. The time operation conventions follow those in Pymagicc.

Default

mag-files

Options

mag-files|mag-files-average-year-start-year|mag-files-average-year-mid-year|mag-files-average-year-end-year|mag-files-point-start-year|mag-files-point-mid-year|mag-files-point-end-year|magicc-input-files|magicc-input-files-average-year-start-year|magicc-input-files-average-year-mid-year|magicc-input-files-average-year-end-year|magicc-input-files-point-start-year|magicc-input-files-point-mid-year|magicc-input-files-point-end-year|tuningstrucs-blend-model

--drs <drs>

Data reference syntax to use to decipher paths. This is required to ensure the output folders match the input data reference syntax.

Default

None

Options

None|MarbleCMIP5|CMIP6Input4MIPs|CMIP6Output

-f, --force, --do-not-force

Overwrite any existing files.

Default

False

--number-workers <number_workers>

Number of worker (threads) to use when wrangling.

Default

4

--target-units-specs <target_units_specs>

csv containing target units for wrangled variables.

Arguments

SRC

Required argument

DST

Required argument

WRANGLE_CONTACT

Required argument