Command-line interface¶
netcdf-scm¶
NetCDF-SCM’s command-line interface
netcdf-scm [OPTIONS] COMMAND [ARGS]...
Options
-
--log-level
<log_level>
¶ - Options
DEBUG | INFO | WARNING | ERROR | EXCEPTION | CRITICAL
crunch¶
Crunch data in src
to netCDF-SCM .nc
files in dst
.
src
is searched recursively and netcdf-scm will attempt to crunch all the files
found. The directory structure in src
will be mirrored in dst
.
Failures and warnings are recorded and written into a text file in dst
. We
recommend examining this file using a file analysis tool such as grep
. We
often use the command grep "\|WARNING\|INFO\|ERROR <log-file>
.
crunch_contact
is written into the output .nc
files’ crunch_contact
attribute.
netcdf-scm crunch [OPTIONS] SRC DST CRUNCH_CONTACT
Options
-
--drs
<drs>
¶ Data reference syntax to use for crunching.
- Default
Scm
- Options
Scm | MarbleCMIP5 | CMIP6Input4MIPs | CMIP6Output
-
--regexp
<regexp>
¶ Regular expression to apply to file directory (only crunches matches). Be careful, if you use a very copmlex regexp directory sorting can be extremely slow (see e.g. discussion at https://stackoverflow.com/a/5428712)!
- Default
^(?!.*(fx)).*$
-
--regions
<regions>
¶ Comma-separated regions to crunch.
- Default
World,World|Northern Hemisphere,World|Southern Hemisphere,World|Land,World|Ocean,World|Northern Hemisphere|Land,World|Southern Hemisphere|Land,World|Northern Hemisphere|Ocean,World|Southern Hemisphere|Ocean
-
--data-sub-dir
<data_sub_dir>
¶ Sub-directory of
dst
to save data in.- Default
netcdf-scm-crunched
-
-f
,
--force
,
--do-not-force
¶
Overwrite any existing files.
- Default
False
-
--small-number-workers
<small_number_workers>
¶ Maximum number of workers to use when crunching files.
- Default
10
-
--small-threshold
<small_threshold>
¶ Maximum number of data points (in millions) in a file for it to be processed in parallel with
small-number-workers
- Default
50.0
-
--medium-number-workers
<medium_number_workers>
¶ Maximum number of workers to use when crunching files.
- Default
3
-
--medium-threshold
<medium_threshold>
¶ Maximum number of data points (in millions) in a file for it to be processed in parallel with
medium-number-workers
- Default
120.0
-
--force-lazy-threshold
<force_lazy_threshold>
¶ Maximum number of data points (in millions) in a file for it to be processed in memory
- Default
1000.0
-
--cell-weights
<cell_weights>
¶ How to weight cells when calculating aggregates. If ‘area-surface-fraction’, land surface fraction weights will be included when taking cell means. If ‘area-only’, land surface fraction weights will not be included when taking cell means, hence cells will only be weighted by their area. If nothing is provided, netCDF-SCM will guess whether land surface fraction weights should be included or not based on the data being processed. See
netcdf_scm.iris_cube_wrappers.ScmCube.get_scm_timeseries_weights()
for more details.- Options
area-only | area-surface-fraction
Arguments
-
SRC
¶
Required argument
-
DST
¶
Required argument
-
CRUNCH_CONTACT
¶
Required argument
stitch¶
Stitch netCDF-SCM .nc
files together and write out in the specified format.
SRC
is searched recursively and netcdf-scm will attempt to stitch all the
files found. Output is written in DST
.
STITCH_CONTACT
is written into the header of the output files.
netcdf-scm stitch [OPTIONS] SRC DST STITCH_CONTACT
Options
-
--regexp
<regexp>
¶ Regular expression to apply to file directory (only stitches matches). Be careful, if you use a very copmlex regexp directory sorting can be extremely slow (see e.g. discussion at https://stackoverflow.com/a/5428712)!
- Default
^(?!.*(fx)).*$
-
--prefix
<prefix>
¶ Prefix to apply to output file names (not paths).
-
--out-format
<out_format>
¶ Format to re-write crunched data into. The time operation conventions follow those in Pymagicc .
- Default
mag-files
- Options
mag-files | mag-files-average-year-start-year | mag-files-average-year-mid-year | mag-files-average-year-end-year | mag-files-point-start-year | mag-files-point-mid-year | mag-files-point-end-year | magicc-input-files | magicc-input-files-average-year-start-year | magicc-input-files-average-year-mid-year | magicc-input-files-average-year-end-year | magicc-input-files-point-start-year | magicc-input-files-point-mid-year | magicc-input-files-point-end-year | tuningstrucs-blend-model
-
--drs
<drs>
¶ Data reference syntax to use to decipher paths. This is required to ensure the output folders match the input data reference syntax.
- Default
None
- Options
None | MarbleCMIP5 | CMIP6Input4MIPs | CMIP6Output
-
-f
,
--force
,
--do-not-force
¶
Overwrite any existing files.
- Default
False
-
--number-workers
<number_workers>
¶ Number of worker (threads) to use when stitching.
- Default
4
-
--target-units-specs
<target_units_specs>
¶ csv containing target units for stitched variables.
-
--normalise
<normalise>
¶ How to normalise the data relative to piControl (if not provided, no normalisation is performed).
- Options
31-yr-mean-after-branch-time | 21-yr-running-mean | 21-yr-running-mean-dedrift | 30-yr-running-mean | 30-yr-running-mean-dedrift
Arguments
-
SRC
¶
Required argument
-
DST
¶
Required argument
-
STITCH_CONTACT
¶
Required argument
wrangle¶
Wrangle netCDF-SCM .nc
files into other formats and directory structures.
src
is searched recursively and netcdf-scm will attempt to wrangle all the
files found.
wrangle_contact
is written into the header of the output files.
netcdf-scm wrangle [OPTIONS] SRC DST WRANGLE_CONTACT
Options
-
--regexp
<regexp>
¶ Regular expression to apply to file directory (only wrangles matches). Be careful, if you use a very copmlex regexp directory sorting can be extremely slow (see e.g. discussion at https://stackoverflow.com/a/5428712)!
- Default
^(?!.*(fx)).*$
-
--prefix
<prefix>
¶ Prefix to apply to output file names (not paths).
-
--out-format
<out_format>
¶ Format to re-write crunched data into. The time operation conventions follow those in Pymagicc.
- Default
mag-files
- Options
mag-files | mag-files-average-year-start-year | mag-files-average-year-mid-year | mag-files-average-year-end-year | mag-files-point-start-year | mag-files-point-mid-year | mag-files-point-end-year | magicc-input-files | magicc-input-files-average-year-start-year | magicc-input-files-average-year-mid-year | magicc-input-files-average-year-end-year | magicc-input-files-point-start-year | magicc-input-files-point-mid-year | magicc-input-files-point-end-year | tuningstrucs-blend-model
-
--drs
<drs>
¶ Data reference syntax to use to decipher paths. This is required to ensure the output folders match the input data reference syntax.
- Default
None
- Options
None | MarbleCMIP5 | CMIP6Input4MIPs | CMIP6Output
-
-f
,
--force
,
--do-not-force
¶
Overwrite any existing files.
- Default
False
-
--number-workers
<number_workers>
¶ Number of worker (threads) to use when wrangling.
- Default
4
-
--target-units-specs
<target_units_specs>
¶ csv containing target units for wrangled variables.
Arguments
-
SRC
¶
Required argument
-
DST
¶
Required argument
-
WRANGLE_CONTACT
¶
Required argument