Skip to main content

Configuration

Resources​

Usage limits​

You may find that pipeline runs occasionally fail due to a particular step of the pipeline requesting more resources than you have on your system.

To avoid these failures, all nf-core pipelines check pipeline-step resource requests against parameters called --max_cpus, --max_memory and --max_time. These should represent the maximum possible resources of a machine or node.

Most pipelines will attempt to automatically restart jobs that fail due to lack of resources with double-requests, these caps keep those requests from getting out of hand and crashing the entire pipeline run. If a particular job exceeds the process-specific default resources and is retried, only resource requests (cpu, memory, or time) that have not yet reached the value set with --max_<resource> will be increased during the retry.

caution

These parameters only act as a cap, to prevent Nextflow submitting a single job requesting resources more than what is possible on your system.

Tuning​

Like all nextflow pipelines, the resources allocated to each process can be tuned in nextflow .config files. We refer to the nf-core documentation on resource tuning for more information.

DuckDB temporary storage​

When Pixelator runs memory-constrained DuckDB work (for example denoise or component filtering), it can spill temporary data to disk. For container profiles, nf-core/pixelator sets PIXELATOR_DUCKDB_TEMP_DIR to TMPDIR if it is set, otherwise to /tmp, so spill files land on local disk rather than next to the .pxl file (which may live on networked storage such as S3/Fusion).

That default works on most systems. Override it when it does not — for example if /tmp or TMPDIR is too small, is on a networked or otherwise unsuitable filesystem, or is not writable inside the container. You can also set PIXELATOR_DUCKDB_MAX_TEMP_DIR_SIZE to cap how much disk DuckDB may use for spill files (unset means no limit). Values use DuckDB size syntax (for example "10GB").

Set either variable in a Nextflow config passed with -c:

env {
    PIXELATOR_DUCKDB_TEMP_DIR = '/path/to/local/tmp'
    PIXELATOR_DUCKDB_MAX_TEMP_DIR_SIZE = '10GB'
}

These environment variables were introduced in Pixelator 0.22.0.

Institutional configuration​

The nf-core community has created a repository of configuration profiles for many institutions, which can be used to run nf-core pipelines on institutional compute clusters. Using an existing configuration can save you the work to get the pipeline working optimally on your compute infrastructure.

You will find all the configurations and information on how to use them here.

Running in the background​

Nextflow handles job submissions and supervises the running jobs. The Nextflow process must run until the pipeline is finished.

The Nextflow -bg flag launches Nextflow in the background, detached from your terminal so that the workflow does not stop if you log out of your session. The logs are saved to a file.

Alternatively, you can use screen / tmux or similar tool to create a detached session which you can log back into at a later time. Some HPC setups also allow you to run Nextflow within a cluster job submitted your job scheduler (from where it submits more jobs).

Nextflow Memory Requirements​

In some cases, the Nextflow Java virtual machines can start to request a large amount of memory. We recommend adding the following line to your environment to limit this (typically in ~/.bashrc or ~./bash_profile):

NXF_OPTS='-Xms1g -Xmx4g'