This release contains some template changes and bug fixes.
Highlights
We refactored the pipelines template a bit:
- The
libdirectory is removed: 👋 Groovy code - Instead we now use nf-core subworkflows for pipeline initialisation
- Some nf-core pipelines adopted this already: nf-core/fetchngs and nf-core/rnaseq
- The
nf-validationplugin is now used to create an input channel from a sample sheet file.
If you haven’t started merging in the template updates yet, it may almost be easier to ignore the template updates for now and try and remove the lib/ directory on the pipeline dev branch by:
- Individually installing the
utils_*subworkflows with the following commands:
nf-core subworkflows install utils_nextflow_pipeline
nf-core subworkflows install utils_nfcore_pipeline
nf-core subworkflows install utils_nfvalidation_plugin- Creating a local
utils_*subworkflow for the pipeline. You can copy the one in rnaseq (dev) or the pipeline template and customise this to your requirements. Make sure you move any custom functions inlib/directory to this file. - Include the
utils_*subworkflows in the main.nf as done in rnaseq (dev). - Including the
utils_*subworkflows in the workflow main.nf as done in rnaseq (dev). - Delete the
lib/directory after you have confirmed everything has been ported across. - Once you have merged this to
devthe template sync PR will be updated and tell you whether you have missed anything. - The
nf-core lintcommand might complain about having to recompute checksum of subworkflow(s).- Be sure to check in the
modules.jsonfile that non of the previously installed subworkflows have dissapeared from it. - A new
subworkflowssection with the newutils_*subworkflows might have been added during the merged. Reinstalling the subworkflows usingnf-core subworkflows installshould fix this, otherwise one could manually edit the modules.json file (only recommended for advanced users).
- Be sure to check in the
💡 It helped to disable running the main workflow whilst wiring all of this in to speed up development.
🎥 Last but not least, we have a short bytesize talk showing the necessary steps:
Additional bug fixes and improvements
-
Have you seen a
no space left on deviceerror or your CI tests lately? This is now fixed by adding a clean up step in the github actions. -
New api-docs on the nf-core website to get clarifications on linting errors etc.
-
Updating a module won’t remove custom nextflow.config files anymore.
-
The commands
nf-core modules testandnf-core subworkflows testnow have a new parameter--profileto specify the container to run nf-test with.
Thanks to all the contributors who made this release possible, especially to Harshil Patel and his team for kicking off the lib restructuring and the utils_* subworkflows.
You can find the whole changelog on GitHub.