Skip to content

Bulk MultiQC reporting as a pipeline-installed Python module#450

Draft
keiran-rowell-unsw wants to merge 3 commits intonf-core:devfrom
Australian-Structural-Biology-Computing:multiqc_bulk_report
Draft

Bulk MultiQC reporting as a pipeline-installed Python module#450
keiran-rowell-unsw wants to merge 3 commits intonf-core:devfrom
Australian-Structural-Biology-Computing:multiqc_bulk_report

Conversation

@keiran-rowell-unsw
Copy link

@keiran-rowell-unsw keiran-rowell-unsw commented Jan 28, 2026 •
edited
Loading

Placeholder draft PR for implementing 'bulk' MultiQC reporting as per #439.

Recommendation was deploying as a setup.py module as its pipeline specific and won't work out-of-the-box on underlying tools (see MultiQC PR)

Still testing on a local UNSW HPC and then OoD-proteinfold deployment. Works fine on pre-computed input.

@keiran-rowell-unsw keiran-rowell-unsw changed the title Bulk MultiQC reporting as a module Bulk MultiQC reporting as a pipeline-installed Python module Jan 28, 2026
@github-actions
Copy link

github-actions bot commented Jan 28, 2026 •
edited
Loading

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit 8d3bf30

+| ✅ 327 tests passed       |+
#| ❔   4 tests were ignored |#
#| ❔   1 tests had warnings |#
!| ❗  33 tests had warnings |!
Details

❗ Test warnings:

  • files_exist - File not found: conf/igenomes.config
  • files_exist - File not found: conf/igenomes_ignored.config
  • pipeline_todos - TODO string in base.config: Check the defaults for all processes
  • pipeline_todos - TODO string in base.config: Customise requirements for specific processes.
  • pipeline_todos - TODO string in methods_description_template.yml: #Update the HTML below to your preferred methods description, e.g. add publication citation for this pipeline
  • pipeline_todos - TODO string in main.nf: Optionally add in-text citation tools to this list.
  • pipeline_todos - TODO string in main.nf: Optionally add bibliographic entries to this list.
  • pipeline_todos - TODO string in main.nf: Only uncomment below if logic in toolCitationText/toolBibliographyText has been filled!
  • pipeline_todos - TODO string in usage.md: Add documentation about anything specific to running your pipeline. For general topics, please point to (and add to) the main nf-core website.
  • pipeline_todos - TODO string in nextflow.config: Specify any additional parameters here
  • schema_description - No description provided in schema for parameter: rosettafold2na_uniref30_link
  • schema_description - No description provided in schema for parameter: rosettafold2na_bfd_link
  • schema_description - No description provided in schema for parameter: rosettafold2na_pdb100_link
  • schema_description - No description provided in schema for parameter: rosettafold2na_weights_link
  • schema_description - No description provided in schema for parameter: rfam_full_region_link
  • schema_description - No description provided in schema for parameter: rfam_cm_link
  • schema_description - No description provided in schema for parameter: rnacentral_rfam_annotations_link
  • schema_description - No description provided in schema for parameter: rnacentral_id_mapping_link
  • schema_description - No description provided in schema for parameter: rnacentral_sequences_link
  • schema_description - No description provided in schema for parameter: rosettafold2na_uniref30_path
  • schema_description - No description provided in schema for parameter: rosettafold2na_bfd_path
  • schema_description - No description provided in schema for parameter: rosettafold2na_pdb100_path
  • schema_description - No description provided in schema for parameter: rosettafold2na_weights_path
  • local_component_structure - post_processing.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - prepare_rosettafold_all_atom_dbs.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - prepare_rosettafold2na_dbs.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - prepare_alphafold3_dbs.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - prepare_colabfold_dbs.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - aria2_uncompress.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - prepare_esmfold_dbs.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - prepare_helixfold3_dbs.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - prepare_boltz_dbs.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure
  • local_component_structure - prepare_alphafold2_dbs.nf in subworkflows/local should be moved to a SUBWORKFLOW_NAME/main.nf structure

❔ Tests ignored:

❔ Tests fixed:

✅ Tests passed:

Run details

  • nf-core/tools version 3.5.1
  • Run at 2026-01-28 03:15:25

@keiran-rowell-unsw
Copy link
Author

keiran-rowell-unsw commented Jan 28, 2026

pip installing will break in a container due to unwritable locations.
Quick fix is a venv which is at least portable. Creating a new container for each module would delay release.

Also since it's pipe-line specific I can't post-process with multiqc .
Will need alerting, for each module, lines like this:
ch_multiqc = ch_multiqc.mix(BOLTZ.out.multiqc_report)

Leave this for now because the refactor effort would fit into v2.1 with neater GENERATE_REPORT utils as well

@keiran-rowell-unsw keiran-rowell-unsw mentioned this pull request Jan 28, 2026
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@keiran-rowell-unsw