Skip to content

include MiXCR toolkit in nf-core/airrflow#346

Draft
mapo9 wants to merge 40 commits intonf-core:devfrom
mapo9:mixcr
Draft

include MiXCR toolkit in nf-core/airrflow#346
mapo9 wants to merge 40 commits intonf-core:devfrom
mapo9:mixcr

Conversation

@mapo9
Copy link
Collaborator

@mapo9 mapo9 commented Sep 23, 2024

This PR includes MiXCR preprocessing in the nf-core/airrflow pipelines. It allows to integrate MiXCR preprocessing in the the pipeline, finishing with immcantation postprocessing. MiXCR postprocessing can be performed additionnally as well.

Tests and Docs included

PR checklist

  • This comment contains a description of changes (with reason).
  • If you've fixed a bug or added code that should be tested, add tests!
  • If you've added a new tool - have you followed the pipeline conventions in the contribution docs
  • If necessary, also make a PR on the nf-core/airrflow branch on the nf-core/test-datasets repository.
  • Make sure your code lints (nf-core lint).
  • Ensure the test suite passes (nextflow run . -profile test,docker --outdir <OUTDIR>).
  • Check for unexpected warnings in debug mode (nextflow run . -profile debug,test,docker --outdir <OUTDIR>).
  • Usage Documentation in docs/usage.md is updated.
  • Output Documentation in docs/output.md is updated.
  • CHANGELOG.md is updated.
  • README.md is updated (including new tool citations and authors/contributors).

@github-actions
Copy link

github-actions bot commented Sep 23, 2024

nf-core pipelines lint overall result: Passed ✅ ⚠️

Posted for pipeline commit c47ba1c

+| ✅ 308 tests passed       |+
#| ❔   7 tests were ignored |#
!| ❗  11 tests had warnings |!
Details

❗ Test warnings:

  • nextflow_config - Config manifest.version should end in dev: 4.1.0
  • pipeline_todos - TODO string in mixcr_qc_align.nf: List required Conda package(s).
  • pipeline_todos - TODO string in mixcr_qc_align.nf: See section in main README for further information regarding finding and adding container addresses to the section below.
  • pipeline_todos - TODO string in mixcr_qc_coverage.nf: List required Conda package(s).
  • pipeline_todos - TODO string in mixcr_qc_coverage.nf: See section in main README for further information regarding finding and adding container addresses to the section below.
  • pipeline_todos - TODO string in mixcr_qc_chainusage.nf: List required Conda package(s).
  • pipeline_todos - TODO string in mixcr_qc_chainusage.nf: See section in main README for further information regarding finding and adding container addresses to the section below.
  • pipeline_todos - TODO string in mixcr_qc_tags.nf: List required Conda package(s).
  • pipeline_todos - TODO string in mixcr_qc_tags.nf: See section in main README for further information regarding finding and adding container addresses to the section below.
  • pipeline_todos - TODO string in mixcr_exportairr.nf: List required Conda package(s).
  • pipeline_todos - TODO string in mixcr_exportairr.nf: See section in main README for further information regarding finding and adding container addresses to the section below.

❔ Tests ignored:

✅ Tests passed:

Run details

  • nf-core/tools version 3.0.2
  • Run at 2024-11-08 14:18:58

@mapo9
Copy link
Collaborator Author

mapo9 commented Sep 23, 2024

Currently the new mixcr tests fail with
Caused by: Required secret is missing: 'MIXCR_LICENSE'

which is caused by MiXCR requiring a license that is given to the pipeline as a nextflow secret.
I need to figure out how to give the test.config such a secret and ask the MiXCR people if they can provide a license for this cause to make them work.

@mapo9 mapo9 requested a review from ggabernet September 23, 2024 13:46
@nf-core-bot
Copy link
Member

nf-core-bot commented Nov 7, 2024

Warning

Newer version of the nf-core template is available.

Your pipeline is using an old version of the nf-core template: 3.0.2.
Please update your pipeline to the latest version.

For more documentation on how to update your pipeline, please see the nf-core documentation and Synchronisation documentation.

@mapo9
Copy link
Collaborator Author

mapo9 commented Nov 7, 2024

I adapted the metromap and also added the svg
I dont have the airrflow logo anywhere, maybe you can add it?

@jonasscheid
Copy link

Hi!
On behalf of nf-core spring cleaning 2025: Is this issue still relevant/actively worked on? Could @mapo9 get a review on this in the near future?

@mapo9
Copy link
Collaborator Author

mapo9 commented Mar 11, 2025

Hi! On behalf of nf-core spring cleaning 2025: Is this issue still relevant/actively worked on? Could @mapo9 get a review on this in the near future?

very much relevant, yes

Copy link
Member

@ggabernet ggabernet left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @mapo9 thanks for your contributions! I've added some general review comments. The main thing is that as I'm also working on porting modules to nf-core, any new modules should be already part of the nf-core modules if possible. Is it a problem with the mixcr license for the tests?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Here the diagram indicates that fastp is not run on fastq files when using the MiXCR workflow. Could that be edited so that it is also run or shown as run in the diagram?

Comment on lines +4 to +9
include { MIXCR_MIXCR } from '../../modules/local/mixcr/mixcr'
include { MIXCR_MIXCREXPORTAIRR } from '../../modules/local/mixcr/mixcr_exportairr'
include { MIXCR_MIXCRQCALIGN } from '../../modules/local/mixcr/mixcr_qc_align'
include { MIXCR_MIXCRQCCOVERAGE } from '../../modules/local/mixcr/mixcr_qc_coverage'
include { MIXCR_MIXCRQCTAGS } from '../../modules/local/mixcr/mixcr_qc_tags'
include { MIXCR_MIXCRQCCHAINUSAGE } from '../../modules/local/mixcr/mixcr_qc_chainusage'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm trying to port all the airrflow modules to nf-core modules, so new processes should be already part of nf-core modules if possible. Is there a reason why this isn't possible?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is the example for the sention module which also requires a license file so I think it should be possible, check out this PR: https://github.com/nf-core/modules/pull/8187/files

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to have the pipeline running first before taking care of the nf-core modules. But you are right, now that it runs we can add them to nf-core

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

only problem is that the mixcr people allowed my personal license to be used in the testing as well, but didnt want to give us a testing license. So, at one point when my email is not valid anymore the tests will start failing

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

well we can always renew the license when this happens 👍

fi

mixcr analyze ${kit} \\
--library imgt \\
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be possible to allow providing other reference sets? Ideally it should work with the reference data that the user provides as input to the pipeline with the --reference_fasta parameter.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think it is possible by providing the library fasta files with the param above and then building the reference file with mixcr buildLibrary (https://mixcr.com/mixcr/guides/create-custom-library/#de-novo-libraries).

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sounds good, I'll add it

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

just had a look at the tool. It requires individual fastas for each V, (D), J, (C) region. This is quite different from the input we expect from "reference_fasta" right?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

one could provide the full fasta folder and write the paths here to the individual files. Would that work?

mapo9 and others added 4 commits April 25, 2025 14:43
Co-authored-by: Gisela Gabernet <gisela.gabernet@gmail.com>
Co-authored-by: Gisela Gabernet <gisela.gabernet@gmail.com>
Co-authored-by: Gisela Gabernet <gisela.gabernet@gmail.com>
@derrik-gratz
Copy link

See the MixCR module I'm working on here. Addresses some of the license & CI issues, though with less functionality. Would love to expand the module to include some of the additional workflows included here. I did get a 5 year license from the Milaboratories folks to use for CI testing so that helps, then we'd just need the nf-core folks to add the license as a secret. Happy to help with the development of mixcr into nextflow.
nf-core/modules#8632

@ggabernet
Copy link
Member

Hi I converted this PR to draft due to inactivity, let me know whenever you've done some progress and it is ready for review

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants