Add Gaudi Functional C++ Class Generator #372

ianna · 2026-01-16T14:24:59Z

BEGINRELEASENOTES

Gaudi Functional C++ Class Generator

ENDRELEASENOTES

Gaudi Functional C++ Class Generator

This script generates Gaudi Functional C++ classes with appropriate structure and boilerplate code based on user-defined specifications. It supports both Gaudi::Functional and k4FWCore frameworks.

Features

Framework support: Both Gaudi and k4FWCore (default: k4FWCore)
4 functional types: Consumer, Producer, Transformer, FilterPredicate
Automatic template generation: Creates proper template signatures based on type
Smart type parsing: Handles complex types like podio::UserDataCollection<float>
Constructor scaffolding: Generates KeyValue/KeyValues initialization automatically
Type-safe operator(): Creates the correct signature and basic implementation template
Gaudi properties: Add configurable properties with descriptions
EDM4hep support: Automatic includes for edm4hep and podio types
Command tracking: Records the generation command in the output file
Consistent conventions: k4FWCore uses struct by default, Gaudi uses class by default
Help system: Run with -h to see all functional types and examples

Input Format

For inputs/outputs: TypeName:LocationName

TypeName: Required (supports templates like podio::UserDataCollection<float>)
LocationName: Optional (defaults to type name without namespace/Collection suffix)

For properties: Type:Name:DefaultValue:Description

Examples

k4FWCore Producer (reproduces ExampleFunctionalProducerMultiple)

python gaudi_gen.py MyProducer producer \
  -o 'podio::UserDataCollection<float>:VectorFloat' \
     'edm4hep::MCParticleCollection:MCParticles1' \
     'edm4hep::MCParticleCollection:MCParticles2' \
     'edm4hep::SimTrackerHitCollection:SimTrackerHits' \
     'edm4hep::TrackerHit3DCollection:TrackerHits' \
     'edm4hep::TrackCollection:Tracks' \
     'edm4hep::ReconstructedParticleCollection:RecoParticles' \
     'edm4hep::RecoMCParticleLinkCollection:Links' \
  -p 'int:ExampleInt:3:Example int that can be used in the algorithm' \
     'int:magicNumberOffset:0:Integer to add to the dummy values'

k4FWCore Consumer

python gaudi_gen.py MyConsumer consumer \
  -i 'edm4hep::MCParticleCollection:MCParticles'

k4FWCore Transformer

python gaudi_gen.py MyTransformer transformer \
  -i 'edm4hep::TrackCollection:InputTracks' \
  -o 'edm4hep::ReconstructedParticleCollection:RecoParticles'

Gaudi Framework Examples

# Simple transformer (like MySum example)
python gaudi_gen.py MySum transformer \
  -i "Input1:Input1Loc" "Input2:Input2Loc" \
  -o "OutputData:OutputLoc" \
  --framework gaudi

# Consumer
python gaudi_gen.py EventMonitor consumer \
  -i "EventData:EventLoc" \
  --framework gaudi

# Filter predicate
python gaudi_gen.py MyFilter filter \
  -i "Track:TrackLoc" \
  -o "bool:FilterResult" \
  --framework gaudi

# With namespace
python gaudi_gen.py MyAlgorithm transformer \
  -i "InputType:InputLoc" \
  -o "OutputType:OutputLoc" \
  -n "MyNamespace" \
  --framework gaudi

Command-Line Options

positional arguments:
  class_name            Name of the C++ class to generate
  functional_type       Type of functional (consumer, producer, transformer, filter)

optional arguments:
  -i, --inputs          Input data specifications
  -o, --outputs         Output data specifications
  -n, --namespace       Namespace for the class
  -f, --output-file     Output file name (default: <ClassName>.cpp)
  --framework           Target framework: gaudi or k4fwcore (default: k4fwcore)
  --class               Generate as class instead of struct (struct is default for k4fwcore)
  -p, --properties      Gaudi properties (format: "Type:Name:Default:Description")
  -h, --help            Show help message

This script generates Gaudi Functional C++ classes with appropriate structure and boilerplate code based on user-defined specifications.

Updated the Gaudi Functional C++ Class Generator to support k4FWCore framework, improved input/output specifications, and added new functionalities for property parsing and class generation.

Added command_line parameter to generate_class function and updated its usage in main.

ianna · 2026-01-16T14:33:00Z

@BrieucF - please, check. Thanks!

jmcarcell · 2026-01-19T08:08:58Z

Without any tests, inevitably this will not work in the future if something changes and we won't be able to know.

ianna · 2026-01-19T08:15:44Z

Without any tests, inevitably this will not work in the future if something changes and we won't be able to know.

Thanks for looking into it. A very good point! Shall I add the generation and compilation for it to CI tests? Thanks

Zehvogel · 2026-01-21T13:07:26Z

@tmadlener weren't you also working on something like this? :)

tmadlener · 2026-01-21T13:12:04Z

Yes, sort of. I have a semi-working thing for a static website. But that also doesn't have the tests it should have. In the end, I don't care too much which version lands as long as one does.

andresailer · 2026-01-21T15:18:33Z

k4FWCore/helpers/gaudi_gen.py

+        class_keyword = "struct" if not use_class else "class"
+        public_keyword = "" if not use_class else "public:\n"
+
+    code = f"""// Generated by Gaudi Functional C++ Class Generator


use textwrap.dedent
https://docs.python.org/3/library/textwrap.html

andresailer · 2026-01-21T15:28:09Z

k4FWCore/helpers/gaudi_gen.py

+                       help='Namespace for the class')
+    parser.add_argument('-f', '--output-file', 
+                       help='Output file name (default: <ClassName>.cpp)')
+    parser.add_argument('--framework', choices=['gaudi', 'k4fwcore'], default='k4fwcore',


Why do we need the gaudi framework option in k4fwcore?

Why do we need the gaudi framework option in k4fwcore?

@andresailer - thanks for reviewing the PR! I thought since the README mentions the Gaudi tutorial it might be useful :-)

tmadlener

Thanks a lot for this. I think the general idea of the script goes into the exact right direction, i.e. we want something that generates boilerplate for something that the user wants. I didn't look in extreme detail, but I see a few general improvements (also partially highlighted in the inline comments):

Currently the user needs to know which functional type they want. But this is entirely specified by the number of inputs and outputs, so I would just determine that from there.
Generally the implementation could probably be improved by sprinkling a few classes into the whole thing for holding intermediate information instead of passing around tuples of strings of various lengths. This would probably also make it easier to simply do some of the parsing / processing up-front and then pass the information around instead of re-parsing it several times.
The current implementation does not handle the possibility of variable length inputs / outputs (I think). This is only possible in k4FWCore Functional algorithms though.

I think this is borderline complex enough to warrant the use of a template engine to handle all the string formatting. It introduces some overhead, and would require writing some templates, but it would simplify the python script parts by quite potentially.

As already mentioned we definitely need tests that ensure that the outputs compile. That would probably be the first thing I do, because once that is in place refactoring and extending the implementation can be done with some guard rails.

tmadlener · 2026-01-21T16:27:37Z

k4FWCore/helpers/gaudi_gen.py

+    return help_text
+
+
+def parse_data_spec(spec: str) -> Tuple[str, str]:


This is called all over the place to do the same things again and again at the top of several functions. Why not call this once at the entry point (generate_class) and pass the outputs of this to all the functions that need it?

tmadlener · 2026-01-21T16:28:40Z

k4FWCore/helpers/gaudi_gen.py

+        if len(out_types) == 0:
+            return "void()"


This case can never happen as this is already ruled out in the parsing of the arguments. Also, I think even if this were to happen, it doesn't compile because a Producer needs at least one output.

tmadlener · 2026-01-21T16:31:48Z

k4FWCore/helpers/gaudi_gen.py

+    if functional_type == 'consumer':
+        in_sig = ', '.join([f"const {t}&" for t in in_types])
+        return f"void({in_sig})"
+    elif functional_type == 'producer':
+        if len(out_types) == 0:
+            return "void()"
+        elif len(out_types) == 1:
+            return f"{out_types[0]}()"
+        else:
+            return f"std::tuple<{', '.join(out_types)}>()"
+    elif functional_type == 'transformer':
+        in_sig = ', '.join([f"const {t}&" for t in in_types]) if in_types else ""
+        if len(out_types) == 1:
+            return f"{out_types[0]}({in_sig})"
+        else:
+            out_sig = ', '.join(out_types)
+            return f"std::tuple<{out_sig}>({in_sig})"
+    elif functional_type == 'filter':
+        in_sig = ', '.join([f"const {t}&" for t in in_types])
+        return f"bool({in_sig})"


This can be simplified. We don't need to differentiate many things based on which functional type we have, as there are a lot of commonalities. The main differentiation points are the number of inputs and the number of outputs (as also visible from the repetition in the code).

tmadlener · 2026-01-21T16:38:56Z

k4FWCore/helpers/gaudi_gen.py

+            # Remove Collection suffix and namespace
+            clean_name = typ.split('::')[-1].replace('Collection', '')
+            loc = clean_name
+        lines.append(f'KeyValues("{loc}", {{"{loc}"}})')


I think from the current implementation there is no way we will ever need the KeyValues, but we always want the KeyValue. Unless the current version already handles inputs/outputs of type std::vector<const XYZCollection*> in which case there should be a differentiation here and only those get a KeyValues (of the appropriate length), single inputs should get a KeyValue.

tmadlener · 2026-01-21T16:40:21Z

k4FWCore/helpers/gaudi_gen.py

+    if functional_type == 'consumer':
+        params = ', '.join([f"const {t}& in{i+1}" for i, t in enumerate(in_types)])
+        return f"void operator()({params}) const override"
+    elif functional_type == 'producer':
+        if len(out_types) == 0:
+            return "void operator()() const override"
+        elif len(out_types) == 1:
+            return f"{out_types[0]} operator()() const override"
+        else:
+            return f"std::tuple<{', '.join(out_types)}> operator()() const override"
+    elif functional_type == 'transformer':
+        params = ', '.join([f"const {t}& in{i+1}" for i, t in enumerate(in_types)])
+        if len(out_types) == 1:
+            return f"{out_types[0]} operator()({params}) const override"
+        else:
+            return f"std::tuple<{', '.join(out_types)}> operator()({params}) const override"
+    elif functional_type == 'filter':
+        params = ', '.join([f"const {t}& in{i+1}" for i, t in enumerate(in_types)])
+        return f"bool operator()({params}) const override"


A similar argument as above applies here. Many things are common and are specialized on the number of inputs / outputs rather than on the type of the algorithm.

tmadlener · 2026-01-21T16:43:06Z

k4FWCore/helpers/gaudi_gen.py

+    if 'edm4hep::' in typ:
+        # Extract all collection types (handle nested templates)
+        # Match patterns like edm4hep::MCParticleCollection
+        pattern = r'edm4hep::(\w+Collection)'
+        matches = re.findall(pattern, typ)


I don't think this differentiation is necessary at all. Algorithms will always take Collections as inputs/outputs. (I think trying to do anything different will not even compile).

tmadlener · 2026-01-21T16:44:42Z

k4FWCore/helpers/gaudi_gen.py

+    elif functional_type == 'transformer':
+        in_sig = ', '.join([f"const {t}&" for t in in_types]) if in_types else ""
+        if len(out_types) == 1:
+            return f"{out_types[0]}({in_sig})"
+        else:
+            out_sig = ', '.join(out_types)
+            return f"std::tuple<{out_sig}>({in_sig})"


I don't think this compiles for multiple outputs. It's either a Transformer (single output) or a MultiTransformer multiple outputs so depending on that we also need to generate a different class here.

ianna added 5 commits January 13, 2026 12:07

Add Gaudi Functional C++ Class Generator script

7c7340c

This script generates Gaudi Functional C++ classes with appropriate structure and boilerplate code based on user-defined specifications.

Enhance gaudi_gen.py for k4FWCore support

0dfad57

Updated the Gaudi Functional C++ Class Generator to support k4FWCore framework, improved input/output specifications, and added new functionalities for property parsing and class generation.

Refactor argument parsing for class generation

49e8064

Add command_line parameter to generate_class function

ddf85c1

Added command_line parameter to generate_class function and updated its usage in main.

Refactor base class handling in gaudi_gen.py

e48e1ac

andresailer reviewed Jan 21, 2026

View reviewed changes

tmadlener reviewed Jan 21, 2026

View reviewed changes

		return help_text


		def parse_data_spec(spec: str) -> Tuple[str, str]:

Add Gaudi Functional C++ Class Generator #372

Are you sure you want to change the base?

Add Gaudi Functional C++ Class Generator #372

Uh oh!

Conversation

ianna commented Jan 16, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Gaudi Functional C++ Class Generator

Features

Input Format

Examples

k4FWCore Producer (reproduces ExampleFunctionalProducerMultiple)

k4FWCore Consumer

k4FWCore Transformer

Gaudi Framework Examples

Command-Line Options

Uh oh!

ianna commented Jan 16, 2026

Uh oh!

jmcarcell commented Jan 19, 2026

Uh oh!

ianna commented Jan 19, 2026

Uh oh!

Zehvogel commented Jan 21, 2026

Uh oh!

tmadlener commented Jan 21, 2026

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tmadlener left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

5 participants

ianna commented Jan 16, 2026 •

edited

Loading