This project defines a Nifi custom processor which integrates Apache Nifi v2 with Apache Camel v4.
In order to work correctly, the user must specify a valid Camel route in YAML or XML format which:
- Reads from "direct:in" endpoint; this is a static camel input endpoint which cannot be changed.
- Redirect valid flow files to one or more "direct:$name" endpoint(s); $name is a dynamic output endpoint which can be changed from the user.
- Writes non-valid flow files to "direct:err" endpoint; this is a static camel output endpoint which cannot be changed.
In "direct:in" endpoint the input flow file content and attributes will be written in the camel message body and headers, respectively. This endpoint should be used in the first camel "from" clause.
The valid flow files redirected to "direct:$name" relationships will be routed to dynamically created nifi relationship with the new content/headers set in the camel route, if changed (flow file attributes can be changed through the camel message headers and flow file content through the camel message body).
The processor can optionally ignore the input flow file content, passing an empty body to Camel while preserving headers, or ignore the Camel output body, preserving the original flow file content while still applying output headers.
If a flow file is routed to the "direct:err" relationship, internally the nifi processor will throw an exception and will write the original flow file in the "failure" relationship.
WARNING: Flow files routed to "direct:$name" relationships whose body is not converted to an instance of "java.io.InputStream" (this can be done through the Camel "convertBodyTo" step) will be routed to "failure" relationship, unless the Camel output body is ignored.
The processor defines the following parameters:
- Camel Route Type: (1) Yaml: allows to specify route in yaml format; (2) Xml: allows to specify route in xml format
- Camel Route: User supplied route in the specified DSL
- Ignore Input Body: allows to avoid passing the flow file content to the Camel route
- Ignore Output Body: allows to preserve the original flow file content and avoid writing the Camel output body to a new flow file
The flow files will inherit the following attributes:
- camel.error.message: contains the error message (only for flow files routed to "failure" relationship)
The NAR is published on Maven Central as it.agilelab:nifi-camel-nar:
<dependency>
<groupId>it.agilelab</groupId>
<artifactId>nifi-camel-nar</artifactId>
<version>1.0.0</version>
<type>nar</type>
</dependency>Or download the NAR and drop it into your NiFi extensions/ directory (see the NiFi processor locations docs).
You can find a list of camel route examples in the following directory: nifi-camel-processors/src/test/resources.
Moreover, this processor comes with a set of camel components, camel languages and camel data formats already installed, which can be used in routes:
At the moment is not possible to dynamically add Apache Camel components.
Routes using Jackson-backed Camel features, such as camel-jsonpath or camel-jackson, use the default Jackson stream constraints. As a result, JSON documents containing a single string value larger than Jackson's default maximum string length can fail during route execution. This limit is not raised globally by the processor.
- Java 21
- Nifi 2.3.0
This project was developed at AgileLab by Alberto Puritano and Luigi Baldari.
For instructions about how to compile, test and package, refer to the ci/cd file (.gitlab-ci.yml).
For instructions about how to import the nifi custom processor in your nifi instance, refer to the following documentation.