Logo Questions Linux Laravel Mysql Ubuntu Git Menu
 

Error during the creating of custom processors apache nifi

I tried to build my package with NiFi custom-processor using mvn clean package command and I got the following output:

[INFO] ------------------------------------------------------------------------
[INFO] BUILD FAILURE
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  3.794 s
[INFO] Finished at: 2021-05-17T14:33:42+02:00
[INFO] ------------------------------------------------------------------------
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:testCompile 
(groovy-tests) on project mycustom-processor: Execution groovy-tests of goal
 org.apache.maven.plugins:maven-compiler-plugin:3.8.1:testCompile failed: 
Plugin org.apache.maven.plugins:maven-compiler-plugin:3.8.1 or one of its dependencies
 could not be resolved: Failed to collect dependencies at 
org.apache.maven.plugins:maven-compiler-plugin:jar:3.8.1 
-> org.codehaus.groovy:groovy-eclipse-batch:jar:2.5.4-01: Failed to read artifact descriptor for 
org.codehaus.groovy:groovy-eclipse-batch:jar:2.5.4-01: Could not transfer artifact
 org.codehaus.groovy:groovy-eclipse-batch:pom:2.5.4-01 from/to bintray 
(https://dl.bintray.com/groovy/maven): 
Access denied to: https://dl.bintray.com/groovy/maven/org/codehaus/groovy/groovy-eclipse-batch/2.5.4-01/groovy-eclipse-batch-2.5.4-01.pom, 
ReasonPhrase: Forbidden. -> [Help 1]
[ERROR] 

Does anybody know how to handle this issue related to the maven-compiler-plugin:3.8.1:testCompile, please ? I use Apache NiFi version 1.13.2 and mvn install -DskipTests does not change anything.

like image 847
Rafayel Avatar asked May 17 '21 12:05

Rafayel


People also ask

What is FlowFile in NiFi?

FlowFiles are at the heart of NiFi and its flow-based design. A FlowFile is a data record, which consists of a pointer to its content (payload) and attributes to support the content, that is associated with one or more provenance events.

What is Output Port in NiFi?

Output port only transfers the data from a process group to Outside processors(or) processor groups. Can you add an Processor Group and use these two processors inside Processor Group then you can enable transmission from Output port. Flow:- Inside the processor group keep all your processors.

What is a processor in NiFi?

Processor: The Processor is the NiFi component that is used to listen for incoming data; pull data from external sources; publish data to external sources; and route, transform, or extract information from FlowFiles. Relationship: Each Processor has zero or more Relationships defined for it.

What is Apache NiFi processor?

Apache NiFi - Processors. Apache NiFi processors are the basic blocks of creating a data flow. Every processor has different functionality, which contributes to the creation of output flowfile. Dataflow shown in the image below is fetching file from one directory using GetFile processor and storing it in another directory using PutFile processor.

How do I write a custom processor in NiFi?

The easiest way to get started writing a custom processor is to use the Maven archetype included with NiFI. From a directory where you want to create your processor bundle, execute the following:

How to install Maven on Apache NiFi?

Apache NiFi is an open source platform and gives developers the options to add their custom processor in the NiFi library. Follow these steps to create a custom processor. Download Maven latest version from the link given below. Add an environment variable named M2_HOME and set value as the installation directory of maven.

What kind of Apache NiFi services does HashMap offer?

At Hashmap, our engineering and consulting teams have developed a wide range of custom Apache NiFi processors and controller services for a variety of clients and business requirements — some generic and some very industry specific. A NiFi Processor is the basic building block for creating an Apache NiFi dataflow.


2 Answers

Add the below property to your pom.xml

    <nifi.groovy.version>2.5.6</nifi.groovy.version>
like image 178
Sateesh Telaprolu Avatar answered Oct 28 '22 12:10

Sateesh Telaprolu


The reason for this error is that the pom's url is not accessable.

Currently, groovy-eclipse-batch-2.5.4-01.pom link is forbidden.(dl.bintray.com/groovy/maven/org/codehaus/groovy/groovy-eclipse-batch/2.5.4-01/groovy-eclipse-batch-2.5.4-01.pom)

Check if groovy-eclipse-batch exists in your maven repository. Or just find a valid version on mvnrepository.com and use it. (https://mvnrepository.com/artifact/org.codehaus.groovy/groovy-eclipse-batch)

For example, change version of groovy-eclipse-batch to 2.5.6-01.

Try adding the following to your parent pom for the bundle project :

<build>
    <plugins>
        <plugin>
            <artifactId>maven-compiler-plugin</artifactId>
            <version>3.8.1</version>            
            <configuration>
                <source>1.8</source>
                <target>1.8</target>
                <encoding>UTF-8</encoding>
            </configuration>                        
            <dependencies>
                <dependency>
                    <groupId>org.codehaus.groovy</groupId>
                    <artifactId>groovy-eclipse-batch</artifactId>
                    <version>2.5.6-01</version>
                </dependency>
            </dependencies>                 
        </plugin>
    </plugins>
</build>

Before add the dependency :

[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:testCompile (groovy-tests) on project mytest: Execution groovy-tests of goal org.apache.maven.plugins:maven-compiler-plugin:3.8.1:testCompile failed: Plugin org.apache.maven.plugins:maven-compiler-plugin:3.8.1 or one of its dependencies could not be resolved: Failed to collect dependencies at org.apache.maven.plugins:maven-compiler-plugin:jar:3.8.1 -> org.codehaus.groovy:groovy-eclipse-batch:jar:2.5.4-01: Failed to read artifact descriptor for org.codehaus.groovy:groovy-eclipse-batch:jar:2.5.4-01: Could not transfer artifact org.codehaus.groovy:groovy-eclipse-batch:pom:2.5.4-01 from/to bintray (https://dl.bintray.com/groovy/maven): Access denied to: https://dl.bintray.com/groovy/maven/org/codehaus/groovy/groovy-eclipse-batch/2.5.4-01/groovy-eclipse-batch-2.5.4-01.pom -> [Help 1]

After add the dependency :

[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for mytest 1.0.0-SNAPSHOT:
[INFO]
[INFO] mytest ............................................. SUCCESS [  3.611 s]
[INFO] nifi-mytestnar-processors .......................... SUCCESS [ 11.278 s]
[INFO] nifi-mytestnar-nar ................................. SUCCESS [  0.983 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  19.448 s
like image 42
Lignaseel Avatar answered Oct 28 '22 12:10

Lignaseel