Logo Questions Linux Laravel Mysql Ubuntu Git Menu
HTML CSS JAVASCRIPT SQL PYTHON PHP BOOTSTRAP JAVA JQUERY R React Kotlin
 

Camel http4 download file using Basic authentication over Https

Tags:

https

apache-camel

basic-authentication

I am trying to download a file from a Https url which requires Basic authentication. I am using HTTP4 I am trying to download from url - https://ebc.cybersource.com/ebc/DownloadReport/xxx.csv?authMethod=Basic&authUsername=scott&authPassword=tiger

After the file is downloaded I need to save it to a folder. Here's what my code looks like

from(xxx)
.to("http4://ebc.cybersource.com/ebc/DownloadReport/xxx.csv?authMethod=Basic&authUsername=scott&authPassword=tiger")
.to("file:target/messages/download");

Here's the error I get - 

java.lang.UnsupportedOperationException: Cannot consume from http endpoint
at org.apache.camel.component.http4.HttpEndpoint.createConsumer(HttpEndpoint.java:120)
at org.apache.camel.impl.EventDrivenConsumerRoute.addServices(EventDrivenConsumerRoute.java:65)
at org.apache.camel.impl.DefaultRoute.onStartingServices(DefaultRoute.java:85)
at org.apache.camel.impl.RouteService.warmUp(RouteService.java:158)
at org.apache.camel.impl.DefaultCamelContext.doWarmUpRoutes(DefaultCamelContext.java:3090)
at org.apache.camel.impl.DefaultCamelContext.safelyStartRouteServices(DefaultCamelContext.java:3020)
at org.apache.camel.impl.DefaultCamelContext.doStartOrResumeRoutes(DefaultCamelContext.java:2797)
at org.apache.camel.impl.DefaultCamelContext.doStartCamel(DefaultCamelContext.java:2653)
at org.apache.camel.impl.DefaultCamelContext.access$000(DefaultCamelContext.java:167)
at org.apache.camel.impl.DefaultCamelContext$2.call(DefaultCamelContext.java:2467)
at org.apache.camel.impl.DefaultCamelContext$2.call(DefaultCamelContext.java:2463)
at org.apache.camel.impl.DefaultCamelContext.doWithDefinedClassLoader(DefaultCamelContext.java:2486)
at org.apache.camel.impl.DefaultCamelContext.doStart(DefaultCamelContext.java:2463)
at org.apache.camel.support.ServiceSupport.start(ServiceSupport.java:61)
at org.apache.camel.impl.DefaultCamelContext.start(DefaultCamelContext.java:2432)
at com.dinesh.MainApp.main(MainApp.java:29)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at com.intellij.rt.execution.application.AppMain.main(AppMain.java:140)

Now the documentation says I need to use HttpContext,

    public class HttpContextFactory {

  private String httpHost = "localhost";
  private String httpPort = 9001;

  private BasicHttpContext httpContext = new BasicHttpContext();
  private BasicAuthCache authCache = new BasicAuthCache();
  private BasicScheme basicAuth = new BasicScheme();

  public HttpContext getObject() {
    authCache.put(new HttpHost(httpHost, httpPort), basicAuth);

    httpContext.setAttribute(ClientContext.AUTH_CACHE, authCache);

    return httpContext;
  }

  // getter and setter
}

I tried that but I am still getting errors.

UPDATE I am actually using a quartz scheduler to trigger

 from("quartz2://foo?cron=0/2+*+*+?+*+MON-FRI").multicast().stopOnException().to("direct:downLoadFile");

        from("direct:downLoadFile")
                .to("http4://ebc.cybersource.com/ebc/DownloadReport/2015/04/16/us_voip/us_voip.au.response.ss.csv?authMethod=Basic&authUserName=xxxx&authPassword=yyyy")
                .log("File Downloaded");

When I run this I get the below error.

[                          main] QuartzComponent                INFO  Starting scheduler.
[                          main] QuartzScheduler                INFO  Scheduler DefaultQuartzScheduler-camel-1_$_NON_CLUSTERED started.
[                          main] DefaultCamelContext            INFO  Total 2 routes, of which 2 is started.
[                          main] DefaultCamelContext            INFO  Apache Camel 2.15.1 (CamelContext: camel-1) started in 0.938 seconds
[artzScheduler-camel-1_Worker-1] RetryExec                      INFO  I/O exception (java.net.SocketException) caught when processing request to {}->http://ebc.cybersource.com:80: Connection reset
[artzScheduler-camel-1_Worker-1] RetryExec                      INFO  Retrying request to {}->http://ebc.cybersource.com:80
[artzScheduler-camel-1_Worker-1] RetryExec                      INFO  I/O exception (java.net.SocketException) caught when processing request to {}->http://ebc.cybersource.com:80: Connection reset
[artzScheduler-camel-1_Worker-1] RetryExec                      INFO  Retrying request to {}->http://ebc.cybersource.com:80
[artzScheduler-camel-1_Worker-1] RetryExec                      INFO  I/O exception (java.net.SocketException) caught when processing request to {}->http://ebc.cybersource.com:80: Connection reset
[artzScheduler-camel-1_Worker-1] RetryExec                      INFO  Retrying request to {}->http://ebc.cybersource.com:80
[artzScheduler-camel-1_Worker-1] DefaultErrorHandler            ERROR Failed delivery for (MessageId: ID-CN-AHONNAVA-LP-49362-1430485742705-0-1 on ExchangeId: ID-CN-AHONNAVA-LP-49362-1430485742705-0-3). Exhausted after delivery attempt: 1 caught: java.net.SocketException: Connection reset

Message History
---------------------------------------------------------------------------------------------------------------------------------------
RouteId              ProcessorId          Processor                                                                        Elapsed (ms)
[route1            ] [route1            ] [quartz2://foo?cron=0%2F2+*+*+%3F+*+MON-FRI                                    ] [       523]
[route1            ] [multicast1        ] [multicast                                                                     ] [       520]
[route1            ] [to1               ] [direct:downLoadFile                                                           ] [       513]
[route2            ] [to2               ] [http4://ebc.cybersource.com/ebc:443/DownloadReport/2015/04/16/us_voip/us_voip.] [       510]

Exchange
---------------------------------------------------------------------------------------------------------------------------------------
Exchange[
    Id                  ID-CN-AHONNAVA-LP-49362-1430485742705-0-3
    ExchangePattern     InOnly
    Headers             {breadcrumbId=ID-CN-AHONNAVA-LP-49362-1430485742705-0-1, calendar=null, CamelRedelivered=false, CamelRedeliveryCounter=0, fireTime=Fri May 01 09:09:04 EDT 2015, jobDetail=JobDetail 'Camel_camel-1.foo':  jobClass: 'org.apache.camel.component.quartz2.CamelJob concurrentExectionDisallowed: false persistJobDataAfterExecution: false isDurable: false requestsRecovers: false, jobInstance=org.apache.camel.component.quartz2.CamelJob@6e61a2e3, jobRunTime=-1, mergedJobDataMap=org.quartz.JobDataMap@2a207ff6, nextFireTime=Fri May 01 09:09:06 EDT 2015, previousFireTime=null, refireCount=0, result=null, scheduledFireTime=Fri May 01 09:09:04 EDT 2015, scheduler=org.quartz.impl.StdScheduler@1e79d438, trigger=Trigger 'Camel_camel-1.foo':  triggerClass: 'org.quartz.impl.triggers.CronTriggerImpl calendar: 'null' misfireInstruction: 1 nextFireTime: Fri May 01 09:09:06 EDT 2015, triggerGroup=Camel_camel-1, triggerName=foo}
    BodyType            null
    Body                [Body is null]
]

Stacktrace
---------------------------------------------------------------------------------------------------------------------------------------
java.net.SocketException: Connection reset
    at java.net.SocketInputStream.read(SocketInputStream.java:196)
    at java.net.SocketInputStream.read(SocketInputStream.java:122)
    at org.apache.http.impl.io.SessionInputBufferImpl.streamRead(SessionInputBufferImpl.java:136)
    at org.apache.http.impl.io.SessionInputBufferImpl.fillBuffer(SessionInputBufferImpl.java:152)
    at org.apache.http.impl.io.SessionInputBufferImpl.readLine(SessionInputBufferImpl.java:270)
    at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:140)
    at org.apache.http.impl.conn.DefaultHttpResponseParser.parseHead(DefaultHttpResponseParser.java:57)
    at org.apache.http.impl.io.AbstractMessageParser.parse(AbstractMessageParser.java:260)
    at org.apache.http.impl.DefaultBHttpClientConnection.receiveResponseHeader(DefaultBHttpClientConnection.java:161)
    at org.apache.http.impl.conn.CPoolProxy.receiveResponseHeader(CPoolProxy.java:153)
    at org.apache.http.protocol.HttpRequestExecutor.doReceiveResponse(HttpRequestExecutor.java:271)
    at org.apache.http.protocol.HttpRequestExecutor.execute(HttpRequestExecutor.java:123)
    at org.apache.http.impl.execchain.MainClientExec.execute(MainClientExec.java:254)
    at org.apache.http.impl.execchain.ProtocolExec.execute(ProtocolExec.java:195)
    at org.apache.http.impl.execchain.RetryExec.execute(RetryExec.java:86)
    at org.apache.http.impl.execchain.RedirectExec.execute(RedirectExec.java:108)
    at org.apache.http.impl.client.InternalHttpClient.doExecute(InternalHttpClient.java:184)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:82)
    at org.apache.http.impl.client.CloseableHttpClient.execute(CloseableHttpClient.java:57)
    at org.apache.camel.component.http4.HttpProducer.executeMethod(HttpProducer.java:258)
    at org.apache.camel.component.http4.HttpProducer.process(HttpProducer.java:158)
    at org.apache.camel.util.AsyncProcessorConverterHelper$ProcessorToAsyncProcessorBridge.process(AsyncProcessorConverterHelper.java:61)
    at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:129)
    at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)
    at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:448)
    at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
    at org.apache.camel.processor.Pipeline.process(Pipeline.java:118)
    at org.apache.camel.processor.Pipeline.process(Pipeline.java:80)
    at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
    at org.apache.camel.component.direct.DirectProducer.process(DirectProducer.java:51)
    at org.apache.camel.processor.SendProcessor.process(SendProcessor.java:129)
    at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)
    at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
    at org.apache.camel.processor.RedeliveryErrorHandler.process(RedeliveryErrorHandler.java:448)
    at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
    at org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:590)
    at org.apache.camel.processor.MulticastProcessor.doProcessSequential(MulticastProcessor.java:518)
    at org.apache.camel.processor.MulticastProcessor.process(MulticastProcessor.java:227)
    at org.apache.camel.management.InstrumentationProcessor.process(InstrumentationProcessor.java:77)
    at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
    at org.apache.camel.processor.CamelInternalProcessor.process(CamelInternalProcessor.java:191)
    at org.apache.camel.processor.loadbalancer.QueueLoadBalancer.process(QueueLoadBalancer.java:44)
    at org.apache.camel.util.AsyncProcessorHelper.process(AsyncProcessorHelper.java:109)
    at org.apache.camel.processor.loadbalancer.LoadBalancerSupport.process(LoadBalancerSupport.java:87)
    at org.apache.camel.component.quartz2.CamelJob.execute(CamelJob.java:56)
    at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
    at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)
[artzScheduler-camel-1_Worker-1] CamelJob                       ERROR Error processing exchange. Exchange[Message: [Body is null]]. Caused by: [org.quartz.JobExecutionException - org.apache.camel.CamelExchangeException: Sequential processing failed for number 0. Exchange[Message: [Body is null]]. Caused by: [java.net.SocketException - Connection reset]]
[artzScheduler-camel-1_Worker-1] JobRunShell                    INFO  Job Camel_camel-1.foo threw a JobExecutionException: 
org.quartz.JobExecutionException: org.apache.camel.CamelExchangeException: Sequential processing failed for number 0. Exchange[Message: [Body is null]]. Caused by: [java.net.SocketException - Connection reset] [See nested exception: org.apache.camel.CamelExchangeException: Sequential processing failed for number 0. Exchange[Message: [Body is null]]. Caused by: [java.net.SocketException - Connection reset]]
    at org.apache.camel.component.quartz2.CamelJob.execute(CamelJob.java:59)
    at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
    at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:573)

FINAL UPDATE: SOLVED Thanks to @Claus Ibsen who provided the correct component. All I had to was use use https4 instead of http. Note this information is missing in camel documentation. Here's the final code, that download the file from the website and saves in in target/download folder.

import org.apache.camel.builder.RouteBuilder;
import org.apache.commons.codec.binary.Base64;

/**
 * Created by darora on 4/30/2015.
 */
public class SchedulerRoute extends RouteBuilder {
    //TODO Move this to properties file
    String userCredentials ="userName:Password";
    String basicAuth = "Basic " + new String(new Base64().encode(userCredentials.getBytes()));
    @Override
    public void configure() throws Exception {
        //from("timer://foo?fixedRate=true&period=6000").to("bean:logBean?method=sayHello");
        //from("scheduler://foo?delay=500").to("bean:logBean?method=sayHello");
        from("quartz2://foo?cron=0/2+*+*+?+*+MON-FRI").multicast().stopOnException().to("direct:downLoadFile","direct:myBean");


        from("direct:myBean").log("Scheduler Started one more time .....")
                .to("bean:downLoadFileBean?method=updatedatabase");
        from("direct:downLoadFile").log("Scheduler Started one more time .....")
            .setHeader("Authorization",constant(basicAuth))
        .to("https4://ebc.cybersource.com/ebc/DownloadReport/2015/04/16/us_voip/us_voip.au.response.ss.csv")
                .to("file:target/download")
                .log("file is downloaded ..........");



    }
}
like image 402
Dinesh Arora Avatar asked May 01 '15 03:05

Dinesh Arora

1 Answers

You cannot start from http4 as the exception tells you, you need a timer or something to trigger the route

from("timer:foo?period=5000")
.to("http4://ebc.cybersource.com/ebc/DownloadReport/xxx.csv?authMethod=Basic&authUsername=scott&authPassword=tiger")
.to("file:target/messages/download");

In this example the timer triggers every 5th second.

like image 61
Claus Ibsen Avatar answered Sep 28 '22 01:09

Claus Ibsen
Sign in to Comment

Related questions

Redirect site url http to https in codeigniter
Block all mixed content
Why does my Java servlet filter not work over HTTPS?
Why does django ignore HTTP_X_FORWARDED_PROTO from the wire but not in tests?
How to capture python https traffic in fiddler?
WKWebView not displaying website (Swift 4, iOS 11, Xcode 9.3)
How to run HTTPS with IIS Express in port number 53135
Spring Boot over HTTPS and SSL certificate on AWS
DotNet not valid certificate found
Can you force an HTML form to use HTTPS for relative URLs?
Git clone over ftps
are cookies secure from hijacking over http
Can a Servlet get HTTPS session id?
Secure GWT application with https
HTTPS node app on Cloud Foundry
Can we access WCF service over https using Jquery $.ajax() call in cross domain?
Browser steps when opening HTTPS page
How to disable “Security Warning” window in Webbrowser control
I'd like to create SSL sertificates for my test environment
HTTPS Proxy Implementation (SSLStream)

Donate For Us

If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!