Rampart PostDispatchVerificationHandler: InvalidSecurity - Security policy not found

Question

Edit:

I tweaked the Rampart configuration a little and I am now stuck at another point.

In Rampart's PostDispatchVerificationHandler an exception is thrown, because the security header hasn't been processed.

// If a security header is there and Rampart is engaged, it has to be processed.  
// If it is not processed, there must have been a problem in picking the policy 

SOAPHeaderBlock secHeader = getSecurityHeader(msgContext);
if (secHeader != null && (secHeader.isProcessed() == false)) {
     throw new AxisFault("InvalidSecurity - Security policy not found");
}

Log:

[DEBUG] [MessageContext: logID=a5012f2f13095af97123a192575c50a7f727850f3a9ecfc5] Invoking Handler 'HTTPLocationBasedDispatcher' in Phase 'Dispatch'
[DEBUG] [MessageContext: logID=a5012f2f13095af97123a192575c50a7f727850f3a9ecfc5] Invoking Handler 'Post dispatch security verification handler' in Phase 'Dispatch'
[ERROR] InvalidSecurity - Security policy not found
org.apache.axis2.AxisFault: InvalidSecurity - Security policy not found
    at org.apache.rampart.handler.PostDispatchVerificationHandler.invoke(PostDispatchVerificationHandler.java:189)
    at org.apache.axis2.engine.Phase.invokeHandler(Phase.java:340)
    at org.apache.axis2.engine.Phase.invoke(Phase.java:313)
    at org.apache.axis2.engine.AxisEngine.invoke(AxisEngine.java:262)
    at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:168)
    at org.apache.axis2.description.OutInAxisOperationClient.handleResponse(OutInAxisOperation.java:364)
    at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:421)
    at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:229)
    at org.apache.axis2.client.OperationClient.execute(OperationClient.java:165)
    at webservices.chargepoint.com.dictionary.ChargepointservicesStub.getCPNInstances(ChargepointservicesStub.java:5361)
    at webservices.chargepoint.com.dictionary.Chargepoint.<init>(Chargepoint.java:180)
    at webservices.chargepoint.com.dictionary.Chargepoint.main(Chargepoint.java:81)
[DEBUG] [MessageContext: logID=a5012f2f13095af97123a192575c50a7f727850f3a9ecfc5] Invoking flowComplete() in Phase "Dispatch"

...

org.apache.axis2.AxisFault: InvalidSecurity - Security policy not found
    at org.apache.rampart.handler.PostDispatchVerificationHandler.invoke(PostDispatchVerificationHandler.java:189)
    at org.apache.axis2.engine.Phase.invokeHandler(Phase.java:340)
    at org.apache.axis2.engine.Phase.invoke(Phase.java:313)
    at org.apache.axis2.engine.AxisEngine.invoke(AxisEngine.java:262)
    at org.apache.axis2.engine.AxisEngine.receive(AxisEngine.java:168)
    at org.apache.axis2.description.OutInAxisOperationClient.handleResponse(OutInAxisOperation.java:364)
    at org.apache.axis2.description.OutInAxisOperationClient.send(OutInAxisOperation.java:421)
    at org.apache.axis2.description.OutInAxisOperationClient.executeImpl(OutInAxisOperation.java:229)
    at org.apache.axis2.client.OperationClient.execute(OperationClient.java:165)
    at webservices.chargepoint.com.dictionary.ChargepointservicesStub.getCPNInstances(ChargepointservicesStub.java:5361)
    at webservices.chargepoint.com.dictionary.Chargepoint.<init>(Chargepoint.java:180)
    at webservices.chargepoint.com.dictionary.Chargepoint.main(Chargepoint.java:81)

That is the security header:

<wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/" soapenv:mustUnderstand="1">
<wsu:Timestamp xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
<wsu:Created>2013-12-16T23:07:03.868Z</wsu:Created>
<wsu:Expires>2013-12-16T23:12:03.868Z</wsu:Expires>
</wsu:Timestamp>
</wsse:Security>

That is Rampart's policy file:

<wsp:Policy wsu:Id="UsernameToken" xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd" xmlns:wsp="http://schemas.xmlsoap.org/ws/2004/09/policy">
  <wsp:ExactlyOne>
    <wsp:All>
      <sp:SupportingTokens xmlns:sp="http://schemas.xmlsoap.org/ws/2005/07/securitypolicy">
        <wsp:Policy>
          <sp:UsernameToken sp:IncludeToken="http://schemas.xmlsoap.org/ws/2005/07/securitypolicy/IncludeToken/AlwaysToRecipient"/>
        </wsp:Policy>
      </sp:SupportingTokens>
    </wsp:All>
  </wsp:ExactlyOne>
</wsp:Policy>

The user credentials are set in the Java code:

Options options = stubWeb._getServiceClient().getOptions();
try
{
    options.setProperty(RampartMessageData.KEY_RAMPART_POLICY,
                        loadPolicy("policy.xml"));
}
catch (XMLStreamException e1)
{
    // TODO Auto-generated catch block
    e1.printStackTrace();
}
options.setUserName("xxx");
options.setPassword("yyy");

stubWeb._getServiceClient().setOptions(options);
stubWeb._getServiceClient().engageModule("rampart");

Original Post:

I've been googling this error for days but I am out of ideas.

The code is generated by Axis2 for a SOAP webservice based on this WSDL: https://webservices.chargepoint.com/cp_api_4.1.wsdl

With soapUI everything works well and I can also get some mock responses with my code from my local Tomcat server. When trying to connect to the web service with my Java client I always receive this error:

org.apache.axis2.AxisFault: Must Understand check failed for header http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd : Security

That's my code (username and password are hidden):

public static void main(String[] args)
{
    log4j.getRootLogger().setLevel(Level.DEBUG);
    GetCPNInstancesResponse resp = new GetCPNInstancesResponse();

    ChargepointservicesStub stubWeb = null;
    try
    {
        ConfigurationContext ctx = ConfigurationContextFactory.createConfigurationContextFromFileSystem("/Users/jonas/projects/workspace_openHAB/axis2-1.6.2_clean/repository", null);   
        stubWeb = new ChargepointservicesStub(ctx);    
        GetCPNInstances cpn = new GetCPNInstances();        
        SOAPFactory sfac = OMAbstractFactory.getSOAP11Factory(); 
        stubWeb._getServiceClient().engageModule("rampart");

        OMFactory omFactory = OMAbstractFactory.getOMFactory();
        OMNamespace wsseNamespace = omFactory.createOMNamespace("http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd", "wsse");
        OMElement omSecurityElement = omFactory.createOMElement("Security", wsseNamespace);
//          omSecurityElement.addAttribute("soapenv:mustUnderstand", "1", null);
//          OMNamespace soapenv = omFactory.createOMNamespace("http://schemas.xmlsoap.org/soap/envelope/", "soapenv");
//          omSecurityElement.addAttribute("mustUnderstand", "1", soapenv);

        OMElement omusertoken = omFactory.createOMElement("UsernameToken", wsseNamespace);
        OMElement omuserName = omFactory.createOMElement("Username", wsseNamespace);
        omuserName.setText("yyy");
        OMElement omPassword = omFactory.createOMElement("Password", wsseNamespace);
        omPassword.setText("xxx");
        omPassword.addAttribute("Type", "http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText", null);

        omusertoken.addChild(omuserName);
        omusertoken.addChild(omPassword);
        omSecurityElement.addChild(omusertoken);

        SOAPHeaderBlock block = ElementHelper.toSOAPHeaderBlock(omSecurityElement, sfac);
        block.setMustUnderstand(true);
        stubWeb._getServiceClient().addHeader(block);

        resp = stubWeb.getCPNInstances(cpn);
    } catch (Exception e)
    {
        // TODO Auto-generated catch block
        e.printStackTrace();
    }
}

That generates this request:

[DEBUG] >> "<?xml version='1.0' encoding='UTF-8'?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
    <soapenv:Header>
    <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" soapenv:mustUnderstand="1">
        <wsse:UsernameToken>
        <wsse:Username>yyy</wsse:Username>
        <wsse:Password Type="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-username-token-profile-1.0#PasswordText">xxx</wsse:Password>
        </wsse:UsernameToken>
    </wsse:Security>
    </soapenv:Header>
    <soapenv:Body>
    <ns1:getCPNInstances xmlns:ns1="urn:dictionary:com.chargepoint.webservices" />
    </soapenv:Body>
</soapenv:Envelope>"

The exact same request works with soapUI.

I also receive a correct response by the server with my code:

<?xml version='1.0' encoding='utf-8'?>
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/">
    <soapenv:Header>
        <wsse:Security xmlns:wsse="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd" soapenv:mustUnderstand="1"><wsu:Timestamp xmlns:wsu="http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-utility-1.0.xsd">
            <wsu:Created>2013-10-25T16:54:30.767Z</wsu:Created>
            <wsu:Expires>2013-10-25T16:59:30.767Z</wsu:Expires></wsu:Timestamp>
        </wsse:Security>
    </soapenv:Header>
    <soapenv:Body>
        <ns1:getCPNInstancesResponse xmlns:ns1="urn:dictionary:com.chargepoint.webservices">
            <CPN>
                <cpnID>1</cpnID>
                <cpnName>USA</cpnName>
                <cpnDescription>ChargePoint Operations</cpnDescription>
            </CPN>
            <CPN>
                <cpnID>2</cpnID>
                <cpnName>EU</cpnName>
                <cpnDescription>ChargePoint Europe</cpnDescription>
            </CPN>
            <CPN>
                <cpnID>3</cpnID>
                <cpnName>AU</cpnName>
                <cpnDescription>ChargePoint Australia</cpnDescription>
            </CPN>
        </ns1:getCPNInstancesResponse>
    </soapenv:Body>
</soapenv:Envelope>

But then the security module messes something up.

This is part of the debug log:

[DEBUG] XMLStreamWriter is org.apache.axiom.util.stax.dialect.WoodstoxStreamWriterWrapper 
[DEBUG] Calling MTOMXMLStreamWriter.flush 
[DEBUG] forceExpand: expanding element {http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd}Security 
[DEBUG] forceExpand stack java.lang.Exception: Debug Stack Trace    
at org.apache.axiom.om.impl.llom.OMSourcedElementImpl.forceExpand(OMSourcedElementImpl.java:264)    
at org.apache.axiom.om.impl.llom.OMSourcedElementImpl.getAttribute(OMSourcedElementImpl.java:416)   
at org.apache.axiom.soap.impl.llom.SOAPHeaderBlockImpl.setAttribute(SOAPHeaderBlockImpl.java:91)    
at org.apache.axiom.soap.impl.llom.soap11.SOAP11HeaderBlockImpl.setMustUnderstand(SOAP11HeaderBlockImpl.java:105)   
at chargepoint.com.dictionary.Chargepoint.main(Chargepoint.java:74)

When I use

omSecurityElement.addAttribute("soapenv:mustUnderstand", "1", null);

instead of

block.setMustUnderstand(true);

The exception above is gone and I get this error:

[DEBUG] serialize OutputStream optimisation: false [DEBUG] getReader [DEBUG] 
XMLStreamReader is org.apache.axiom.util.stax.dialect.WoodstoxStreamReaderWrapper 
org.apache.axiom.om.OMException: com.ctc.wstx.exc.WstxParsingException: Undeclared namespace prefix "soapenv" (for attribute "mustUnderstand")  at [row,col {unknown-source}]: [1,137] 
at org.apache.axiom.om.impl.builder.StAXOMBuilder.next(StAXOMBuilder.java:296)  
at org.apache.axiom.om.impl.llom.OMDocumentImpl.buildNext(OMDocumentImpl.java:153)  
at org.apache.axiom.om.impl.llom.OMDocumentImpl.getFirstOMChild(OMDocumentImpl.java:242) 
at org.apache.axiom.om.impl.llom.OMDocumentImpl.getChildren(OMDocumentImpl.java:204)

With this code the error is gone but the issue remains.

OMNamespace soapenv = omFactory.createOMNamespace("http://schemas.xmlsoap.org/soap/envelope/", "soapenv");
omSecurityElement.addAttribute("mustUnderstand", "1", soapenv);

That are the most interesting parts from the debug log:

[DEBUG] [MessageContext: logID=ca3f0db9a4fb557a33edc579a48f31508ff1f2b457c617f6] Invoking phase "Security"
[DEBUG] [MessageContext: logID=ca3f0db9a4fb557a33edc579a48f31508ff1f2b457c617f6] Invoking Handler 'SecurityOutHandler' in Phase 'Security'
[DEBUG] WSDoAllReceiver: enter invoke() 
[DEBUG] Signature crypto property file is not set. Property file key - signaturePropFile
[DEBUG] Signature crypto property file is not set. Property file key - signaturePropFile
[DEBUG] WSDoAllReceiver: exit invoke()

...

[DEBUG] [MessageContext: logID=ba3f0db9a4fb557a33edc579a48f31508ff1f2b457c617f6] Checking post-conditions for phase "OperationInPhase"
[DEBUG] MustUnderstand header not processed or registered as understood{http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd}Security
[DEBUG] org.apache.axis2.i18n.resource::handleGetObject(mustunderstandfailed)
[ERROR] Must Understand check failed for header http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd : Security
org.apache.axis2.AxisFault: Must Understand check failed for header http://docs.oasis-open.org/wss/2004/01/oasis-200401-wss-wssecurity-secext-1.0.xsd : Security
    at org.apache.axis2.engine.AxisEngine.checkMustUnderstand(AxisEngine.java:105)

Here you can see the whole debug log: https://drive.google.com/file/d/0B_iw7qzMyYhOSDNEMW5oOHgzY0k/edit?usp=sharing

If you have any advice how to solve that problem I would be very grateful.

Answer 1

I confess I am not familiar with these technologies, but I found the Javadoc for SOAPHeaderBlock.setProcessed interesting:

"We need to know whether all the mustUnderstand headers have been processed by the node. This will done by a specific validation handler at the end of the execution chain. For this all the handlers who process a particular header block must explicitly say that he processesd [sic] the header by calling setProcessed()"

So maybe something like this has to happen?

SOAPHeaderBlock block = ElementHelper.toSOAPHeaderBlock(omSecurityElement, sfac);
block.setProcessed();

Another possibility can be found here, where the poster changes the contents of the Rampart configuration file META-INF/module.xml by adding this:

<InFaultFlow>
    <handler name="PolicyBasedSecurityInHandler" class="org.apache.rampart.handler.RampartReceiver">
        <order phase="Security" phaseFirst="true"/>
    </handler>
    <handler name="SecurityInHandler" class="org.apache.rampart.handler.WSDoAllReceiver">
        <order phase="Security"/>
    </handler>
    <handler name="PostDispatchVerificationHandler" class="org.apache.rampart.handler.PostDispatchVerificationHandler">
        <order phase="Dispatch" phaseLast="true"/>
    </handler>
</InFaultFlow> 

I find this less likely to be the issue, but your problem seems so frustrating that I figured I would throw it out there.

Good luck.

Answer 2

This may not be a solution. I had same kind of situation where i used a stub generated by wsdl2java tool. the security headers were not included when i send the requests. So i created the request using ServiceClient class and then import the policy. It looks thing like this

ConfigurationContext ctx = ConfigurationContextFactory.createConfigurationContextFromFileSystem("clientrepo", null);

ServiceClient sc = new ServiceClient(ctx, null);
sc.engageModule("rampart");   

        // create option object
Options opts = new Options();
        // setting target EPR
opts.setTo(new EndpointReference(serviceUrl);

opts.setAction("urn:something");

try {
    opts.setProperty(RampartMessageData.KEY_RAMPART_POLICY,
                          loadPolicy("policy.xml"));
    } catch (Exception e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
    }
sc.setOptions(opts);
OMElement res = sc.sendReceive(payload);

And to load the policy

 public Policy loadPolicy(String xmlPath) throws Exception {

    StAXOMBuilder builder = new StAXOMBuilder(xmlPath);
    Policy policy = PolicyEngine.getPolicy(builder.getDocumentElement());

    RampartConfig rc = new RampartConfig();
    rc.setUser(username);
    rc.setPwCbClass(PWDCallBackHandler.class.getName());

    CryptoConfig sigCryptoConfig = new CryptoConfig();
    sigCryptoConfig.setProvider("org.apache.ws.security.components.crypto.Merlin");

    Properties prop1 = new Properties();
    prop1.put("org.apache.ws.security.crypto.merlin.keystore.type", "JKS");
    prop1.put("org.apache.ws.security.crypto.merlin.file", AppConstants.KEYSTORE);
    prop1.put("org.apache.ws.security.crypto.merlin.keystore.password",
              password);
    sigCryptoConfig.setProp(prop1);

    rc.setSigCryptoConfig(sigCryptoConfig);
    policy.addAssertion(rc);
    return policy;
}