[SOLVED] root namespace is added into the <xades:SignedProperties>

library: apache Santuario + xades4j.

using xpathto select elements and sign them.

If I try to sign a simple XML without a namespace and verify the signature, it works well , but if the XML defines a namespace, for example the XML following:

<ClinicalDocument xmlns="urn:hl7-org:v3">
    <element1tobesigned.../>
    <element2tobesigned.../>
</ClinicalDocument>

and the exception was found when verifying the signature


    858  WARN  [main] org.apache.xml.security.signature.Reference     - Verification failed for URI "#xmldsig-5fb20abe-b14c-4d84-a908-e22e776cd6f1-signedprops"
    858  WARN  [main] org.apache.xml.security.signature.Reference     - Expected Digest: q0WnWFf9j0kcT46t5cXmcPnVvu5o51oAcmej/SjCazQ=
    858  WARN  [main] org.apache.xml.security.signature.Reference     - Actual Digest: 41zXKVkRCsxUYpNZXW5b9KkZlTC9LM9WA8O7WHQz1Rg=

    xades4j.verification.ReferenceValueException: Reference '#xmldsig-5fb20abe-b14c-4d84-a908-e22e776cd6f1-signedprops' cannot be validated

the cause is that XML namespace (urn:hl7-org:v3) was added into the xades:SignedProperties then the digest became different .

858  DEBUG [main] org.apache.xml.security.utils.DigesterOutputStream     - Pre-digested input
858  DEBUG [main] org.apache.xml.security.utils.DigesterOutputStream   - <xades:SignedProperties xmlns="urn:hl7-org:v3" ........./>

here is the signature generation code


    XadesTSigningProfile profile = new XadesTSigningProfile(keyProvider);
    profile.withTimeStampTokenProvider(TestTimeStampTokenProvider.class)
    .withAlgorithmsProviderEx(ExclusiveC14nForTimeStampsAlgorithmsProvider.class);  

    XadesSigner signer = profile.newSigner();   

    DataObjectDesc obj1 = new DataObjectReference("")
    .withTransform(new ExclusiveCanonicalXMLWithoutComments())
    .withTransform( new XPathTransform(xPath);

    SignedDataObjects dataObjs = new SignedDataObjects().withSignedDataObject(obj1);

changed 2012-11-20 begin

//  signer.sign(dataObjs, docToSign.getDocumentElement() ); 
       new Enveloped(signer).sign(docToSign.getDocumentElement());

changed 2012-11-20 end

and here is the verify code

NodeList signatureNodeList = getSigElement(getDocument("my/my-document.signed.bes.countersign.xml"));

for (int i = 0; i < signatureNodeList.getLength(); i++) {
    Element signatureNode = (Element) signatureNodeList.item(i);
    verifySignature(signatureNode, new XadesVerificationProfile(VerifierTestBase.validationProviderMySigs));
    log.info("successful validation");          
}

public static XAdESForm verifySignature(Element sigElem,
            XadesVerificationProfile p) throws Exception {
        XAdESVerificationResult res = p.newVerifier().verify(sigElem, null);

        return res.getSignatureForm();
    }

It looks like that there is a document about this problem in Apache Santuario FAQ ,

2.6. I sign a document and when I try to verify using the same key, it fails
After you have created the XMLSignature object, before you sign the document, you must embed the signature element in the owning document (using a call to XMLSignature.getElement() to retrieve the newly created Element node from the signature) before calling the XMLSignature.sign() method,

During canonicalisation of the SignedInfo element, the library looks at the parent and ancestor nodes of the Signature element to find any namespaces that the SignedInfo node has inherited. Any that are found are embedded in the canonical form of the SignedInfo. (This is not true when Exclusive Canonicalisation is used, but it is still good practice to insert the element node prior to the sign() method being called).

If you have not embedded the signature node in the document, it will not have any parent or ancestor nodes, so it will not inherit their namespaces. If you then embed it in the document and call verify(), the namespaces will be found and the canonical form of SignedInfo will be different to that generated during sign().

also there is a document about this problem as following

https://stackoverflow.com/a/12759909/1809884

It looks like that it is not a bug of xades4j, but a xml signature problem.

--add 2012-11-15

here is how to get the docToSign . in fact , i just reused the code in class  SignatureServicesTestBase . so i am sure that it is namespaceaware. 
static
    {
           DocumentBuilderFactory dbf = DocumentBuilderFactory.newInstance();
            dbf.setNamespaceAware(true);
           db = dbf.newDocumentBuilder();
    }
 public static Document getDocument(String fileName) throws Exception
    {
        String path = toPlatformSpecificXMLDirFilePath(fileName);
        Document doc = db.parse(new FileInputStream(path));
        // Apache Santuario now uses Document.getElementById; use this convention for tests.
        Element elem = doc.getDocumentElement();
        DOMHelper.useIdAsXmlId(elem);
        return doc;
    }

and docToSign  is return by calling SignatureServicesTestBase.getDocument()

Document docToSign = SignatureServicesTestBase.getDocument("my/cdamessage.xml");

and the SignedProperties element as following

<xades:SignedSignatureProperties>
<xades:SigningTime>2012-11-15T13:58:26.167+09:00</xades:SigningTime>
<xades:SigningCertificate>
<xades:Cert>
<xades:CertDigest>
<ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"/>
<ds:DigestValue>4btVb5gQ5cdcNhGpvDSWQZabPQrR9jf1x8e3YF9Ajss=</ds:DigestValue>
</xades:CertDigest>
<xades:IssuerSerial>
<ds:X509IssuerName>CN=Itermediate,OU=CC,O=ISEL,C=PT</ds:X509IssuerName>
<ds:X509SerialNumber>-119284162484605703133798696662099777223</ds:X509SerialNumber>
</xades:IssuerSerial>
</xades:Cert>
<xades:Cert>
<xades:CertDigest>
<ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"/>
<ds:DigestValue>vm5QpbblsWV7fCYXotPhNTeCt4nk8cLFuF36L5RJ4Ok=</ds:DigestValue>
</xades:CertDigest>
<xades:IssuerSerial>
<ds:X509IssuerName>CN=TestCA,OU=CC,O=ISEL,C=PT</ds:X509IssuerName>
<ds:X509SerialNumber>-46248926895392336918291885380930606289</ds:X509SerialNumber>
</xades:IssuerSerial>
</xades:Cert>
<xades:Cert>
<xades:CertDigest>
<ds:DigestMethod Algorithm="http://www.w3.org/2001/04/xmlenc#sha256"/>
<ds:DigestValue>AUaN+IdhKQqxIVmEOrFwq+Dn22ebTkXJqD3BoOP/x8E=</ds:DigestValue>
</xades:CertDigest>
<xades:IssuerSerial>
<ds:X509IssuerName>CN=TestCA,OU=CC,O=ISEL,C=PT</ds:X509IssuerName>
<ds:X509SerialNumber>-99704378678639105802976522062798066869</ds:X509SerialNumber>
</xades:IssuerSerial>
</xades:Cert>
</xades:SigningCertificate>
</xades:SignedSignatureProperties>
</xades:SignedProperties>

also , i use xpath to get the elements to be signed, and namespace(xmlns="urn:hl7-org:v3" ) is add into the result as well .

543  DEBUG [main] org.apache.xml.security.utils.ElementProxy     - setElement("ds:Transform", "null")
544  DEBUG [main] org.apache.xml.security.utils.ElementProxy     - setElement("dsig-xpath:XPath", "null")
658  DEBUG [main] org.apache.xml.security.utils.DigesterOutputStream     - Pre-digested input:
658  DEBUG [main] org.apache.xml.security.utils.DigesterOutputStream     - <component xmlns="urn:hl7-org:v3" Id="ES" contextConductionInd="true" typeCode="COMP">
        <section classCode="DOCSECT" moodCode="EVN">
          <code code="ES" codeSystem="2.16.840.1.113883.6.1" codeSystemName="SectionCode" codeSystemVersion="1.0" displayName="english"></code>
          <text>english</text>
        </section>
      </component>

something wrong with xpath ? xpath is driving my crazy . I think i have to study xpath from beginning now.

chris