javamavenmaven-wagon-plugin

Maven custom wagon Could not validate integrity of download Troubleshooting


I have implemented a custom maven wagon interacting with cloud based storage. The files are uploaded successfully to the cloud storage and the checksums are correct.

md5 of maven-metadata.xml matches with the contents of maven-metadata.xml.md5

The same happens with the jar and pom files

  1. md5 of jar matches the contents of *.jar.md5
  2. md5 of maven-metadata.xml matches the contents of maven-metadata.xml.md5
  3. md5 of pom matches the contents of *.pom.md5

When I retrieve the files through http instead of using the custom wagon they are downloaded without any checksum validation exceptions.

However when I retrieve the files using the custom wagon the I get the integrity of download warnings.

[WARNING] Could not validate integrity of download from gs://mavenbucket/snapshot/com/gkatzioura/storage/CloudStorageTest/1.0-SNAPSHOT/maven-metadata.xml
 org.eclipse.aether.transfer.ChecksumFailureException: Checksum validation failed, expected 7e0c3c33db781362483c0baed3ba175352945028 but is da39a3ee5e6b4b0d3255bfef95601890afd80709
at org.eclipse.aether.connector.basic.ChecksumValidator.validateExternalChecksums (ChecksumValidator.java:174)
at org.eclipse.aether.connector.basic.ChecksumValidator.validate (ChecksumValidator.java:103)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$GetTaskRunner.runTask (BasicRepositoryConnector.java:456)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$TaskRunner.run (BasicRepositoryConnector.java:360)
at org.eclipse.aether.util.concurrency.RunnableErrorForwarder$1.run (RunnableErrorForwarder.java:75)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector$DirectExecutor.execute (BasicRepositoryConnector.java:583)
at org.eclipse.aether.connector.basic.BasicRepositoryConnector.get (BasicRepositoryConnector.java:232)

Once the artifact is downloaded through the custom wagon successfully i check the md5 of the files downloaded from the cloud storage and they match with the *.md5 file in the cloud storage.

What is the process of troubleshooting this issue and how can I debug it?


Solution

  • The reason why this was happening had to do with not calling the transferProgress method of the TransferListeners.

    In order to do so you have to provide a mechanism so that when your OutputStream (while downloading) and your InputStream (while uploading) get copied are also going to call the transferProgress method.

    For example

    import java.io.File;
    import java.io.FileNotFoundException;
    import java.io.FileOutputStream;
    import java.io.IOException;
    
    import org.apache.maven.wagon.events.TransferEvent;
    import org.apache.maven.wagon.events.TransferListener;
    
    public class WrappedOutputStreamExample extends FileOutputStream {
    
        private TransferListener transferListener;
    
        public WrappedOutputStreamExample(File file,TransferListener transferListener) throws FileNotFoundException {
            super(file);
            this.transferListener = transferListener;
        }
    
        @Override
        public void write(int b) throws IOException {
            super.write(b);
            TransferEvent transferEvent = null; //provide corrent transfer event
            this.transferListener.transferProgress(transferEvent,new byte[]{(byte) b}, 1);
        }
    
        @Override
        public void write(byte b[]) throws IOException {
            super.write(b);
            TransferEvent transferEvent = null; //provide corrent transfer event
            this.transferListener.transferProgress(transferEvent,b, b.length);
        }
    
        @Override
        public void write(byte b[], int off, int len) throws IOException {
            TransferEvent transferEvent = null; //provide corrent transfer event
    
            super.write(b, off, len);
            if (off == 0) {
                this.transferListener.transferProgress(transferEvent,b, len);
            } else {
                byte[] bytes = new byte[len];
                System.arraycopy(b, off, bytes, 0, len);
                this.transferListener.transferProgress(transferEvent,bytes, len);
            }
        }
    
    }