gradle - download and unzip file from url

Question

What would be the proper gradle way of downloading and unzipping the file from url (http)?

If possible, I'd like to prevent re-downloading each time I run the task (in ant.get can be achieved by skipexisting: 'true').

My current solution would be:

task foo {
  ant.get(src: 'http://.../file.zip', dest: 'somedir', skipexisting: 'true')
  ant.unzip(src: 'somedir' + '/file.zip', dest: 'unpackdir')
}

still, I'd expect ant-free solution. Any chance to achieve that?

Answer 1

Let's say you want to download this zip file as a dependency:

https://github.com/jmeter-gradle-plugin/jmeter-gradle-plugin/archive/1.0.3.zip

You define your ivy repo as:

repositories {
    ivy {
        url 'https://github.com/'

        patternLayout {
            artifact '/[organisation]/[module]/archive/[revision].[ext]'
        }

        // This is required in Gradle 6.0+ as metadata file (ivy.xml) 
        // is mandatory. Docs linked below this code section
        metadataSources { artifact() } 
    }
}

reference for required metadata here

The dependency can then be used as:

dependencies {
    compile 'jmeter-gradle-plugin:jmeter-gradle-plugin:1.0.3@zip'
    //This maps to the pattern: [organisation]:[module]:[revision]:[classifier]@[ext]         
}

To unzip:

task unzip(type: Copy) {

  def zipPath = project.configurations.compile.find {it.name.startsWith("jmeter") }
  println zipPath
  def zipFile = file(zipPath)
  def outputDir = file("${buildDir}/unpacked/dist")

  from zipTree(zipFile)
  into outputDir

}

---

optional:

If you have more than one repository in your project, it may also help (for build time and somewhat security) to restrict dependency search with relevant repositories.

Gradle 6.2+:

repositories {
    mavenCentral()
    def github = ivy {
        url 'https://github.com/'
        patternLayout {
            artifact '/[organisation]/[module]/archive/[revision].[ext]'
        }
        metadataSources { artifact() }
    }
    exclusiveContent {
        forRepositories(github)
        filter { includeGroup("jmeter-gradle-plugin") }
    }
}

Earlier gradle versions:

repositories {
    mavenCentral {
        content { excludeGroup("jmeter-gradle-plugin") }
    }
    ivy {
        url 'https://github.com/'
        patternLayout {
            artifact '/[organisation]/[module]/archive/[revision].[ext]'
        }
        metadataSources { artifact() }
        content { includeGroup("jmeter-gradle-plugin") }
    }
}

Answer 2

    plugins {
        id 'de.undercouch.download' version '4.0.0'
    }

    /**
     * The following two tasks download a ZIP file and extract its
     * contents to the build directory
     */
    task downloadZipFile(type: Download) {
        src 'https://github.com/gradle-download-task/archive/1.0.zip'
        dest new File(buildDir, '1.0.zip')
    }

    task downloadAndUnzipFile(dependsOn: downloadZipFile, type: Copy) {
        from zipTree(downloadZipFile.dest)
        into buildDir
    }

https://github.com/michel-kraemer/gradle-download-task

Answer 3

There isn't currently a Gradle API for downloading from a URL. You can implement this using Ant, Groovy, or, if you do want to benefit from Gradle's dependency resolution/caching features, by pretending it's an Ivy repository with a custom artifact URL. The unzipping can be done in the usual Gradle way (copy method or Copy task).