srcdeps-maven-plugin an Intro

Peter Palaga, Red Hat
2015-09-29

The problem (1/2)

An organization produces a deliverable (a.k.a. main distro) using Java and Maven

  • The distro consists of several components living in separated source repositories
  • Components have formally independent release cycles
  • During development, changes happen on all levels
components dependencies

The problem (2/2)

  • How can we propagate the changes from components to the main distro?
components dependencies

Conventional solutions

Maven -SNAPSHOT s

  • The code behind a -SNAPSHOT version of an artifact can change in the course of time
  • Therefore the builds of projects depending on -SNAPSHOT s are non-reproducible in course of time

Maven timestamped SNAPSHOT s

Release components often

  • 100% reproducible
  • Many short living dev releases clutter maven repositories
  • Releasing costs developers' time
  • The components may be governed outside of our organization and they do not release fast enough

Going unconventional?

  • The build of every git commit is perfectly deterministic
  • Why don’t we depend on git commits directly?

  • We could build the components on the fly from their sources!

    • No need for releasing and even having a maven repo (in an extreme case)
    • Nevertheless we can fully get rid of releases done just for the sake of testing and integration

srcdeps-maven-plugin -SRC- dependendcies

A Maven plugin that allows a Maven Project A to depend on source revision, tag or branch of another project B.

<!-- In project A's pom.xml -->
<dependencies>
  <dependency>
    <groupId>org.example.b</groupId>
    <artifactId>artifact-b</artifactId>
    <version>0.0.1-SRC-revision-66ea95d8</version>(1)
  </dependency>
</dependencies>
1 -SRC- is the marker that makes Srcdeps Plugin to interpret the rest of the version string as a scmVersionType, scmVersion pair. In this case scmVersionType is revision and scmVersion is 66ea95d8 - i.e. the sha1 of the git commit we want to build as a dependency.

srcdeps-maven-plugin configuration

<plugin>
  <groupId>org.l2x6.maven.srcdeps</groupId>
  <artifactId>srcdeps-maven-plugin</artifactId>
  <version>0.0.4</version>
  <extensions>true</extensions>(3)
  <configuration>
    <repositories>
      <repository>
        <id>group-b</id>(4)
        <selectors>
          <selector>org.example.b</selector>(1)
        </selectors>
        <url>scm:git:https://github.com/example/project-b.git</url>(2)
      </repository>
      ...
    </repositories>
  </configuration>
</plugin>
1 This is how the plugin knows which SCM URL belongs to which artifact maked with -SRC- version. For now <selector> is the groupId a source dependency
2 The SCM (Source Code Management) URL to checkout the sources from.
3 srcdeps-maven-plugin is a Maven extension rather than a usual Maven plugin because it needs to listen to basic Maven lifecycle events
4 An id that should map 1:1 to the URL

How srcdeps-maven-plugin works

The build of org.example.main-distro issued:

cd project-a; mvn clean install
  • An afterProjectsRead() listener method of srcdeps-maven-plugin is called very early in the Maven build process, even before the dependencies of the project are resolved.
  • Scan the dependencies for versions containing -SRC-
  • For each -SRC- dependency, find source repository in srcdeps-maven-plugin 's configuration

  • For each source repository:

    • Checkout the sources to ~m2/../dependency-sources
    • Switch to revision/tag/branch specified after -SRC- in the version string
    • Change the version of the sources to string declared in the main project using mvn versions:set -DnewVersion=0.0.1-SRC-revision-66ea95d8
    • Issue mvn clean install in the checkout directory

  • This can result in several levels of nested builds
  • When srcdeps-maven-plugin finishes its work, all -SRC- dependencies are installed in the local Maven repository
  • Maven continues with its usual build steps, the dependency resolution succeeds because all necessary -SRC- dependenciies were recently installed in the local Maven repository.

Miscelaneous srcdeps-maven-plugin features

SCM agnostic

scmVersionType in -SRC-{scmVersionType}-{scmVersion} can be anything the given SCM supports * for most SCMs scmVersionType : {revision|tag|branch}

srcdeps.quiet use -q for nested builds to reduce log length (sometimes important for Travis)

srcdeps.skipTests, srcdeps.mavenTestSkip to make the nested builds faster

A pro tip: Fallback URLs

A way how I can build the project A that has a -SRC- dependency on project B on my machine even before the change was accepted in the upstream of project A.

  • If a revision is not found using the SCM URL configured in plugin’s pom.xml configuration, the fallback URLs are used
  • ATM (plugin version 0.0.4), the fallback URLs can be passed only through properties following a special naming scheme:
mvn clean install -Dsrcdeps.url.1.group-b=scm:git:file:///home/ppalaga/git/project-b/.git

Where:

  • 1 is an index (1 is the lowest possible index)
  • group-b is the id of the SCM repository URL from srcdeps-maven-plugin’s configuration that this fallback should apply to

srcdeps-maven-plugin project

← →

/