-
Type:
New Feature
-
Status: Resolved (View Workflow)
-
Priority:
Major
-
Resolution: Fixed
-
Component/s: jenkins-test-harness
-
Labels:
-
Similar Issues:
Lots of plugin functional tests using JenkinsRule assume some kind of software environment on the testing host, typically for running external processes (for example SCM tools), but this is rarely explicit. That makes it complicated to assure that both developer machines and CI builders have all required software packages installed and in the correct versions and configurations.
The acceptance-test-harness includes a very useful DockerFixture system which addresses this for acceptance tests. But acceptance tests are slow, flaky, and external to plugin sources, so most test code is written using functional tests.
We need to be able to reuse the same system in both kinds of tests.
In the case of mercurial-plugin it turns out that merely adding fixtures does not absolve you of the need to have hg installed to run most tests, since many are either designed to run on master specifically, or do or could use an agent but would still need to run hg to implement caching—and JenkinsRule cannot run the master in a container. (ATH can, but not at the discretion of the test, so it does not do much better in this regard.)
Another problem I find is that existing fixtures are not set up to define exact, reproducible software versions. In fact it seems this is impossible to do on Ubuntu, since the package repositories do not host historical versions. For a particular software product you could do an explicit build, though this seems clumsy.
I guess the best workaround is to use a tagged image from DockerHub and only add those things to Dockerfile which seem version-independent.
It's worth noting that JenkinsRule tests that run builds running Docker have problems on OS X due to symlinking of /tmp. Not sure if that'd be a problem here, though.
tests that run builds running Docker have problems on OS X
Here I am only porting something that has been in broad use for a long time in acceptance-test-harness, so if those tests passed for you then this should too.
Code changed in jenkins
User: Jesse Glick
Path:
pom.xml
src/main/java/org/jenkinsci/test/acceptance/docker/Docker.java
src/main/java/org/jenkinsci/test/acceptance/docker/DockerContainer.java
src/main/java/org/jenkinsci/test/acceptance/docker/DockerFixture.java
src/main/java/org/jenkinsci/test/acceptance/docker/DockerImage.java
src/main/java/org/jenkinsci/test/acceptance/docker/DynamicDockerContainer.java
src/main/java/org/jenkinsci/test/acceptance/docker/fixtures/JavaContainer.java
src/main/java/org/jenkinsci/test/acceptance/docker/fixtures/SshdContainer.java
src/main/resources/org/jenkinsci/test/acceptance/docker/fixtures/JavaContainer/Dockerfile
src/main/resources/org/jenkinsci/test/acceptance/docker/fixtures/SshdContainer/Dockerfile
src/main/resources/org/jenkinsci/test/acceptance/docker/fixtures/SshdContainer/unsafe
src/main/resources/org/jenkinsci/test/acceptance/docker/fixtures/SshdContainer/unsafe.pub
src/main/resources/org/jenkinsci/test/acceptance/docker/fixtures/SshdContainer/unsafe_enc_key
src/main/resources/org/jenkinsci/test/acceptance/docker/fixtures/SshdContainer/unsafe_enc_key.pub
src/main/resources/org/jenkinsci/test/acceptance/docker/fixtures/SshdContainer/unsafe_enc_key_passphrase.txt
src/test/java/org/jenkinsci/test/acceptance/docker/DockerFixtureTest.java
src/test/java/org/jenkinsci/test/acceptance/docker/DockerImageTest.java
src/test/java/org/jenkinsci/test/acceptance/docker/DynamicDockerContainerTest.java
src/test/resources/org/jenkinsci/test/acceptance/docker/DockerFixtureTest/TestContainer/Dockerfile
src/test/resources/test/Dockerfile
http://jenkins-ci.org/commit/acceptance-test-harness/d682a3da859cc67ea86749642b6b1a102256f25f
Log:
Merge pull request #252 from jglick/docker-fixtures
JENKINS-40808 Using docker-fixtures repository
Compare: https://github.com/jenkinsci/acceptance-test-harness/compare/c26b4e0cec68...d682a3da859c
Code changed in jenkins
User: Jesse Glick
Path:
Jenkinsfile
pom.xml
src/main/java/hudson/plugins/mercurial/MercurialInstallation.java
src/main/resources/hudson/plugins/mercurial/MercurialInstallation/help-executable.html
src/test/java/hudson/plugins/mercurial/ChangeComparatorTest.java
src/test/java/hudson/plugins/mercurial/CustomConfigTest.java
src/test/java/hudson/plugins/mercurial/DisableChangeLogTest.java
src/test/java/hudson/plugins/mercurial/EnvVarTest.java
src/test/java/hudson/plugins/mercurial/FunctionalTest.java
src/test/java/hudson/plugins/mercurial/HgExeFunctionalTest.java
src/test/java/hudson/plugins/mercurial/MatrixProjectTest.java
src/test/java/hudson/plugins/mercurial/MercurialContainer.java
src/test/java/hudson/plugins/mercurial/MercurialContainerTest.java
src/test/java/hudson/plugins/mercurial/MercurialRule.java
src/test/java/hudson/plugins/mercurial/MercurialSampleRepoRule.java
src/test/java/hudson/plugins/mercurial/PipelineTest.java
src/test/java/hudson/plugins/mercurial/SCMTestBase.java
src/test/resources/hudson/plugins/mercurial/MercurialContainer/Dockerfile
http://jenkins-ci.org/commit/mercurial-plugin/2903930fd367b73312f305b9c82b356162493ae0
Log:
Merge pull request #93 from jglick/docker-fixtures
JENKINS-40808 Using docker-fixtures repository
Compare: https://github.com/jenkinsci/mercurial-plugin/compare/5cc6e1c2b06b...2903930fd367
An alternative would be to use Arquillian Cube Standalone as in this demo by Alex Soto/Alex Soto. The upside is of course use of a third-party library which is presumably maintained by others. Downsides:
Closer to the spirit of DockerFixture would be junit-docker-rule or docker-junit-rule, neither of which seems to support building images on demand from Dockerfile; or docker-compose-rule, which sounds good for multi-container situations, though it imposes an additional requirement that Compose be installed.
Also none of these seem to have any equivalent to ATH’s ability to subclass DockerContainer to define reusable, type-safe container classes.