wyrdforge

How do I bring together automated tests and a layered application?

Currently I am working on a (modular) project around access card and alarm coin management.
Basically I am following the principles in “Desiging Elixir Systems with OTP”, which seems a reasonable choice, regarding how and by whom the rest of the application parts is written.

That out of the way, my git repo now has the functional core project, a phoenix project and an (Ecto) persistence project as a “Poncho” project and deployed in a container.

Every layer, as per the book, will have it’s own tests and docs and I’m struggling a bit to find the right way to run the subproject’s tests and generators in a (Gitlab-)CI pipeline, as they are deployed in a single container. So, my best bet would be to trigger a shell script, that runs mix test --trace, mix coveralls.html and mix docs in every layer’s project and copies the results to some “artifact” dir, or split tests out into different containers (all layers for end-to-end and integration, persistence and core for DB, core only), before compiling them together in one.

But somehow both options feel not, as if they were the way it was meant to be done. Am I missing something there, or is Gitlab-CI just not the right tool for this task?

5 comments

#deployment #testing #ci #gitlab

1 1749 5

2020-10-20 13:21:21 UTC

Marked As Solved

evadne

Overall the solution we have come up with is to create a proper execution environment with Docker.

From the codebase we’d build 2 containers — 1 for deployment (running an Elixir release, configured to listen on a specific port etc) and 1 for testing (containing source code, MIX_ENV=test, able to run mix test, mix dialyzer, etc).

Then we write a Docker Compose file to bring up associated services such as PostgreSQL. Then it is a simple matter of running the desired command on TeamCity

I think the key is to recognise that you can build more than 1 Docker container from the same project and that if you can keep things simple (so running mix test from the root of the umbrella project tests everything) the solution is also simple.

As to coveralls… it would seem that you actually have different kinds of tests — Unit Tests, Code Coverage Tests, etc. Don’t write any custom code to try and combine this into 1 test and 1 piece of output. Instead, leverage whatever CI/CD platform you have and run each kind of test separately.

You can probably create some kind of dependencies between the different types of tests/checks/builds as well. Again, this depends on what your CI/CD platform is capable of. I would recommend taking a few days to properly research your options.

Post #4

Also Liked

wyrdforge

For reference and hopefully as a little starter for others, I will put my current Gitlab CI configuration here.

This setup leads to the following pipeline with DAG configured for speeding up:

.gitlab-ci.yml:

stages:
  - build:engine
  - build:interface
  - lint
  - test:engine
  - test:interface
  - create:deployment
  - deploy

include:
  - local: "ci/build.yml"
  - local: "ci/lint.yml"
  - local: "ci/test.yml"
  - local: "ci/create.yml"
#  - local: "ci/deploy.yml"

variables:
  MIX_ENV: 'test'

ci/build.yml:

.elixir_default: &elixir_default
  image: wyrdforge/elixir:1.11.1-alpine
  before_script:
    - mix local.hex --force
    - mix local.rebar --force

compile:persistence:
  <<: *elixir_default
  stage: build:engine
  script:
    - cd stackgate_persistence
    - mix deps.get --only-test
    - mix compile --warnings-as-errors
  artifacts:
    when: on_success
    expire_in: 1 hrs
    paths:
      - stackgate_persistence/_build
      - stackgate_persistence/deps

compile:engine:
  <<: *elixir_default
  stage: build:engine
  script:
    - cd stackgate_engine
    - mix deps.get --only-test
    - mix compile --warnings-as-errors
  artifacts:
    when: on_success
    expire_in: 1 hrs
    paths:
      - stackgate_engine/_build
      - stackgate_engine/deps

compile:interface:
  <<: *elixir_default
  stage: build:interface
  needs: ["compile:persistence", "compile:engine"]
  script:
    - cd stackgate_interface
    - mix deps.get --only-test
    - mix compile --warnings-as-errors
  artifacts:
    when: on_success
    expire_in: 1 hrs
    paths:
      - stackgate_interface/_build
      - stackgate_interface/deps

ci/lint.yml:

.elixir_default: &elixir_default
  image: wyrdforge/elixir:1.11.1-alpine
  before_script:
    - mix local.hex --force
    - mix local.rebar --force

lint:persistence:
  <<: *elixir_default
  stage: lint
  needs: ["compile:persistence"]
  script:
    - cd stackgate_persistence
    - mix format --check-formatted
  dependencies:
    - compile:persistence
    
lint:engine:
  <<: *elixir_default
  stage: lint
  needs: ["compile:engine"]
  script:
    - cd stackgate_engine
    - mix format --check-formatted
    - mix credo --strict
  dependencies:
    - compile:engine

lint:interface:
  <<: *elixir_default
  stage: lint
  needs: ["compile:interface"]
  script:
    - cd stackgate_interface
    - mix format --check-formatted
  dependencies:
    - compile:interface

ci/test.yml:

.elixir_default: &elixir_default
  image: wyrdforge/elixir:1.11.1-alpine
  before_script:
    - mix local.hex --force
    - mix local.rebar --force

test:persistence:
  <<: *elixir_default
  stage: test:engine
  needs: ["lint:persistence", "compile:persistence"]
  script:
    - cd stackgate_persistence
    - mkdir cover
    - mix test --trace --no-color | tee cover/test_report.txt
  dependencies:
    - compile:persistence
  artifacts:
    expire_in: 15 min
    paths:
       - stackgate_persistence/cover

test:engine:
  <<: *elixir_default
  stage: test:engine
  needs: ["lint:engine", "compile:engine"]
  script:
    - cd stackgate_engine
    - mkdir cover
    - mix coveralls.html --trace --no-color | tee cover/test_report.txt
  dependencies:
    - compile:engine
  artifacts:
    expire_in: 15 min
    paths:
       - stackgate_engine/cover


test:interface:
  <<: *elixir_default
  stage: test:interface
  needs: ["lint:interface", "test:persistence", "test:engine", "compile:interface"]
  script:
    - cd stackgate_interface
    - mkdir cover
    - mix test --trace --no-color | tee cover/test_report.txt
  dependencies:
    - compile:interface
  artifacts:
    expire_in: 15 min
    paths:
       - stackgate_interface/cover

ci/create.yml:

.elixir_default: &elixir_default
  image: wyrdforge/elixir:1.11.1-alpine
  before_script:
    - mix local.hex --force
    - mix local.rebar --force

create:documentation:
  <<: *elixir_default
  stage: create:deployment
  needs: ["test:persistence", "test:engine", "test:interface", "compile:persistence", "compile:engine", "compile:interface"]
  script:
    - cd stackgate_persistence
    - mix docs --output ../stackgate_persistence/docs/
    - cd ../stackgate_engine
    - mix docs --output ../stackgate_engine/docs/
    - cd ../stackgate_interface
    - mix docs --output ../stackgate_interface/docs/
  dependencies:
    - compile:persistence
    - compile:engine
    - compile:interface
    - test:engine
    - test:persistence
    - test:interface
  artifacts:
    paths:
      - stackgate_engine/cover
      - stackgate_persistence/cover
      - stackgate_interface/cover
      - stackgate_engine/docs
      - stackgate_persistence/docs
      - stackgate_interface/docs

create:app_container:
  <<: *elixir_default
  stage: create:deployment
  needs: ["test:persistence", "test:engine", "test:interface"]
  variables:
    MIX_ENV: prod
  script:
    - echo "We will use a multistage Dockerfile for compiling a distillery release here and build the container for staging and production."
    - echo "Automatic deployment to staging and manual deployment after revision to prod will follow later on in the 'deploy' stage."
  dependencies: []

Attention: The job create:app_container must explicitly clear dependencies. This way, we omit all test/dev-debris in artifacts for this job and we can compile from scratch with MIX_ENV=prod.

Post #6