SCAVA Development Process

Introduction

This document presents the guidelines about the processes and tools supporting the development of the Scava platform. In particular, Sec 2 presents an overview of the main development workflow by highlighting the main tools of the supporting development infrastructure. Section 3 mentions peculiar conventions that have to be adopted to write source code and to perform changes. Section 4 makes a list of tools supporting communication and collaboration among the different partners. Section 5 gives some important remarks related to the Scava project and its development.

Process overview

The development of the Scava components and of the integrated platform should follow the process shown in the following figure. Essentially, the development relies on the availability of a source code repository (to enable collaborative development) and of a Continuous Integration Server (to do the building and integration on behalf of the developers). Different deployment environments can be considered for deploying the built system.

It is important to remark that Continuous Integration (CI) requires that the different developers have self-testing code. This is code that tests itself to ensure that it is working as expected, and these tests are often called unit tests. When all the unit tests pass after the code is integrated, developers will get a green build. This indicates that they have verified that their changes are successfully integrated together, and the code is working as expected by the tests.

The different elements shown below are individually described in the remaining of the section.

Dev Process for Scava
Figure 1. Overview of the CROSSMINER development process and tools.

Source Code Repository

Different branches will be created in the repository. In particular, the master branch is the main branch and represents the released state of the product. It is always clean, builds and runs all the tests successfully. The dev branch contains the most recent code integrated by partners before release.

Other branches will be created for the different features/components, which will be eventually merged in the dev branch. The name of each branch contains the id of the issue it is supposed to implement.

All source code, binary code and configuration needed to build a working implementation of the system will reside in a GitHub repository.

Since all source code will be stored in a single GitHub repository, each developer will have a working copy of the entire project. To simplify collaboration, each component will be contained in one subdirectory within the source structure (see Sec. 3 for details).

By taking inspiration from https://datasift.github.io/gitflow/IntroducingGitFlow.html the branching model that will be followed will resemble that shown in Fig. 2 without making use of the release and hotfixes branches. Details about the change conventions, pull requests, etc. are given in Sec. 3.

Git development workflow
Figure 2: Branching model

Tests

Each branch must have the corresponding unit tests. The dev branch must have integration tests and the unit tests of all the features that have been already merged as shown below.

Master and branches
Figure 3: Explanatory master and branches

BRANCH: Set of branches
TEST: Set of tests
INTGRATION_TEST < TEST
UNIT_TEST < TEST

function test: BRANCH -> TEST

test(master) -> test(cool-feature)
    ⋃ {itu, … itz}

Continuous Integration Server

Continuous integration (CI) involves integrating early and often, so as to avoid the pitfalls of "integration hell". The practice aims to reduce rework and thus reduce cost and time.

A complementary practice to CI is that before submitting work, each programmer must do a complete build and run (and pass) all unit tests. Integration tests are usually run automatically on a CI server when it detects a new commit. In particular, a CI server provides developers with at least the following functionalities:

The currently used CI server is available at http://ci5.castalia.camp:8080/

Development and Production environments

According to the DoW we have not promised the availability of a public installation of the CROSSMINER platform (i.e., a production environment). However, in order to implement the different use cases, use case partners are provided with docker images enabling the local installation of the whole CROSSMINER platform.

Naming and change conventions

The repository will contain the code of the high-level components shown in Table 1

Components Corresponding folder in repository Leader
DevOps Dashboard BIT
Workflow Diagram Editor /crossflow YORK
Administration Web Application /administration SFT
IDE /eclipse-based-ide FEA
API Gateway /api-gateway SFT
DevOps Backend BIT
Knowledge Base /knowledge-base UDA
Project Analyser /metric-platform SFT
Data Collector /metric-platform SFT
Project Scheduler /metric-platform SFT
Metric Providers /metric-platform YORK, CWI, AUEB, EHU
Data Storage /metric-platform SFT
Web-based dashboards /web-dashboards BIT

Table 1: Leaders and Contributors of the CROSSMINER Deployment diagram nodes

Scava components architecture
Figure 4: Scava components architecture

For each component a corresponding folder is available in the repository. Changes must have a CR (i.e. issue) entry for traceability. In particular, the requirements and objectives of each Scava components will be detailed in the relevant deliverables and they will be added in the form of items in the provided Bug Tracking infrastructure. In this way progress on the implementation of each requirement can be tracked. Thus:

The partner in charge of managing the development of a component will be free to organize the content of the corresponding subdirectory in any way as long as the following standard files/directories are contained in the component folder:

As a reference, please have a look at https://github.com/crossminer/scava/tree/master/knowledge-base

The following items are typical practices when applying CI: * Maintain a code repository * Automate the build * Make the build self-testing * Everyone commits to the baseline every day * Every commit (to baseline) should be built * Keep the build fast * Test in a clone of the production environment * Make it easy to get the latest deliverables * Everyone can see the results of the latest build * Automate deployment

Communication and collaboration means

The development activities can be done by exploiting different communication and collaboration means. In particular, currently the consortium members have the availability of: