# Definition of Done for Simantics Platform Releases
-1. The `simantics/platform` and `simantics/third-party` Git repositories have a branch `release/x.y.z[.w]` and tag `x.y.z[.w]`.
+1. All Git repositories that are part of the Simantics release train have a branch called `release/x.y.z[.w]` and tag called `x.y.z[.w]`.
2. [A change log entry](https://www.simantics.org/redmine/projects/simantics-platform/wiki/ChangeLog) is compiled from the issues in this release and made available to the general public separately for the platform and for the open products included in the release train.
3. [Roadmap](http://dev.simantics.org/index.php/Roadmap) is up-to-date.
4. [Tutorials](http://dev.simantics.org/index.php/Tutorials) are up-to-date and coherent with the platform.
Plug-in components are installable features that are deployed online as P2 repositories for general availability. Products are deployed as ZIP files and made available online in designated locations on simantics.org.
+Core components of the release train:
+* External third party library dependencies required by the platform - [simantics/third-party.git](https://gerrit.simantics.org/r/gitweb?p=simantics/third-party.git;a=summary)
+* The Simantics Platform - [simantics/platform.git](https://gerrit.simantics.org/r/gitweb?p=simantics/platform.git;a=summary)
+
Plug-in components that are part of the release train:
-* Simupedia - [members/simupedia.git](https://www.simantics.org:8088/r/gitweb?p=members/simupedia.git;a=summary)
-* FMIL - [simantics/fmil.git](https://www.simantics.org:8088/r/gitweb?p=simantics/fmil.git;a=summary)
-* FMI Studio - [members/fmi.git](https://www.simantics.org:8088/r/gitweb?p=members/fmi.git;a=summary)
-* Interoperability components - [simantics/interop.git](https://www.simantics.org:8088/r/gitweb?p=simantics/interop.git;a=summary)
-* Simantics R binding - [simantics/r.git](https://www.simantics.org:8088/r/gitweb?p=simantics/r.git;a=summary)
-* Matlab SCL binding - [simantics/matlab.git](https://www.simantics.org:8088/r/gitweb?p=simantics/matlab.git;a=summary)
-* Python SCL binding - [simantics/python.git](https://www.simantics.org:8088/r/gitweb?p=simantics/python.git;a=summary)
-* Simantics System Dynamics - [simantics/sysdyn.git](https://www.simantics.org:8088/r/gitweb?p=simantics/sysdyn.git;a=summary)
-* Simantics District modelling components - [simantics/district.git](https://www.simantics.org:8088/r/gitweb?p=simantics/district.git;a=summary)
+* Simupedia - [members/simupedia.git](https://gerrit.simantics.org/r/gitweb?p=members/simupedia.git;a=summary)
+* FMIL - [simantics/fmil.git](https://gerrit.simantics.org/r/gitweb?p=simantics/fmil.git;a=summary)
+* FMI Studio - [members/fmi.git](https://gerrit.simantics.org/r/gitweb?p=members/fmi.git;a=summary)
+* Interoperability components - [simantics/interop.git](https://gerrit.simantics.org/r/gitweb?p=simantics/interop.git;a=summary)
+* Simantics R binding - [simantics/r.git](https://gerrit.simantics.org/r/gitweb?p=simantics/r.git;a=summary)
+* Matlab SCL binding - [simantics/matlab.git](https://gerrit.simantics.org/r/gitweb?p=simantics/matlab.git;a=summary)
+* Python SCL binding - [simantics/python.git](https://gerrit.simantics.org/r/gitweb?p=simantics/python.git;a=summary)
+* Simantics System Dynamics - [simantics/sysdyn.git](https://gerrit.simantics.org/r/gitweb?p=simantics/sysdyn.git;a=summary)
+* Simantics District modelling components - [simantics/district.git](https://gerrit.simantics.org/r/gitweb?p=simantics/district.git;a=summary)
+* Simantics 3D modelling components - [simantics/3d.git](https://gerrit.simantics.org/r/gitweb?p=simantics/3d.git;a=summary)
+* Simantics Proteus toolset - [gold-members/proteus.git](https://gerrit.simantics.org/r/gitweb?p=gold-members/proteus.git;a=summary)
Products that are part of the release train:
* Simantics Desktop
# Simantics Platform Release - Step by Step
-## Create release branch from selected commit
-
-When release stabilisation starts, branch `simantics/platform` and `simantics/third-party` repositories:
+## Create release engineering issue(s) in Gitlab
- git clone ssh://<user>@www.simantics.org:29418/simantics/platform.git
- cd platform
- git branch release/x.y.z[.w] <commit>
- git push origin release/x.y.z[.w]
+Ensure that Gitlab has a release engineering issue for the branched release,
+such as [Simantics 1.34.0 release engineering](https://gitlab.simantics.org/simantics/platform/issues/16)
+* Make a new release issue with title `$version release engineering`.
+* Add a link to the previous previous release's issue in the new issue description, e.g. `simantics/platform#16`.
+* Add labels `releng` and `$version` to the new issue
- git clone ssh://<user>@www.simantics.org:29418/simantics/third-party.git
- cd third-party
- git branch release/x.y.z[.w] <commit>
- git push origin release/x.y.z[.w]
+## Create release branch from selected commit
+When release stabilisation starts, all relevant repositories need to be branched.
When creating major/minor releases `<commit>` is usually a commit in the `master` branch.
With service releases, branch from an existing `release/*` branch instead.
-## Prepare release branch for use
-
-In release 1.31.0 Simantics started using uses the wonderful tool from Obeo ([https://github.com/mbarbero/fr.obeo.releng.targetplatform](https://github.com/mbarbero/fr.obeo.releng.targetplatform)) to generate `.target` files from `.tpd` files which allow much simpler specification of the target contents and also composition of .tpd files via inclusion.
-
-In the following steps, it is recommended to ensure every `.target` file is up-to-date by regenerating them from `.tpd` files. While doing so, take care to use up-to-date online contents to perform the generation.
-
-### Prepare `.tpd` files
-
-1. Retrieve release branch of the platform repository
-
- git clone ssh://<user>@www.simantics.org:29418/simantics/platform.git
- cd platform
- git branch release/x.y.z[.w] remotes/origin/release/x.y.z[.w]
- git checkout release/x.y.z[.w]
-
-2. Edit all target platform files in `releng/org.simantics.sdk.build.targetdefinition/`, i.e.
- * `org.simantics.sdk.build.targetdefinition.tpd`
- * `simantics.tpd`
-
- Replace the following rows in both mentioned files:
-
- ~~~
- location "http://www.simantics.org/download/master/external-components/maven" {
- location "http://www.simantics.org/download/master/external-components/manual" {
- include "http://www.simantics.org/download/master/org.simantics.sdk.build.targetdefinition.tpd"
- location "http://www.simantics.org/download/master/sdk" {
- ~~~
-
- with
-
- ~~~
- location "http://www.simantics.org/download/release/x.y.z[.w]/maven" {
- location "http://www.simantics.org/download/release/x.y.z[.w]/manual" {
- include "http://www.simantics.org/download/release/x.y.z[.w]/org.simantics.sdk.build.targetdefinition.tpd"
- location "http://www.simantics.org/download/release/x.y.z[.w]/sdk" {
- ~~~
-
-3. Edit version number of `org.simantics.sdk` feature in `features/org.simantics.sdk.feature/feature.xml` to `x.y.z[.w]`.
- ~~~
- <feature
- id="org.simantics.sdk"
- label="Simantics SDK"
- version="x.y.z"
- provider-name="VTT Technical Research Centre of Finland">
- ~~~
-
- An example of these changes can be seen in [gitweb](https://www.simantics.org:8088/r/gitweb?p=simantics/platform.git;a=commit;h=bab5c9bd68277c76dc5c20bc7a60a9896cbd1540).
-
-4. Ensure that Redmine has a release engineering issue for the branched release, such as [Simantics 1.30.0 release engineering](https://www.simantics.org/redmine/issues/7263). Make a copy of the previous release issue to create the new one. Include link to original issue while copying.
-
-5. Commit the changes made
-
- git commit -a
-
- with the commit message
-
- Configured release/x.y.z[.w] branch for SDK builds.
-
- refs #xxxx
-
- where `#xxxx` is the number of the x.y.z[.w] release engineering issue and push them to remote
-
- git push origin release/x.y.z[.w]
-
-6. If you are branching from `master`, bump the revision of master right now to start the next release cycle in master.
- An example of these changes can be seen in [gitweb](https://www.simantics.org:8088/r/gitweb?p=simantics/platform.git;a=commitdiff;h=ae93c9930c6345c32219e6845b9e72e9d9d2d28c).
-
- Commit the changes with the following commit message
-
- Bumped master target and org.simantics.sdk feature versions to x.y.z[.w].
-
- refs #yyyy
-
- where `#yyyy` is the number of the next release's release engineering issue.
-
-### Initialize release branch distribution web site
-
-* Run [SDK/Deploy External Components to Web](https://www.simantics.org/jenkins/job/SDK/job/Deploy%20External%20Components%20to%20Web/) build with parameters:
- * **GERRIT_REFNAME:** `release/x.y.z[.w]`
- * **PUBLISH_ARTIFACTS:** `true`
-* Run the [SDK/Simantics SDK](https://www.simantics.org/jenkins/job/SDK/job/Simantics%20SDK/) build with parameters:
- * **GERRIT_REFNAME:** `release/x.y.z[.w]`
- * **PUBLISH_ARTIFACTS:** `true`
-
-Running these two builds will ensure that both the external components required to build the SDK and the Simantics SDK for the new release branch are published online at `http://www.simantics.org/download/release/x.y.z[.w]/`.
-
-After this, whenever changes are pushed/merged to `release/x.y.z[.w]` branch in Gerrit, new **SDK/Simantics SDK** builds are triggered automatically and they will publish the results at the same location online.
-
-This means that one does not have to do any tricks after this to build and publish the SDK as a P2 repository online. It is an automated process that is performed by the [SDK/Simantics SDK](https://www.simantics.org/jenkins/job/SDK/job/Simantics%20SDK/) Jenkins job.
-
-### Update `.target` files
-
-Lastly, we want to change the `.target` files to point to the correct P2 repository locations. This happens by opening the previously edited `.tpd` files in Obeo's editor and regenerating the `.target` files by pressing Alt+R.
-
-Push the changed files to remote again with commit message:
-
- Configured release/x.y.z[.w] branch .target files.
-
- refs #xxxx
+Instead of doing this clone+branch+push sequence for every SDK repository separately, please use the included `release-helper.sh` shell script to perform this mechanical work.
+The script supports many useful basic git commands which all perform the requested operation on all the release train repositories.
+The operations can be seen from the help text printed by the script when given no arguments:
+
+```
+Usage: release-helper.sh <version> <branch-name> <user-name> <command> [<command-arguments>]
+
+Commands:
+ clone The first thing that needs to be done before anything else
+ Clones all platform repositories under directory <version>
+
+Inspection commands:
+ diff [args] Run git diff [args] for each platform repository
+ log [args] Run git log [args] for each platform repository
+ status [args] Run git status [args] for each platform repository
+ list-tags Run git tag -l for each repository
+
+Action:
+ add
+ branch Run git branch <branch-name> for each platform repository
+ checkout Run git checkout <branch-name> for each repository
+ commit
+ fetch Run git fetch --all for each repository
+ pull Run git pull --all for each repository
+ push Run git push origin <branch> for each repository
+ push-tags Run git push --tags for each repository
+ remove-tag Run git tag -d v<branch> for each repository
+ reset [args] Run git reset [args] for each repository
+ tag Run git checkout <branch> and
+ git tag -a v<branch> -m "Simantics <branch> simultaneous release"
+ for each repository
+
+Compound release commands:
+ prepare-release-branch <from-branch>
+ <from-branch> the name of the branch that the codebase is currently on
+
+Top-level release commands:
+ bump-master-version <from-version> <to-version>
+ <from-version> the version string to replace
+ <to-version> the replacing new version string
+
+ branch-release <from-branch>
+ <from-branch> the branch to create the service branch from
+ e.g. master or release/x.y.z
+```
+
+Begin by cloning all the repositories for working on
+
+ user=<your-git-username>
+ version=x.y.z
+ branch=release/x.y.z
+ ./release-helper.sh $version $branch $user clone
+
+This will create a directory called `x.y.z` under the current working directory and clone all the repositories under it.
+The first `version` argument is only used for the base working directory `x.y.z`.
+The second `branch` argument tells what branch to specify to some of the available actions, e.g. `branch`, `checkout`, etc.
+
+Creating the release branch from master and pushing the new branches to remote happens as follows:
+
+ # When the commit messages appear, add references to created
+ # Gitlab release engineering issues by adding this to the commit message:
+ #
+ # gitlab #xxxx
+ #
+ # where xxxx is the release engineering issue for the respective project
+ ./release-helper.sh $version $branch $user branch-release master
+ ./release-helper.sh $version $branch $user push
+
+If you're creating a service release `x.y.w` from `x.y.z` where `w > z`, do this:
+
+ fromBranch=release/x.y.z
+ branch=release/x.y.w
+ ./release-helper.sh $version $branch $user branch-release $fromBranch
+ ./release-helper.sh $version $branch $user push
+
+If you branched a new release from `master`, you should also bump the revision of the master branch right now to start the next release cycle in master.
+An example of these changes can be seen in [gitweb](https://gerrit.simantics.org/r/gitweb?p=simantics%2Fplatform.git;a=commit;h=564ac84a2949b19ce5c1c7c838b575527ec42b09) or in [review 1362](https://gerrit.simantics.org/r/#/c/1362).
+The changes include:
+ * Simantics Desktop splash screen update (version number)
+ * Simantics target platform name to state correct development version
+ * org.simantics.sdk feature version bump
+ * org.simantics.sdk.repository project version bump
+
+Perform and push the changes *in master* by running:
+
+ branch=master
+ ./release-helper.sh $version $branch $user checkout
+ ./release-helper.sh $version $branch $user bump-master-version x.y.z X.Y.Z
+ ./release-helper.sh $version $branch $user push
+
+where `x.y.z` is the old version number of master and `X.Y.Z` is the new version number.
### Test Release Train build
Trigger an execution of the [simantics-release-train](https://www.simantics.org/jenkins/job/simantics-release-train/) Jenkins job with parameters
-* GERRIT_REFNAME: `release/x.y.z[.w]`\r
-* PUBLISH_ARTIFACTS: `true`\r
+* GERRIT_REFNAME: `release/x.y.z[.w]`
+* PUBLISH_ARTIFACTS: `true`
to ensure that all components of the release train build fine and to publish first versions of P2 repositories of everything online.
+The release train build is currently never ran automatically.
+However, after following all these steps, whenever changes are pushed/merged to `release/x.y.z[.w]` branch in Gerrit, new Jenkins builds are triggered automatically and they will publish the results (P2 repositories/products) at their designated locations online.
+
+#### [Optional reading] Release train build details
+
+The release train build does two very important steps as the first things,
+i.e. build the external dependency P2 repositories for both the
+Platform SDK and Simupedia:
+~~~
+stage('External Dependencies') {
+ node {
+ build job: 'SDK/Deploy External Components to Web', parameters: params1
+ build job: 'Member Components/Simupedia/fi.semantum.simupedia.build.p2.site', parameters: params1
+ }
+}
+~~~
+After that it builds the Platform SDK and Simupedia before building anything else:
+~~~
+stage('SDK') {
+ node {
+ build job: 'SDK/Simantics SDK', parameters: params1
+ // Note that Simantics SDK builds simupedia-git as well
+ }
+}
+~~~
+
## Review documentation
Documentation to review:
When the release branches are ready for the release, tag them with the tag `vx.y.z[.w]`:
- git clone ssh://<user>@www.simantics.org:29418/simantics/platform.git
- cd platform
- git checkout release/x.y.z[.w]
- git tag vx.y.z[.w] -m "Simantics x.y.z[.w] simultaneous release"
- git push origin --tags
-
- git clone ssh://<user>@www.simantics.org:29418/simantics/third-party.git
- cd third-party
- git checkout release/x.y.z[.w]
- git tag vx.y.z[.w] -m "Simantics x.y.z[.w] simultaneous release"
- git push origin --tags
+ ./release-helper.sh x.y.z release/x.y.z your-git-username checkout
+ ./release-helper.sh x.y.z release/x.y.z your-git-username pull
+ ./release-helper.sh x.y.z release/x.y.z your-git-username tag
+ ./release-helper.sh x.y.z release/x.y.z your-git-username push-tags
-> Note The -m argument must be supplied to create an [annotated tag](https://git-scm.com/book/en/v2/Git-Basics-Tagging).
+> Note: release-helper.sh will supply the -m argument to git tag to create an [annotated tag](https://git-scm.com/book/en/v2/Git-Basics-Tagging).
> Only annotated or signed tags can be pushed to Gerrit.
## Backup documentation wiki databases
+**NOTE: This step is now deprecated since we are in the process of moving all mediawiki-based documentation into the platform git repository.**
+
This step is only necessary for major/minor releases, not for service releases.
The wiki databases to be backed up are: