SAP CCV2 – How to do nightly builds and deploys (CI/CD)

Continuous integration and continuous delivery or in short CI/CD is a common practice nowadays and therefor a widely adopted way to deliver your software to its intended destination.
CI/CD is commonly used for:

• Building
• Unit testing
• Integration testing
• Code analyses (like SonarCloud for example)
• Reports
• Releases

Each of the mentioned use cases come with their own challenges and considerations when employing CI/CD, but ‘consistency is key’ applies to all of them. The actions specified in your CI/CD tool should always operate the same way, making it predictable and repeatable. The goal is to replace procedures of manual actions with automated processes that can be started with the press of a single button (or scheduled).
On the net there are a variety of articles that can be found that will go much deeper into the CI/CD philosophy and will give a more comprehensive explanation on the topic.

I am going to show you how we implemented nightly builds using the following tools.

• SAP Commerce
• Docker
• Bitbucket Pipelines
• SAP CCV2 API for builds and deployments

Let’s start with one big difference in V1 and V2 of the commerce cloud platform.
On V2 you can’t deploy your own build packages, this is simply not possible anymore. In the cloud portal you connect your git repository to the platform and create builds from the web interface SAP provides you with.
This of course has some pros and cons on its own.

Pros

• Anyone with the correct access rights can trigger a build/deploy on the platform.
• Little to no technical knowledge required
• Builds are done by SAP managed servers (their build lane, so it must good right?)

Cons

• When using SAP cloud extension pack you cannot specify your patch version and wil always get the latest patch version released by SAP. They claim to always be backwards compatible and that the patches are non-breaking, however we have noticed that this not always the case
• CI/CD you had with V1 could not be used in the early stages of the V2 platform
• Theres no way to integrate in the CI/CD lane of SAP, its closed!

So how do we deploy through CI/CD onto the V2 cloud platform?

SAP introduced build and deployment APIs at the end of 2019.

Below a copy pasted list from there documentation demonstrating which APIs they made available.

• getBuildsFind the API command and parameters that support the request to get all Commerce Cloud builds associated with a specific subscription.
• createBuildFind the API command and parameters that support the request to create a Commerce Cloud build.
• getBuildFind the API command and parameters that support the request to get Commerce Cloud build details.
• getBuildLogsFind the API command and parameters that support the request to get the logs for a Commerce Cloud build.
• getBuildProgressFind the API command and parameters that support the request to get Commerce Cloud build progress.
• getDeploymentsFind the API command and parameters that support the request to get all Commerce Cloud deployments associated with a specific subscription.
• createDeploymentFind the API command and parameters that support the request to create a Commerce Cloud deployment.
• getDeploymentFind the API command and parameters that support the request to get Commerce Cloud deployment details.
• createDeploymentCancellationFind the API command and parameters that support the request to cancel a Commerce Cloud deployment.
• getDeploymentCancellationOptionsFind the API command and parameters that support the request to get Commerce Cloud deployment cancellation options.
• getDeploymentProgress

In addition to the above listed items, they also made a CLI tool available called sapccm to be used through a terminal or automated process.
We tried the CLI tool to be used in our CI/CD solution and even locally on our development machine, but we came to the conclusion that this tool was not stable enough to be used in a CI/CD solution. It threw SocketTimeOutExceptions (yes it is written in Java) constantly when waiting on a build process to finish, losing the status of the progress. Progress that was needed to trigger a deploy.

So we decided to write our own CLI tool called SAP CX CLI Acorel (SCCA), scca which consumes the SAP REST API, we also dockerized it so it was easy to use in a CI/CD solution. We won’t go into depth on the code itself but rather on how we used it in our CI/CD solutions.
Although we did decide to make it open source for anyone to benefit from our solutions (Dockerfile included). You can find it on Bitbucket here
Feel free to improve it, use it, make PR, leave feedback or make a suggestion.

We will be discussing the following topics in this example:

1. Build your code CI/CD (We will not be discussing this)
2. Trigger a build in SAP Commerce Cloud V2 (And wait for it to finish)
3. Trigger a deploy on the D1 environment (And forget about it)

We host our projects on Atlassian Bitbucket Cloud and there the CI/CD solution provided is called Bitbucket Pipelines.

When point one is finished we want to build (again), only then on the ccv2 platform and when finished successfully trigger a deployment to the D1 environment, using our own CLI tool in the CI/CD.

Prerequisites

• Dockerized version of SCCA ( SAP CX CLI Acorel)
• SAP API token secret variable in Bitbucket Pipelines
• SAP Subscription id secret variable in bitbucket Pipelines
  More about variables in Bitbucket pipelines can be found here.

Below an example how we use scca to trigger a build on the CCV2 platform:

- step: &cloud-build
    clone:
      enabled: false
    options:
      size: 1x
    name: Trigger build SAP Cloud
    image:
      name:  /sap-cx-cli-acorel:latest
      aws:
        access-key: $AWS_ACCESS_KEY
        secret-key: $AWS_SECRET_KEY
    artifacts: # defining the artifacts to be passed to each future step.
      - build-output.txt
    script:
      - scca config set --api-token=$SAP_API_TOKEN --sap-subscription-id=$SAP_SUBSCRIPTION_ID
      - scca build create --branch=$BITBUCKET_BRANCH --name=$(echo "$BITBUCKET_BRANCH-$BITBUCKET_BUILD_NUMBER" | sed -e "s/^.*///") > $BITBUCKET_CLONE_DIR/build-output.txt

Lets go over what we see here in the script section of the step.

- scca config set --api-token=$SAP_API_TOKEN --sap-subscription-id=$SAP_SUBSCRIPTION_ID

This configures scca to use the correct API Token and subscription id.
Next we have the build create, there is a lot more going on here.

- scca build create --branch=$BITBUCKET_BRANCH --name=$(echo "$BITBUCKET_BRANCH-$BITBUCKET_BUILD_NUMBER" | sed -e "s/^.*///") > $BITBUCKET_CLONE_DIR/build-output.txt

• –branch=$BITBUCKET_BRANCH # Specifies the branch used for this build
• –name=$(echo “$BITBUCKET_BRANCH-$BITBUCKET_BUILD_NUMBER” | sed -e “s/^.*///”) > # Sets the name that will be displayed in the cloud portal for this build.
• > $BITBUCKET_CLONE_DIR/build-output.tx # The output generated by this command is stored in the file build-output.txt

The reason why we store the output of this create build command is that we need to share de build code generated by SAP CCV2 to the next step, the deploy to the D1 environment. Because this a separate step the docker instance used will be reset and forgets about the earlier progress or tasks it has done. The create build command of scca doing poll for the progress of the build thus when complete the build is done on the SAP platform.

You can skip this by appending the –no-wait to the command.

Now we have build ready to be deployed on the SAP Commerce cloud v2, below an example on how to the build this using scca.
Deploy step

- step: &cloud-deploy
    clone:
      enabled: false
    options:
      size: 1x
    name: Deploy SAP Cloud D1
    image:
        name: <url to your docker image>/sap-cx-cli-acorel:latest
        aws:
          access-key: $AWS_ACCESS_KEY
          secret-key: $AWS_SECRET_KEY
    script:
      - scca config set --api-token=$SAP_API_TOKEN --sap-subscription-id=$SAP_SUBSCRIPTION_ID
      - scca build deploy --build-code=$(grep -Po '(?<=build-code:s).*' $BITBUCKET_CLONE_DIR/build-output.txt) --environment-code=$environment --no-wait

On the first row of the script we are setting the configuration again, just like we did with the creating a build step discussed earlier.The magic happens on the second row, where we use the scca build deploy command.

 - scca build deploy --build-code=$(grep -Po '(?<=build-code:s).*' $BITBUCKET_CLONE_DIR/build-output.txt) --environment-code=$environment --no-wait

• –build-code, but with grep. This ensures that we grep the build code that was generated by SAP and use it to trigger this build.
• –environment-code this is a variable specified as an deployment variable in Bitbucket. More on the topic can be found here
• –no-wait, this tells scca to not wait for the deploy to finish.

When this step is completed bitbucket considers the deploy to be successful.
You may have noticed we did not specify any update strategies and extras like that, thats because scca has some default if not specified but can be overriden by adding the following parameters to the deploy command.

--strategy= The way the deployment is handled [default: ROLLING_UPDATE]
--database-update-mode= DB update mode [default: UPDATE]

Possible options for database update mode are NONE, UPDATE, and INITIALIZE
Possible options for strategy are ROLLING_UPDATE and RECREATE.

Now we are able to build and deploy on the SAP Commerce cloud v2, only thing left is to schedule a nightly trigger.
How to do that can be found here for Bitbucket pipelines

scca sources