Teaching how to use Developer Cloud Service to promote ICS Integrations into new Environments

Why not to have the best of the two worlds? That is, a simple web UI that allows you to easily integrate your SaaS and On-Premise applications, as well as a mature DevOps tooling, that allows you to store your integrations in a corporate version control repository and fully automate your deployments, continuous integration and continuous delivery of your integration projects.

Well, in this blog I am going to show you how simple it is to use Developer Cloud Service to manage your ICS Integrations in a DevOps fashion.

That is:

1. DevOps person pushes a change in an ICS Integration into the corporate Git repository,
2. A build task is triggered based on a Git code change being detected. Hudson will automatically trigger a build task,
3. Hudson will build and package the ICS Integration and archive the result as a release for future deployment,
4. A deployment task is triggered to deploy the ICS Integration into a configurable target ICS environment.
5. Optionally, we could run tests and report status, to ensure the new code release is functional as expected.

This blog assumes current knowledge on how to interact with ICS system APIs. If you need more information refer to this other article that I wrote some time ago, that focuses on the ICS system REST APIs that make it possible to interact with ICS remotely.

Pre-requisites

• It is assumed that you already have access to an Oracle Integration Cloud Service and Oracle Developer Cloud Service instances. If you don’t have an instance yet, now days with Cloud, this is simple, you can get provisioned an account in minutes, simply go to: https://cloud.oracle.com/tryit and request a new trial.
• Also, Git will be used extensively, make sure to open a terminal window and make sure git is installed by checking its version (i.e. git –version). if you don’t have git installed you can simply: sudo apt-get install git (or yum it if using OEL/RH).
• Make sure you have Java installed and if using Linux and also make sure that your user belongs to the sudo group. In my case, since my Ubuntu env was new, my user was not in the sudo group and I did not have Java installed, which is needed to Jar and un-Jar the ICS Integration archives (IARs)… You might surely not need these steps, but I recorded them below just in case you do:

• To add your user to the sudo group in Ubuntu:

usermod -aG sudo [USERNAME]

• To install Java SDK, so that I could use jar (remove the proxy if not needed):

sudo http_proxy=http://www-proxy.au.oracle.com apt-get update

sudo http_proxy=http://www-proxy.au.oracle.com apt-get install default-jre -y

sudo http_proxy=http://www-proxy.au.oracle.com apt-get install default-jdk -y

java -version // Validate Java exists now…

Exporting and Importing ICS Integration into DevCS Git repo

There are multiple ways you can export a given ICS Integration:

• You can export the integration manually from within the ICS Console:

• Or you can do it programmatically via a REST API call (see this blog that will show you how)

  Once you have downloaded the ICS Integration archive (IAR), we simply need to push it into your DevCS Git repo. For this, you can use any IDE that understands Git, such as Eclipse, JDeveloper, IntelliJ, NetBeans, etc. Another way to do it is via command line (either Windows or Linux). In this case I am going to use a Linux based command line:

• If your DevCS is new, make sure that you have created a Git repo in your DevCS project

• However, perhaps this is not a new Git repo. Maybe you already have other ICS Integrations there and you want to simply add a new one or register a few changes made through the ICS console.
• Copy the HTTPS location of your Git repo

• Use the HTTPS Git repo location to clone the git repo locally. Normally you want to work on your own branch. However for the sake of this blog, we are simply working on master branch to avoid adding extra concepts that are irrelevant to this article.

If you want to understand how to work with git branches, I recommend this other blog that Ali Mukadam wrote.

git clone https://user@devcsURL/GIT_REPO_NAME.git

e.g.

git clone https://user@developer.us.oracle.com/devservice-templates/s/devservice-templates_s2v-iia_10850/scm/ICS_Integrations.git

Enter the password when prompted.

This will download the current project within a new folder.

• Move to this directory:

  cd ICS_Integrations
  

• Now, we are going to add the ICS Integration into this new folder. Given the IAR is mainly formed out of text/metadata configuration files, I prefer to expand the IAR. That way, I am able to easily run diff commands and easily track changes in the Git repository… But this is totally up to you.
• Create a new folder with the same name as your Integration IAR file:

  mkdir S2VIIA_SC_GETCONTA_INT
  

• Move to this directory:

  cd S2VIIA_SC_GETCONTA_INT
  

• Copy the S2VIIA_SC_GETCONTA_INT.iar that you exported previously into this directory
• Expand this IAR:

  jar xvf S2VIIA_SC_GETCONTA_INT.iar
  

• Perhaps you want to remove the original IAR file. This is to avoid confusion later and maintain just one source of truth.

  rm S2VIIA_SC_GETCONTA_INT.iar
  

• This will create a folder called icspackage within which all the ICS Integration files including metadata, configurations, resources, etc. are located.

  find .
  

Let’s build our ICS Integration Environment Properties File

As part of the DevCS Build job, we need to call a few ICS system APIs. An easy way to so it is by using Shell Steps in Hudson within DevCS.

It’s up to us how generic we make these shell commands. I like the idea of making it in a way that:

• Everything is parametrised, so that we can easily reuse the code to deploy new ICS integrations
• Also, I like the idea of using a local env-[DEV|TEST|PROD] file that get’s pushed into our Git repo. This makes sure that we can also version our ICS Integration environment configuration properties and we reduce the amount of effort needed to run new Build jobs in Hudson.
• However, I don’t like the idea of leaving username and passwords in clear text in properties files that get pushed into a Git repository. For this reason, we are going to leave these username and password configuration properties out of the env-[DEV|TEST|PROD] and instead configure them in DevCS as password properties. More of this in the next section…

For now, let’s move into a properties file all the configurations that we are going to need as part of the ICS Integration deployment via the ICS system REST APIs

Note: If you need a refresh on the ICS system REST APIs that we are going to be using, read this this blog, as indicated earlier.

For this particular ICS Integration that I am building, I am integrating into Sales Cloud and exposing simple to consume REST APIs, thus I need to configure as part of the Integration a Sales Cloud Connector and a REST Connector.

The way you decide to parametrise your ICS system REST APIs is totally up to you. Below there is an example of how I chose to build my “env” configuration file, use it as a guide to build yours:

NOTE: Depending on your integration, you might have more or less connectors. It’s the DevOps engineer’s responsibility to know what adapters belong to each Integration.

Further things could be parameterized, for example the ICS integration version, ICS server, Datacentre, etc. – This is totally up to your needs.

#    Integration:

export ICS_INTEGRATION_IAR_FILENAME=”S2VIIA_SC_GETCONTA_INT.iar”

export ICS_INTEGRATION_POST_IMPORT_URI=https://ics-icsIdDomain.integration.us2.oraclecloud.com/icsapis/v1/integrations/import

export ICS_INTEGRATION_POST_ACTIVATE_URI=
“https://ics-crisoacsjan17.integration.us2.oraclecloud.com/icsapis/v1/integrations/S2VIIA_SC_GETCONTA_INT/01.00.0000/activate”

#    Connector: Sales Cloud:

export ICS_CONNECTOR_SC_CONFIG_NAME=”S2VIIA_SALESCLOUD_CONN1.json”

export ICS_CONNECTOR_SC_URI=https://ics-icsIdDomain.integration.us2.oraclecloud.com/icsapis/v1/connections/S2VIIA_SALESCLOUD_CONN1

#    Connector: REST:

export ICS_CONNECTOR_REST_CONFIG_NAME=”S2VIIA_REST_CONN.json”

export ICS_CONNECTOR_REST_URI=”https://ics-icsIdDomain.integration.us2.oraclecloud.com/icsapis/v1/connections/S2VIIA_REST_CONN”

#    Security:

#ICS_USERNAME – TO BE SET AT DEVCS TO AVOID USERNAMES BEING STORED IN GIT REPO

#ICS_PASSWD – TO BE SET AT DEVCS AS A PASSWORD PARAMETER, TO AVOID PASSWORDS BEING STORED IN GIT REPO

NOTE: Since this configuration file refers to this specific ICS Integration, it make sense to save this file within the integration folder itself (S2VIIA_SC_GETCONTA_INT/)

Let’s configure our ICS Connectors according to my target environment

As part of the DevCS Build job, when we export an ICS Integration, it comes in full including its associated connectors, however, these are not configured. It makes sense if you think about it. You wouldn’t like to export a TEST ICS Integration and promote it to PROD and by mistake forget to re-configure the connectors which are still pointing to a TEST environment… And find that out later on when this is a big issue. Even worse, you wouldn’t like to be creating a TEST environment from a PROD environment and forget to modify the connector configurations and keep pointing to PROD from TEST.

Well, sleep assured that this won’t happen, because the ICS Connectors don’t come fully configured after an export. You need to configure them as part of a re-deployment into another ICS environment.

In a previous blog (see here), I explained that in order to re-configure an ICS Connector to point to a designated environment as part of a re-deployment, you need to send a json file attached to the API Call. You don’t need to send all the configuration parameters, but only those that you want to modify. These could be WSDL locations, username, passwords, etc. However in order to avoid having to build your own JSON documents, which personally I don’t find exciting, a trick that I normally do is to GET the connector’s configuration first and then simply add the modifications. That way we can be sure that the format is right and even better, this way we can version control the environment connector’s configuration.

That said, GET all your Connectors’ JSON files, configure them (if needed) and store them once again within your ICS integration expanded folder, so that you can Git push them together with the rest of your project.

This is the API that you need to use (see this bog for more information on how to configure the Connector once you GET it):

curl -u username:password -H “Content-Type:application/json” -X GET https://server[:port].integration.us2.oraclecloud.com/icsapis/v1/connections/CONNECTOR_NAME

Note: Before proceeding, make sure all your Connectors’ configurations are in the right folder and fully configured, i.e. Based on the nature of the ICS connector, you might need to setup WSDL locations, WS endpoints, usernames, passwords, etc. For example, for my ICS Integration I don’t need to do anything for my REST Connector JSON file. However, for my Sales Cloud Connector, I need to setup the targetWSDLURL, eventCatalogURL and Sales Cloud credentials:

NOTE: Once again, it’s the DevOps engineer’s responsibility to configure these connectors properly and for each specific ICS target deployment environment.

Once the fully configured your ICS connector based on the target environment, place your JSON files within the ICS project. A good place to locate it is together with the env-[DEV|TEST|PROD] file and the main icspackage filder..

Let’s Git push our ICS Integration to DevCS

• Move back at the ICS_Integrations level (git root) and run the following git commands to push your ICS Integration into your DevCS project.

• Add all new files and directories into your local git repo:

git add -A

• Commit all changes into your local git repo:

git commit -m “Adding S2VIIA_SC_GETCONTA_INT ICS project”

Note:
If you get a legend that says: *** Please tell me who you are. Followed by: fatal: unable to auto-detect email address (got ‘cciturria@middlewarepc1.(none)’)

Run:

git config user.email “you@example.com”

git config user.name “Your Name”

For example:

git config user.email “carlos.rodriguez.iturria@oracle.com”

git config user.name “Carlos Rodriguez Iturria”

And then, repeat your commit command:

git commit -m “Adding S2VIIA_SC_GETCONTA_INT ICS project”

• Push all committed changes into the remote git repo, that in this case is your DevCS git repo.

  git push origin master
  

    Enter the password when prompted.

• When it completes, run: git status to make sure that you have nothing else to commit.

• Go back to Developer Cloud Service and make sure that your git push came through:

• If you go to Code on the left menu, you will be able to drill down into the structure being created.

• Make sure that your env-[DEV|TEST|PROD] configuration file, as well as your ICS Connectors JSON config files are under the Integration folder.

  Look closely: This is crucial to be able to deploy the ICS Integration in the next section.
  

• You can also see this commit in the Graph section. Right now, there is only 1 commit on the master branch, but eventually as we start working on different branches and with plenty of commits, this Graph view becomes very useful.

Let’s configure DevCS to trigger a Build job

Follow the next few steps to configure your build and deployment jobs:

• Go to the Build tab in DevCS and create a new Job.

• Give it a sensible name and Save it.

• In the Main tab, enter a good Description to help others understand its purpose.
• In the Build Parameters tab, tick “This build is parameterized”
  

  NOTE: We are going to create some Parameters that we don’t want to store in our “dev-INTEGRATION.properties” file that lives in our Git repo… For example, username and passwords.

  Also, I like the idea of creating an ICS_INTEGRATION_NAME property, that we can use to load our environment properties during the Build steps. That way we can very easily switch across DEV, Test and Production environments.

  Let’s create these properties next…

• Add a new Parameter of type “String Parameter”

• Call it: ICS_INTEGRATION_ENV
  
• Then set its Default Value as the ICS environment that you want to deploy to. This has to match with the name of your environmental file that you created previously and that you pushed into your git repo, i.e. env-[DEV|TEST|PROD] – In my case I used TEST.
  

• Repeat the last few steps and create those properties that you did not want to keep in the Git repo. For example, the various usernames and passwords.

  Note: I recommend using the type “Password Parameter” instead of String Parameter for those sensitive properties that you don’t want to disclose.
  

  • Integration Name:
    • ICS_INTEGRATION_NAME = [SET YOUR ICS INTEGRATION NAME]
• ICS Access Account:
  • ICS_USERNAME = [SET YOUR ICS USERNAME]
  • ICS_PASSWD = [SET YOUR ICS PASSWORD]

• Now Click on the “Source Control” sub-tab. Drop down the URL and select your ICS_Integrations Git repository.
• Expand the “Advanced Repository Settings” and set Name to origin.
• Then click “Add” on Branches and click the drop down menu to select the branch where your ICS Project was pushed in, in this case it is on the “master” branch.
• The final configuration will look like this:

• Click on the “Environment” sub-tab and tick the “Abort the build if it’s stuck” option and increase the Timeout to something a bit larger. Although the build should not take that long, when having multiple ICS Integrations to be deployed, this can take a few minutes…

• Click on the “Build Steps” sub-tab. This is where we are going to invoke the ICS system APIs to deploy and activate our ICS Integrations. This APIs are well explained in this blog that I have been referring across this article.
• Add a new Shell step by clicking on Build Step > Execute Shell.

• These are the steps of things to do in the shell script step:

  1. We are going to set our environmental properties, by sourcing the env-[DEV|TEST|PROD] file that you wish to use.

  2. Package the ICS Integration project as an IAR integration archive – We are going to use jar for this.

  3. Import the ICS Integration archive (IAR)

  4. Configure and activate my ICS Connectors:

  • Sales Cloud Connector
  • REST Connector

  5. Activate the Integration

This is a script that I put together, have a look and take the bits that work for you:

# 1. We are going to set our environmental properties, by sourcing the env-[DEV|TEST|PROD] file that you wish to use.

cd ${ICS_INTEGRATION_NAME}

chmod 755 env-${ICS_INTEGRATION_ENV}

source ./env-${ICS_INTEGRATION_ENV}

# 2. Package the ICS Integration “package” as an IAR integration archive – We are going to use jar for this.

jar cvf ${ICS_INTEGRATION_IAR_FILENAME} icspackage

# 3. Import the ICS Integration archive (IAR)

curl -u “${ICS_USERNAME}:${ICS_PASSWD}” -H “Accept: application/json” -X POST -F “file=@${ICS_INTEGRATION_IAR_FILENAME};type=application/octet-stream” ${ICS_INTEGRATION_POST_IMPORT_URI} -v

# Sleep 5 seconds to give time to complete before configuring the adapters.

sleep 5

# 4. Configure and activate my ICS Connectors:

#     4.1 Sales Cloud Connector

# Configure Sales Cloud Connector:

curl -u “${ICS_USERNAME}:${ICS_PASSWD}” -H “Content-Type:application/json” -X PUT -d @${ICS_CONNECTOR_SC_CONFIG_NAME} ${ICS_CONNECTOR_SC_URI} -v

# Sleep 5 seconds to give time to complete before configuring the next adapter.

sleep 5

#    4.2 REST Connector

# Configure REST Connector, in this case I just want to force Test it, so it gets activated:

curl -u “${ICS_USERNAME}:${ICS_PASSWD}” -H “Content-Type:application/json” -X PUT -d @${ICS_CONNECTOR_REST_CONFIG_NAME} ${ICS_CONNECTOR_REST_URI} -v

# Sleep 5 seconds to give time to complete before Activating the ICS Integration.

sleep 5

# 5. Activate the Integration

curl -u “${ICS_USERNAME}:${ICS_PASSWD}” -H “Accept: application/json” -X POST ${ICS_INTEGRATION_POST_ACTIVATE_URI} -v

• Make sure to Save all your changes.
• Click on Build Now to test your deployment.

• A window will pop up to ask you to override (if needed) any of the Configuration properties that we defined in DevCS. Since we already set the default values, simply click Build.

    Having said that, you can always re-set any of the parameters at runtime. For example, pointing to a different environment by re-setting the parameter ICS_INTEGRATION_ENV to DEV.

• Your Build job will be Queued to be processed…

• If you are the only one using this environment or if there are no other Builds taking priority, this job will be taken almost immediately.
• Make sure the job is successful. Otherwise review the console output for more information.

• If you click on the job, you can see the Console output and all the history of previous builds

• Now, go to the target ICS Console and make sure your ICS integration was deployed and is fully activated!!!

• Obviously, the Connectors will also be there. Make sure they are configured properly:

Congratulations!!!

You just learned how to fully automate the deployment of ICS Integrations using tools like Git and Hudson within Developer Cloud Service.

I hope you found this article useful. If you have any question or comment, do not hesitate to contact any of the authors of solutionsanz.blog – We are here to help you!

Thanks for your time.

Author: Carlos Rodriguez Iturria

I am extremely passionate about people, technology and the most effective ways to connect the two by sharing my knowledge and experience. Working collaboratively with customers and partners inspires and excites me, especially when the outcome is noticeable valuable to a business and results in true innovation. I enjoy learning and teaching, as I recognise that this is a critical aspect of remaining at the forefront of technology in the modern era. Over the past 10+ years, I have developed and defined solutions that are reliable, secure and scalable, working closely with a diverse range of stakeholders. I enjoy leading engagements and am very active in the technical communities – both internal and external. I have stood out as a noticeable mentor running technology events across major cities in Australia and New Zealand, including various technology areas such as, Enterprise Integrations, API Management, Cloud Integration, IaaS and PaaS adoption, DevOps, Continuous Integration, Continuous Automation among others. In recent years, I have shaped my role and directed my capabilities towards educating and architecting benefits for customers using Oracle and AWS Cloud technologies. I get especially excited when I am able to position both as a way to exceed my customers’ expectations. I hold a bachelor degree in Computer Science and certifications in Oracle and AWS Solutions Architecture. View all posts by Carlos Rodriguez Iturria