Over the past few years, in everything from personal photo storage to enterprise development there has been an incredible uptake of cloud storage buckets which provide a simple, low-cost mechanism for storing unstructured data. For much longer than that, there has been a need to consume, stage and produce unstructured data in enterprise integration scenarios. File servers accessible via protocols such as FTP/SFTP, etc. are commonly used to meet the staging requirements of traditional file-based integration use cases.
The practice of using an FTP/SFTP server alongside Oracle’s cloud integration platform – Oracle Integration Cloud (henceforth, OIC), is nothing new and is well documented.
In a series of upcoming blogs, I will explore how we can use Oracle’s cloud storage buckets (Oracle Cloud Infrastructure – Object Storage) instead of a traditional FTP server in cloud-based integrations. In this initial blog I will focus on:
- Connecting to Object Storage with Oracle Integration Cloud
- Writing Files to Object Storage with Oracle Integration Cloud
- Enforcing an archival policy for files at rest
Oracle’s two major ground breaking innovation last year were Autonomous Data warehouse (ADW) and Autonomous Database Transaction processing (ATP) both are database offering suitable for different workload and are self-driving, self-securing, and self-repairing in nature. If you want to read more about these services then please go through above links.
ADW/ATP both can be quickly provisioned on Oracle Cloud Infrastructure, it’s take less than 5 minute to spin ADW/ATP instance and database is ready to connect.
User can use Oracle SQL Developer to connect to ADW/ATP database as long as they are supported version. These DBaaS services also offers out-of-box browser based SQL Developer tool which can be used to run any kind of SQL statements.
Here is sample snap of browser based SQL Developer capabilities –
Once user has Database ready, obviously there could be requirement to access data residing inside ADW/ATP instances.
Fortunately, Oracle Integration Cloud provide Adapter for connecting ADW/ATP instance, click here to know more about ATP Adapter capabilities –
In this blog I will be covering simple steps how you can connect to ADW/ATP instances using OIC Autonomous Transaction Processing Adapter (ATP) Adapter.
I made assumption that ADW/ATP instance already exists. if you not sure how to create ADW/ATP instance then refer this blog which was written by my colleague who already explained how to create ADW/ATP database instance and connect from SQL developer.
So, let move forward. Login to your Oracle Integration Cloud (OIC) home page >> Integration >> Connection >> Create >> search for “Oracle ATP” >> select the same
Continue reading “Oracle Integration Cloud Autonomous Transaction Processing Adapter Configuration !!!”
Kubernetes is a great platform to run microservices, there is no question about it. It has great features like Horizontal Pod Autoscaler and Cluster Autoscaler that make it very easy to scale whole applications depending on current or forecasted load. However with auto-scaling there are a few considerations that we need to keep in mind and one of the most important ones is that containers are ephemeral, which implies that we need to design our applications in such a way that they can scale, without compromising data persistency. There are multiple techniques available to make this possible. A common way to achieve this, is by using Persistent Volumes (PV) and Persistent Volumes Claims (PVC), that hook via the CSI (Container Storage Interface) into external disk volumes. This helps maintain state outside containers, allowing them to scale without compromising the data.
Also, with the constant embrace of Cloud providers to kubernetes, these solutions are quickly also evolving and becoming more sophisticated and easier to use. For example, now days we can extend the use of PVC with Storage Classes, implemented by the different Cloud vendors. This make the whole PV/PVC experience so enjoyable, as these storage classes become responsible to interface into the Cloud vendor IaaS land and create resources that we simply declared, while we keep reading and writing data in persistent disks.
Now, with this constant multi-cloud endorsement with kubernetes, it was a matter of time, until Cloud vendors decided to differentiate themselves by allowing the use of foreign cloud services, as first-class citizens in kubernetes. Just imagine, having the ability to use a PaaS service from “Cloud Vendor A”, seamlessly from within my kubernetes cluster that is running on “Cloud Vendor B”. The piece of magic that makes this possible is called, Open Service Broker (OSB), which is really not magic, but just a bunch of APIs that allow the control plane in kubernetes to interact with Cloud services.
In this blog, I am going to show you how to consume Oracle Cloud Infrastructure (OCI) resources from within kubernetes using the Open Service Broker. Specifically, I am going to let my kubernetes control plane to fully manage an OCI Autonomous Transaction Processing DB (ATP), as if it was a native kubernetes resource… And by the way, I am going to use OKE (Oracle managed Kubernetes), but you could very well use Google/AWS/Azure Kubernetes elsewhere and still consume OCI resources. How cool is that?
Continue reading “OCI – Using Open Service Broker within Kubernetes to bind to Autonomous Databases”
Version 1.0.0 of the Consumer Data Right standard was released in September, and it introduces a common set of Banking APIs in line with Australian government legislation. The principles behind the standards design are very solid, though the some of the specific requirements are pretty wild and they result in a bit of rethinking of some of the classical API conventions. The most prominent example of this is the approach the CDR standards take towards ‘object identifiers’, in the ID Permanence section, and I considered the requirements for this interesting enough to spend some time thinking about and documenting.
In this context, an ‘object identifier’ refers to the way in
which you refer to an individual instance of an object from your API, such as
the ‘accountId’ in the following URI:
In this blog post we will look at what the CDR requires for these types of identifiers, and provide some sample code which implements the obfuscation requirements specified in the standard.
Continue reading “Consumer Data Right (CDR) – User-specific Identifiers for ID Permanence”
In a recent blog, I explained how to approve in Kubernetes external certificate signing requests from end users. This way, users can then simply use their private keys to authenticate into Kubernetes API server. Further to this, Role Based Access Control (RBAC) can be put in place to authorise access to resources in kubernetes clusters. This is amazing and provides some level of governance, however there is a caveat, since kubernetes does not hold neither users nor groups, the identities must exist outside the cluster somewhere else. If we create external CSRs with non-existent users and groups, soon it will become very hard to properly manage all the identities, especially if we have to maintain multiple users accessing the cluster. This is yet another area that gets highly simplified when Cloud vendors embrace kubernetes as first class citizens.
In this blog, I am going to show you how to create and manage your identities in OCI IAM and simply, using such identities to access Oracle Container Engine for Kubernetes (OKE) clusters and authorise access to resources.
In a nutshell, this is what I am going to do:
- Create a new user/group in OCI IAM
- Configure an OCI policy to grant access to my user’s group to access the OKE clusters
- Create Roles and Role Bindings (RBAC) in OKE to authorise our user to access OKE resources
- Download a kubeconfig set for my new user using token validation
- Use kubectl to access only granted resources.
Ok, let’s have fun!!!
Continue reading “OCI – Oracle Container Engine for Kubernetes (OKE) – Using RBAC with IAM”
This blog will be discussing very specific use case requirement which is more developer oriented and providing a quicker and efficient solution to invoke Oracle Integration REST API using OAuth access_token for testing purpose.
As an integration developer time to time you need to invoke Oracle Integration REST API to test API functionality. All the REST API in OIC needs a header parameter called “Authorization” which must needs to hold a valid access_token value in this format “Bearer access_token“.
In above format, “Bearer” is static world, However, access_token is the token value which we get after successfully OAuth Authentication from Oracle Identity Cloud Service. Getting access_token from IDCS using code credentials flow is multi steps and cumbersome process.
However, developer can leverage Postman environment and variable features to simplify the process of getting access_token. That’s what, I will be covering in this blog.
Before I proceed further, I must redirect you to read my colleague blog which has greater in-depth explanation about what is IDCS, how IDCS govern security aspect of all Oracle PaaS products such as Oracle Integration Cloud, Analytics Cloud, Digital Assistant and so on and OAuth client and token generation process etc. I am recommending you to read his blog because I will be using few artefacts e.g. IDCS URL, Client ID, Client Secret etc which we need to get from IDCS OAuth Client Application as per the process given in his blog.
Continue reading “Simplified OAuth Config for Oracle Integration Cloud REST API using Postman !!!”
Kubernetes has been proven the best way that we have today to run microservices deployments, whether it is via a Serverless approach or by managing your own deployments in the cluster. This has solidified with the strong adoption of Kubernetes by all the major Cloud Vendors, as the strategic way to orchestrate containers and run serverless functions.
However, one of the situations that we need to be mindful, is that kubernetes creates by default a super powerful user that has full access to almost every resource in the cluster (accessible via kubectl or directly though APIs). This is very convenient for most dev & test scenarios, but it is imperative that for production workloads, we limit such power and use Role Base Access Control (RBAC), stable since version 1.8, for fine-grained authorisation access control to kubernetes cluster resources.
For the purpose of this demo, I am assuming some familiarity with Kubernetes and kubectl. I will mainly focus on the Authentication and Authorisation aspects that allow us to use Client certificates to get access to protected resources in a Kubernetes cluster.
In a nutshell this is what I am going to do:
- Create and use Client certificates to authenticate into a Kubernetes cluster
- Create a Role Base Access Control to fine grain authorise resources in the Kubernetes cluster
- Configure kubectl with the new security context, to properly limit access to resources in the Kubernetes cluster.
This is a super simplified visual representation:
Continue reading “OCI – Oracle Container Engine for Kubernetes (OKE) – Using Client Certificates and RBAC”
Recently, I came across one issue while working with one Customer where they were facing bit of challenges to update existing working Integration with new JSON field input.
So, in this use case, two integration orchestration process flow are involved, Master Integration and Child Integration both working well in production. However, time to time developer gets requirement from business to enrich this working flow by adding more fields into Child Integration which was interacting with Oracle Responsys System. The Master Integration getting data from NetSuite, passing Child Integration same data which was interacting with Responsys and completing business flow.
Here is flow diagram with existing working flow Vs flow with new requirements and highlighted problem area.
Once developer add new fields into Child OIC orchestration flow, saved, activated and tries to refresh the Parent OIC orchestration flow, it breaks existing mapping. Behind the scene obviously since Child Integration orchestration flow input payload got changed it has cascading impact on calling Integration flow and calling Integration flow fails to deduct those changes and breaks existing mapping and wiped out existing mapping. see the relevant snaps –
Continue reading “Advance XSL Mapping inside Oracle Integration Cloud !!!”
Oracle PaaS services are typically protected by Identity Cloud Service (IDCS), which provides unified access management for the Oracle Cloud. This protection extends to their exposed APIs, which usually require you to present a valid access token as part of your invocation. Unfortunately, the documentation for a number of the services tends to assume that the reader has an in-depth knowledge of OAuth concepts, and IDCS operations.
Typically my blog posts are around specific proof of concept things I have been working on, or go in-depth into more advanced IDCS concepts, but I have realised (mostly because people kept asking me…) that many developers using these services don’t have a strong practical understanding of OAuth, and struggle to interpret the documentation around the authentication requirements for the services. This blog post is designed to provide a step by step explanation of what is required from an IDCS perspective, as well as an explanation of the techniques to obtain a an access token to use the service APIs.
A non-exhaustive list of PaaS services for which these instructions
Continue reading “Invoking Oracle PaaS service APIs protected by OAuth”
- Oracle Integration Cloud
- Oracle Blockchain Platform
- Oracle API Platform Cloud Service
- Oracle Mobile Hub
- Oracle Digital Assistant
This blog is showcasing very specific use case related to NetSuite Custom Field retrieval and how we accomplish that using Oracle Integration Cloud NetSuite Adapter.
In this example we will be retrieving custom fields values e.g. birthDt from NetSuite by passing specific Customer Id. It may sound very easy but it was bit complex to deal with because the way how NetSuite was responding Custom Fields values using NetSuite Adapter.
Here is the use-case diagram –
In my previous blog I have already shown all the configuration which needs to be done to connect to NetSuite using Oracle Integration Cloud NetSuite Adapter and deal with NetSuite Custom fields. Hence, this blog will just focus on specific mapping challenges which I have faced initially during implementation.
Continue reading “Retrieve Custom Fields from NetSuite using Oracle Integration Cloud NetSuite Adapter !!!”