Jupyter Notebook is an open source web application for Machine Learning and Data Exploration.
In this post I will show you how to connect a Jupyter Notebook to Oracle Autonomous Database and explore the data using Python.
The assumption is we already have a Jupyter notebook sandbox running on Oracle Cloud compute instance.
Continue reading “Connecting Jupyter Notebook to Oracle Autonomous Database”
Over the past few years, in everything from personal photo storage to enterprise development there has been an incredible uptake of cloud storage buckets which provide a simple, low-cost mechanism for storing unstructured data. For much longer than that, there has been a need to consume, stage and produce unstructured data in enterprise integration scenarios. File servers accessible via protocols such as FTP/SFTP, etc. are commonly used to meet the staging requirements of traditional file-based integration use cases.
The practice of using an FTP/SFTP server alongside Oracle’s cloud integration platform – Oracle Integration Cloud (henceforth, OIC), is nothing new and is well documented.
In a series of upcoming blogs, I will explore how we can use Oracle’s cloud storage buckets (Oracle Cloud Infrastructure – Object Storage) instead of a traditional FTP server in cloud-based integrations. In this initial blog I will focus on:
- Connecting to Object Storage with Oracle Integration Cloud
- Writing Files to Object Storage with Oracle Integration Cloud
- Enforcing an archival policy for files at rest
Oracle’s two major ground breaking innovation last year were Autonomous Data warehouse (ADW) and Autonomous Database Transaction processing (ATP) both are database offering suitable for different workload and are self-driving, self-securing, and self-repairing in nature. If you want to read more about these services then please go through above links.
ADW/ATP both can be quickly provisioned on Oracle Cloud Infrastructure, it’s take less than 5 minute to spin ADW/ATP instance and database is ready to connect.
User can use Oracle SQL Developer to connect to ADW/ATP database as long as they are supported version. These DBaaS services also offers out-of-box browser based SQL Developer tool which can be used to run any kind of SQL statements.
Here is sample snap of browser based SQL Developer capabilities –
Once user has Database ready, obviously there could be requirement to access data residing inside ADW/ATP instances.
Fortunately, Oracle Integration Cloud provide Adapter for connecting ADW/ATP instance, click here to know more about ATP Adapter capabilities –
In this blog I will be covering simple steps how you can connect to ADW/ATP instances using OIC Autonomous Transaction Processing Adapter (ATP) Adapter.
I made assumption that ADW/ATP instance already exists. if you not sure how to create ADW/ATP instance then refer this blog which was written by my colleague who already explained how to create ADW/ATP database instance and connect from SQL developer.
So, let move forward. Login to your Oracle Integration Cloud (OIC) home page >> Integration >> Connection >> Create >> search for “Oracle ATP” >> select the same
Continue reading “Oracle Integration Cloud Autonomous Transaction Processing Adapter Configuration !!!”
Kubernetes is a great platform to run microservices, there is no question about it. It has great features like Horizontal Pod Autoscaler and Cluster Autoscaler that make it very easy to scale whole applications depending on current or forecasted load. However with auto-scaling there are a few considerations that we need to keep in mind and one of the most important ones is that containers are ephemeral, which implies that we need to design our applications in such a way that they can scale, without compromising data persistency. There are multiple techniques available to make this possible. A common way to achieve this, is by using Persistent Volumes (PV) and Persistent Volumes Claims (PVC), that hook via the CSI (Container Storage Interface) into external disk volumes. This helps maintain state outside containers, allowing them to scale without compromising the data.
Also, with the constant embrace of Cloud providers to kubernetes, these solutions are quickly also evolving and becoming more sophisticated and easier to use. For example, now days we can extend the use of PVC with Storage Classes, implemented by the different Cloud vendors. This make the whole PV/PVC experience so enjoyable, as these storage classes become responsible to interface into the Cloud vendor IaaS land and create resources that we simply declared, while we keep reading and writing data in persistent disks.
Now, with this constant multi-cloud endorsement with kubernetes, it was a matter of time, until Cloud vendors decided to differentiate themselves by allowing the use of foreign cloud services, as first-class citizens in kubernetes. Just imagine, having the ability to use a PaaS service from “Cloud Vendor A”, seamlessly from within my kubernetes cluster that is running on “Cloud Vendor B”. The piece of magic that makes this possible is called, Open Service Broker (OSB), which is really not magic, but just a bunch of APIs that allow the control plane in kubernetes to interact with Cloud services.
In this blog, I am going to show you how to consume Oracle Cloud Infrastructure (OCI) resources from within kubernetes using the Open Service Broker. Specifically, I am going to let my kubernetes control plane to fully manage an OCI Autonomous Transaction Processing DB (ATP), as if it was a native kubernetes resource… And by the way, I am going to use OKE (Oracle managed Kubernetes), but you could very well use Google/AWS/Azure Kubernetes elsewhere and still consume OCI resources. How cool is that?
Continue reading “OCI – Using Open Service Broker within Kubernetes to bind to Autonomous Databases”
Version 1.0.0 of the Consumer Data Right standard was released in September, and it introduces a common set of Banking APIs in line with Australian government legislation. The principles behind the standards design are very solid, though the some of the specific requirements are pretty wild and they result in a bit of rethinking of some of the classical API conventions. The most prominent example of this is the approach the CDR standards take towards ‘object identifiers’, in the ID Permanence section, and I considered the requirements for this interesting enough to spend some time thinking about and documenting.
In this context, an ‘object identifier’ refers to the way in
which you refer to an individual instance of an object from your API, such as
the ‘accountId’ in the following URI:
In this blog post we will look at what the CDR requires for these types of identifiers, and provide some sample code which implements the obfuscation requirements specified in the standard.
Continue reading “Consumer Data Right (CDR) – User-specific Identifiers for ID Permanence”
In a recent blog, I explained how to approve in Kubernetes external certificate signing requests from end users. This way, users can then simply use their private keys to authenticate into Kubernetes API server. Further to this, Role Based Access Control (RBAC) can be put in place to authorise access to resources in kubernetes clusters. This is amazing and provides some level of governance, however there is a caveat, since kubernetes does not hold neither users nor groups, the identities must exist outside the cluster somewhere else. If we create external CSRs with non-existent users and groups, soon it will become very hard to properly manage all the identities, especially if we have to maintain multiple users accessing the cluster. This is yet another area that gets highly simplified when Cloud vendors embrace kubernetes as first class citizens.
In this blog, I am going to show you how to create and manage your identities in OCI IAM and simply, using such identities to access Oracle Container Engine for Kubernetes (OKE) clusters and authorise access to resources.
In a nutshell, this is what I am going to do:
- Create a new user/group in OCI IAM
- Configure an OCI policy to grant access to my user’s group to access the OKE clusters
- Create Roles and Role Bindings (RBAC) in OKE to authorise our user to access OKE resources
- Download a kubeconfig set for my new user using token validation
- Use kubectl to access only granted resources.
Ok, let’s have fun!!!
Continue reading “OCI – Oracle Container Engine for Kubernetes (OKE) – Using RBAC with IAM”
This blog will be discussing very specific use case requirement which is more developer oriented and providing a quicker and efficient solution to invoke Oracle Integration REST API using OAuth access_token for testing purpose.
As an integration developer time to time you need to invoke Oracle Integration REST API to test API functionality. All the REST API in OIC needs a header parameter called “Authorization” which must needs to hold a valid access_token value in this format “Bearer access_token“.
In above format, “Bearer” is static world, However, access_token is the token value which we get after successfully OAuth Authentication from Oracle Identity Cloud Service. Getting access_token from IDCS using code credentials flow is multi steps and cumbersome process.
However, developer can leverage Postman environment and variable features to simplify the process of getting access_token. That’s what, I will be covering in this blog.
Before I proceed further, I must redirect you to read my colleague blog which has greater in-depth explanation about what is IDCS, how IDCS govern security aspect of all Oracle PaaS products such as Oracle Integration Cloud, Analytics Cloud, Digital Assistant and so on and OAuth client and token generation process etc. I am recommending you to read his blog because I will be using few artefacts e.g. IDCS URL, Client ID, Client Secret etc which we need to get from IDCS OAuth Client Application as per the process given in his blog.
Continue reading “Simplified OAuth Config for Oracle Integration Cloud REST API using Postman !!!”