Field Level Encryption with Oracle Integration and OCI Vault

Integration platforms are often required to handle confidential information such as personal details, payment information or other data protected by compliance and regulatory standards such as HIPAA, GDPR, PII and PCI.

Various methods exist to protect data from unauthorized access while data is in transit and at rest. These approaches typically encrypt the entire payload. As a complementary approach Field Level Encryption has an important role to play by ensuring that only appropriately configured clients can read sensitive data fields. This approach also allows clients without the encryption keys to work with the non-sensitive data which would be impossible to do with a fully encrypted payload.

Although Field Level Encryption (FLE) is not natively supported in Oracle Integration (OIC) today, this blog will explore several options that will allow you to implement FLE with OIC. In this blog, I will present these options, discuss some guiding principles and showcase some sample implementations.

In the context of Oracle Integration, an implementation of Field Level Encryption should allow a developer to easily encrypt/decrypt individual field(s) as part of an integration flow. Let’s explore several use cases where this may be applicable:   

• Oracle Integration may be required to receive or send encrypted information to other systems as part of a bigger data payload. For instance: OIC developers may be required to encrypt some but not all fields for a new hire sourced from Oracle HCM prior to sending them to an external system.
• A Visual Builder application that is capturing some sensitive data may need to call an OIC integration to encrypt this data before passing it on to the target application or storing it in the underlying database. 
• An API created in OIC that exposes data from a backend system may need to also expose a sensitive data field for certain clients who have the related key to decrypt this data.

When implementing Field Level Encryption we need to be aware that we are working with sensitive data (the data fields which need encrypting but also the encryption keys themselves). As such, the following need to be considered:

• All keys used for encryption/decryption should be stored and managed appropriately through the use of a dedicated Key Management System (KMS). My recommendation is that Oracle Integration customers use OCI Vault for this. OCI Vault is a cloud native service that allows customers to securely store and manage their encryption keys and sensitive configuration information. The OCI Vault service supports several key encryption algorithms such as the Advanced Encryption Standard (AES), the Rivest-Shamir-Adleman (RSA) algorithm, and the elliptic curve digital signature algorithm (ECDSA). For more information about the service, please check out the official documentation which is available here. Existing Oracle Integration customers should know that the OCI Vault service is priced separately (though the first 20 key versions are free!).

• As encryption in OIC occurs within an integration flow, payload logging needs to be disabled. 

---

High-Level Approach:

When deciding on a field level encryption approach, you may already have your requirements clearly defined. If not; this is a good time to think about the following questions which may guide you to the approach that is right for you:

• Do you want to use symmetric or asymmetric encryption?
• Is your preferred encryption algorithm supported natively by OCI Vault (refer to the documentation here)?  
• What experience are you expecting your OIC developers to have in regards to encryption?
• If you need to write some code to meet your requirements, what language would you feel comfortable with and what is your organization setup to support?
• Are the OIC integration(s) that will be responsible for encryption/decryption sensitive to response times?

Here are a few of the possible approaches to implement field level encryption with OIC and my views on when you should use these:

• OPT1: Encryption using the OCI Vault native encrypt/decrypt APIs
  • Use when:
    • You can use one of the encryption algorithms that is natively supported by OCI Vault (e.g AES 256 GCM)
    • You do not need to encrypt more than a handful of fields in the same integration flow

• OPT2: Encryption using ephemeral Data Encryption Keys and an OIC JavaScript function
  • Use when:
    • You want to create a native function in Oracle Integration that can be re-used across integrations by one or more developers.
    • You are comfortable with using JavaScript(JS) and related crypto libraries (JSCrypto)
    • You have encryption requirements that are not addressed by the default encrypt/decrypt OCI Vault API (e.g. your requirement is to use AES_128_CBC or similar)

• OPT3: Encryption using Keys from OCI Vault and OCI Functions
  • Use when:
    • You prefer to use a language other than JS (e.g. Python, Java) and their associated crypto libraries
    • You have a short running function that is suitable to the server-less architecture of the Oracle Functions service
    • When function warm times are not going to impact your flow (batch or async style integrations)

Implementation:

The below section aims to present a simple implementation of each of the above patterns. I hope to give you just enough here so that you could take the idea and apply it to your specific use case. I have decided to use symmetric key encryption based on the AES standard for my implementations as symmetric key encryption is somewhat easier to understand and demonstrate.

Asymmetric key encryption is not explored in depth here, though if you have requirements to use asymmetric keys, I would recommend leveraging a similar approach to OPT2 – generating a random Data Encryption Key (DEK), using it to encrypt the sensitive fields using AES, then encrypting the Data Encryption Key using the public key for transmission, so the recipient can then use the private key to decrypt the Data Encryption Key, and then decrypt the data in turn. Unfortunately OCI Vault doesn’t support generation of a DEK using asymmetric keys, so that will need to be done externally (however it can support encrypting and decrypting of the generated DEK).

OPT1: Encryption using the OCI Vault native encrypt/decrypt APIs

This is perhaps the easiest option to implement as it relies on leveraging a RESTfull API already exposed by OCI Vault to do the encryption and decryption for you. To leverage this option you will need to ensure that you can use one of the default encryption algorithms already supported by the OCI Vault service. It is also worth noting that this approach is appropriate for cases where you don’t need to encrypt too many fields as you will need to call the above APIs for each field you need to obfuscate.

Pre-requisites:

1. Create an AES 256 bit key in OCI Vault OR Alternatively import your own AES Key (using the key wrapping service)
2. Create a REST connection to your OCI Vault Crypto endpoint  in Oracle Integration

For details on the above steps see the appendix at the end of this blog.

To demonstrate this approach, I have created two OIC integrations; one that will encrypt a plaintext payload and another that will receive the encrypted message and decrypt it:

Note: if you find the images to small to read, feel free to click on them to zoom in.

Let’s take a look at the encrypt flow in a little more detail:

In the above flow I have created a simple OIC integration which will receive a request containing a plaintext value that will need to be encrypted. OIC will then make a call out to the OCI Vault Encrypt API (see above) passing in the OCID of the Master Encryption Key and the plaintext value (base64encoded). The response from the OCI Vault API contains a ciphertext element which captures the encrypted value of our plaintext string. Remember that while the plaintext value is passed to OCI Vault; it is done so over TLS so the entire payload is encrypted in transit. Once we receive the encrypted value we can pass it to the downstream system/ application. In my case this is another OIC flow, as I also want to show you the decryption side.

On the decryption side the flow looks like this:

The above integration simply receives the encrypted payload and then calls the OCI Vault service passing in the encrypted payload and the OCID of the Master Encryption Key. OCI Vault returns a base64 encoded plaintext value which we can easily decode in OIC and extract the actual plaintext value.

I love the simplicity of this approach and it works well when we only need to do field level encryption for a small number of fields. However if there are many fields to encrypt/decrypt then performance may become an issue due to the large number of requests that we would need to make to the OCI Vault service which currently does not offer a batch encrypt API.

OPT2: Encryption using ephemeral Data Encryption Keys and an OIC Javascript function

In this pattern, I will use a custom JavaScript function to do the actual encryption/decryption however I also want to ensure that I store the Master Encryption Key in OCI Vault. To achieve this I will use the following high-level flow for encryption:

1. OIC will make a call to the OCI Vault service to generate a Data Encryption Key (DEK)
2. A custom JavaScript function is implemented in OIC that will accept the DEK as input in addition to the plaintext value and return the encrypted value (see the appendix at the end of this blog for more on this)
3. The encrypted text and encrypted DEK will then be passed to the target application

And to decrypt our application (in this case OIC) will need to:

1. Call the OCI Vault Decrypt API passing in the encrypted DEK and retrieving the decrypted DEK.
2. Implement a JavaScript Decrypt function that will receive the decrypted DEK as an input in addition to the ciphertext and return the decrypted/ plaintext value.

You may be slightly confused here, but bear with me. Let’s go over the sample flow and hopefully it will be much clearer.

Pre-requisites:

1. Create an AES 256 bit key in OCI Vault OR Alternatively import your own AES Key (using the key wrapping service)
2. Create a REST connection to your OCI Vault Crypto endpoint  in Oracle Integration
3. Create and import a JavaScript library and dependant crypto libraries to perform the encryption/decryption in OIC

For details on the above steps see the appendix at the end of this blog.

To demonstrate this approach, I have once again created two OIC integrations that will demonstrate encryption/decryption using a local JS library:

The encryption integration flow is captured below:

There a couple of things to highlight here. Firstly I am using the generateDataEncryptionKey API to generate a Data Encryption Key for every instance of the above integration. This Data Encryption Key is effectively a onetime use key (or ‘ephemeral key’). You can see above that I can actually define the key shape of the DEK (specifying size and algorithm) in the request to OCI Vault. The generated DEK is returned to OIC and in my case I am getting its plaintext and encrypted format. Note the ciphertext DEK is encrypted with the Master Encryption Key which we created in OCI Vault earlier.

In step 3 of the above diagram, you can see that the JavaScript function which I uploaded earlier is available in the mapper under User Defined functions. This offers the best experience as OIC developers can use it just like any other native xslt function. In addition to usability there is a point to be made here about re-use. In the one mapper I can reuse the function to encrypt many different fields. This pattern of a single call to OCI Vault and a native encrypt function, offers in my opinion the best user experience when you have requirements to encrypt multiple fields or to do encryption/decryption across multiple integrations.

With the data now encrypted we can make a call to the target (in this case another OIC integration). You may be able to see from the above that here we are also passing the encrypted DEK. This is because we encrypted our data with an ephemeral key, which will be needed by the other side in order to decrypt the data.

Now let’s have a look at the decryption side:

On this side the integration receives the encrypted data as well as the encrypted DEK and makes a call to the OCI Vault decrypt API passing the encrypted DEK and the reference (OCID) of the Master Encryption Key. The OCI Vault service returns the plaintext value of the one-time Data Encryption Key. OIC then calls the uploaded JavaScript function (decryptAES) passing in the plaintext DEK and encrypted data and retrieving the plaintext data value. Again all interaction with the JavaScript function is native. In the above example you can see that we are referencing the decryptAES function in an assign activity.

There we have it … a JavaScript approach to do AES encryption/decryption whilst leveraging the OCI Vault service for key management.

Note as the above flow requires us to pass an encrypted DEK this pattern will need to be applied in appropriate scenarios where there is a handoff from one integration layer to another or to a system which can be configured to use OCI Vault or its own KMS to store the symmetric Master Encryption Key.

A variation of the above approach is NOT to use dynamic keys but rather simply retrieve the symmetric key (stored in OCI Vault as a key or secret) in encrypt/decrypt integrations prior to invoking your JavaScript functions. This would mean that you no longer need to pass the DEK to your target endpoint and would still give you great performance although it would not be as secure as the dynamic key approach, discussed above.

Note: see the appendix at the end of the blog for more information on deploying the encrypt/decrypt JavaScript function in OIC.

OPT3: Encryption using OCI Vault and OCI Functions

In this pattern I will leverage Oracle Functions to hold my encryption & decryption logic. If you are not familiar with the concept of Oracle Functions, please have a read of this article. Oracle Functions provide the flexibility to implement custom encryption / decryption scenarios based on the code that a developer chooses to implement. With this approach I will once again recommend the use of the OCI Vault service to store the Master Encryption Key. To expose the encryption/decryption function(s) for OIC consumption I will also use the OCI API Gateway.

Therefore from an encryption/decryption flow point of view we will see the following steps:

1. OIC will make a call to the API Gateway  (via its REST adaptor)
2. The OCI API Gateway will enforce authentication and provide a route to our backend crypto functions
3. A dedicated Oracle Function for each operation (encrypt/decrypt) will execute the custom code which should in turn leverage the Master Encryption Key stored in OCI Vault.
4. The function output (encrypted/decrypted string) will be returned to OIC

Pre-requisites:

1. Create an AES 256 bit key in OCI Vault OR Alternatively import your own AES Key (using the key wrapping service)
2. Create your custom encryption function(s) that leverage OCI Vault Keys and implements your desired encryption/decryption logic.
3. Deploy Function(s) to the API GW defining your desired authorization policy.
4. Create required OCI IAM policies allowing API Gateway to access your functions and allowing your functions to access the OCI Vault service
5. Create a REST connection to your API Gateway in Oracle Integration

For details on the above steps see the appendix at the end of this blog.

Let’s take a quick look at the OCI artefacts:

In the API Gateway I have created an AES API with a /crypto endpoint:

I have also configured two backend routes that will surface my respective functions through the /decrypt and /encrypt resource path:

From an Oracle Functions point of view I have created two functions (encrypt & decrypt) under the same application:

In my implementation I have decided to use Python to create these functions, but you can use whatever language you feel most comfortable with. As I did not have any specific encryption requirements, I also chose to leverage the encrypt/decrypt API exposed via the OCI Vault service through the OCI Python SDK. You can find the OCI SDK reference documentation here.

One final note; as my Oracle Function references the cryptographic endpoint & master encryption key OCID I decided to also add these as configuration variables in my function application; this way all OIC needs to pass to the encryption/decryption function is the plaintext or encrypted value. 

With the functions created and exposed via the API Gateway, I can now go to OIC and configure a connection to the Gateway endpoint. To do this, I have defined a REST connection using the URL or hostname of my public API Gateway.

To test this last flow, I have once again created two integrations:

My encryption integration will again receive a plain text request via the REST adaptor but this time will invoke the API Gateway service which in-turn will route to the AES encrypt function hosted on Oracle Functions. The response from this function will be the encrypted field which will subsequently be passed on to our decrypt integration: 

In the decrypt flow, OIC will need to call the corresponding decrypt function to extract the plaintext value before using or passing it downstream.  

Pretty simple from an OIC point of view right? But watch out for those function warm times. If your flow is not synchronous in nature and you don’t mind looking after a few OCI components this is a very good option. The only other downside of this approach is that if you have multiple fields that you need to encrypt it can become somewhat chatty. In this case you may want to create your function in such a way that allows you to handle multiple elements at the same time (effectively creating a batch API to encrypt multiple individual fields).

Summary

In this blog I set out to demonstrated some high-level options that will allow you to meet field level encryption requirements with Oracle Integration and OCI Vault. I have presented my thoughts on when each approach should be used and summarised my implementation experience. Although these options may not be exhaustive I hope that i have given you enough to feel confident in meeting field level encryption requirements with Oracle Integration.

As a parting note I wanted to acknowledge the invaluable contributions of my friend and colleague @callanhp in writing this article.

Appendix

Creating an OCI Vault & Master Encryption Key (Applicable to OPT 1, 2 & 3):

Regardless of the approach you decided to take, one common step that you will need to do based on my recommendations is to store your encryption key in OCI Vault.

To do this you will need to log into Oracle Cloud Infrastructure and create a Vault. You can find the specific instructions on how to do this here. Once the vault is created, the service will present you with a cryptographic and a management endpoint. Copy the cryptographic endpoint as you will need to reference it later in OIC.

Finally go ahead and create or import a master encryption key. You can find the specific steps to do this here.

Creating a connection to  OCI Vault in OIC (Applicable to OPT 1 & 2):

If you are planning to call the OCI Vault service directly from OIC (applicable to option 1 and 2 above) then we will need to create an OIC connection for it. To do this, head over to Oracle Integration and create a REST connection with the following details:

Connection URL: <Cryptographic Endpoint from OCI Vault>

Security Policy: <OCI Signature Version 1>

If you need help configuring the details of the OCI Signature Policy, refer to this documentation.

Creating and importing a JavaScript library and dependant crypto libraries (Applicable to OPT 2):

If you have decided to go ahead and implement your encryption/decryption function as a JavaScript library within OIC then you will also need to bring in your JS script plus any dependent libraries such as CryptoJS.

In a previous blog by Harish, we demonstrated how JavaScript libraries can be imported and used in Oracle Integration.

In this specific case I created a simple JS application with 2 functions (encrypt and decrypt) and leveraged symmetric encryption using AES via the CryptoJS library. As my functions were dependant on CryptoJS I ended up bundling my JavaScript application and the dependant libraries into a single JAR. Effectively this is just a process of selecting the required JS files, archiving them to a zip file and then renaming the archive file to a JAR.

The imported JavaScript libraries are then displayed and you can select the functions that you want to make available in OIC:

It is important to note that the contents of your JavaScript application can vary to support your particular use case and cryptographic requirements.

In my case I have defined 2 inputs for my encryption function (the plain text message and the key) and a single output which will contain the encrypted value. Likewise decryption function does the reverse, accepting (the encrypted message and the key) as input and returns the decrypted/plain text value.

After giving this some thought, i decided against providing my JS function code in this blog. After all, I set out to show the high-level approach or pattern not the specific cryptographic implementation of my function (besides this will usually vary depending on requirements). However if you would like me to share my iars or function code with you to help you get started, please reach out to me.

Creating a connection to an OCI Function (Applicable to OPT 3):

If you have chosen to leverage Oracle Functions for your encryption/decryption requirements you will need to:

1. Create your encryption/decryption functions (ensure that you are leveraging the OCI Vault service within your function to store your encryption key).
   • If you would like some help with this, please see the Oracle Functions Sample libraries available here. There are several sample functions that will show you how to interact with OCI Vault.
   • Once again, also happy to share my iars and OCI Function code
2. Deploy them to Oracle Cloud Infrastructure (Oracle Functions).
   • There are a number of ways to get started with Oracle Functions but I would recommend the use the OCI Cloud Shell here. You can find detailed steps on this here. 
3. Expose your deployed function via the OCI API Gateway and add desired policies (authorization/ rate limiting, etc) as needed.
   • This article documents this process in detail
4. Add your functions to a dynamic group and create a policy which allows it to use keys from the OCI Vault service
   • To do this you can follow the instructions here.
5. Add a policy that will allow the API Gateway service to use your functions.
   • See here for detailed steps.
6. Test your functions with and without the API Gateway
7. Create a connection to the API Gateway in OIC (this can be done via the REST adaptor and is shown above)

I hope you guys have enjoyed this article. Feel free to reach out to me via Linked in and share your thoughts.