Even in a day and age where event-based and real-time data exchange is prevalent and growing, the truth is there are still massive amounts of data exchanged using file transfer mechanisms. Oracle has always played in this realm, but with limited success. The Oracle Managed File Transfer (MFT) application is pretty good, but nothing to write home about… or write a blog about! 😀
One reason is because moving files around and using FTP servers is not very glamorous. Another reason is because the Oracle MFT management dashboard is pretty limited. It is able to monitor various aspects of a file transfer, report success or failure and allow you to resend files. But it has no business context or the ability to understand how a file might be part of a larger business process.
Integration between Oracle MFT and other products such as SOA/OSB or even Oracle Integration Cloud (OIC) is not difficult to setup. But this deals with moving and processing the files, not the management of these files in a larger business process.
I was thinking to myself.
… self… wouldn’t it be nice if I could use OIC Integration or even OIC Process to monitor multiple file transfers and other actions within a larger business process? The answer to myself… YES, that would be great!
As a target exercise, I imagined having a file transfer from system-A to system-B; which did work on the file and then returned a response file from system-B back to system-A. All this would be part of an overarching business process. This is a very common pattern and managing larger, more complex scenarios would simply be more of the same.
We need to get a handle on the “three step” process within an MFT scenario and include it into a larger business process.
I found a hook into the MFT application which allowed me to build a plugin and capture data at each of these points. They are called custom callouts and the documentation can be found here:
At first, I created a generic REST callout; which took URI user/pass and a couple of parameters. I then created a specific OIC callout where the URI user/pass details were placed in a config file on the MFT server. Either will work, but the OIC callout is simpler to use once the initial config is done. Both of these are on github:
https://github.com/gravesjohnr/OracleMFTPlugins.git
So now I have the hooks. I just need some OIC REST services to accept the data. Since the format of the three possible hooks is different, I setup three different REST services in OIC. These deal with SourcePRE, TargetPre and TargetPOST. Samples of these three data formats can be found in the github repository under samples:
https://raw.githubusercontent.com/gravesjohnr/OracleMFTPlugins/master/MFTPluginToOICApplication/MFTPluginForOIC/samples/sampleSourcePre.json
https://raw.githubusercontent.com/gravesjohnr/OracleMFTPlugins/master/MFTPluginToOICApplication/MFTPluginForOIC/samples/sampleTargetPre.json
https://raw.githubusercontent.com/gravesjohnr/OracleMFTPlugins/master/MFTPluginToOICApplication/MFTPluginForOIC/samples/sampleTargetPost.json
Now I have the OIC hooks I needed.
I was able to use the same three for both directions by using one of the parameters in the hook I called MFT-Action. Inside the OIC flow, I do a simple switch on that action to know which transfer I’m working with.
So now that I have this data in OIC, what can be done?
I had many ideas.
- Send the data to Insight
The Insight product inside OIC would be a great way to visualise the flows, timings and errors. - Send the data to Process
The process cloud is an obvious destination. This could manage the overall business flow as well as incorporate other aspects that might be involved such as human approvals and exception handling. - Send the data to Visual Builder
The Visual Builder product would enable me to build a custom dashboard to view the process however I want. This is the most flexible option of the three, but requires a bit more work.
Insight
For Insight, I created corresponding milestones and a few key fields I wanted to report on. For example, filenames and sizes, transfer times and any errors.
Just for fun, I used two methods to get data into Insight to see if I might like one over the other. One way is to use the OIC Integration “Map to Insight” tool and the other is to call APIs into Insight from Process cloud. The latter option was potentially a better option if I was going to include milestones outside this process. Of course Insight would support a mixture of the two options and still report everything correctly as long as I had a good correlation mechanism.
Process
To add things to process, I just needed to make sure the correlation ID matched for each call. The rest was quite easy.
I setup an initial step when the first file arrived and listened for additional messages at each step. Process cloud allowed me to setup human interaction in case the entire process took too long or errors were found. I created a few basic forms to handle these cases.
Visual Builder
The last destination was Visual Builder. I wanted to setup a basic data model to handle the six steps in my process. But I wanted it to be a bit smart in how things were tied together and the relationships between the data.
Once the data model was created, it automatically provided REST services to populate and update the data.
There was a bit of coding to do in OIC Integration, but it was pretty simple.
Once the data was in Visual Builder, I went to work building custom dashboards.
Results
Here are some screenshots showing some of the dashboards I was able to create after the data was made available. There is also a demo video where I walk through the scenarios.
Insight
The default dashboards in Insight were great for this scenario. I could clearly see how many processes were in a given step and how many were processed over time.
I was also able to quickly see details by clicking a given task and looking at the timeline.
I was also able to look at errors
These were some great visualisations with very little effort!
Process
I was able to get a great visual on the business process inside the standard Process Cloud Workspace application. I am able to clearly see what step a given process is on, when it completes or if there are errors.
I have a concise view when things are taking too long.
And I can see the flow in graphical and tabular formats.
Visual Builder
Visual Builder was where I was able to create a custom dashboard based on the data. I’m not a visual builder expert, but I was able to get things done and even add a few Oracle JET components to make it look pretty good. I used the oj-collapsible component to make things look clean and yet easy to drill down into more detail.
Video Demo
Conclusions
This exercise showed me a couple of things. One, that it is possible to enrich the MFT or other applications using OIC. Two, the out-of-the box visualisations in OIC are quite good and Visual Builder lets you do even more. I guess OIC is for more than just Integration! It can to Integration, Process, Visualisation, Management, Analytics, Human Interaction and more! But I guess OIPVMAHIM would be an acronym too difficult to remember!
RedThunder.Blog 