This blog is the second part of an end-to-end exercise that starts explaining the steps to clone a GitHub repository that contains an agnostic Medical Records application, built by us in NodeJS and which exposes REST API endpoints via a Swagger API-descriptor running locally on Swagger UI (all included as part of the repository). The previous part of this 2-blogs series also explains the steps required to run the MedRec NodeJS application on Docker containers either locally or in the Oracle Public Cloud. For more information about this first part, go here.
Moving to this second part, we are going to cover the following steps:
- Create an Apiary account used to Design APIs (API First approach) and create a new API Project using the existing MedRec Swagger API-definition.
We are going to spend a little bit of time playing with Apiary to feel comfortable in areas such as:
- Validating API definitions
- Testing API endpoints
- Switching across out-of-the-box Mock Servers and real Production MedRec service end-points.
Login to Oracle API Platform and configure an API, this includes:
- Enforcing Security and other policies.
- Deploy API and securing access level to on-premise and Cloud-based API Gateways.
- Publishing APIs into the API Developers Portal.
- Linking API to Apiary Swagger API-definition living document.
Login to API Developers Portal (API Catalog)
- Register a New Application
- Understanding the role of API Keys
- Reviewing MedRec API Documentation
- Registering to consume MedRec APIs
- Testing APIs.
- Understand API Analytics, consumption, metrics and monitoring dashboards.
The image below shows an easy to understand architecture with the various components mentioned above:
Let’s recap on the different components mentioned in the image above:
- API Designers (top right) initiate an API First approach by working on Apiary to Design, Validate and Test their target APIs.
API Managers (top left) continue the API Lifecycle by defining the target APIs in the API Manager Portal.
- They can link these APIs to its corresponding API definition/design document in Apiary
- They will decide what policies to enforce on these target APIs and their level of access by communities and groups
- They will decide where to deploy these target APIs. That is, on which API Gateways the target APIs will run (on-premise or in any Cloud, e.g. Oracle, AWS, Azure, Google, etc.)
- They can monitor API consumption in real-time and produce reports, invoices, etc.
API Consumers (top-centre) will login to the API Developers Portal and will only see those APIs that they have access to.
- They can register new Applications and obtain Application Keys
- They can subscribe to needed APIs (self-service registration or via an approval workflow)
- They can observe the full API documentation in Apiary and can Validate and Test their target APIs in place.
Are you ready? Let’s do it!
It is expected that you have access to the Oracle API Platform, if not, you can request a free trial at https://cloud.oracle.com/tryit
Also, it is assumed that you have already created your GitHub repository by forking from the MedRec SolutionsANZ repository and obtaining with this the full MedRec NodeJS application with its own API Swagger definition. If you have not completed these pre-requisites steps, please refer to this precursor blog.
API Design – Creating the API Definition with Apiary
In this section you are going to create a new personal Apiary account (in case you still don’t have one) and create a new API Project using the existing MedRec API Swagger definition from the MedRec SolutionsANZ repository.
If you still don’t have a personal Apiary account, follow the next steps:
- Go to https://apiary.io/
- You can click on Sign up on the top right corner or in this case we can sign up for free git GitHub, assuming that you already have a GitHub account from the previous blog.
- Enter your GitHub username and password
- Accept the authorisation that is being requested
- Confirm your name and email and click on “Continue to Apiary”
- You can create your first API Project, straightaway. Enter the name of your first API, in this case, we want to call it something like MedRec. Also make sure to click on Blueprint link below and select Swagger as the default API specification format to be used. Then, click on Save and Start Using Apiary.
You will notice that Apiary will create a default Swagger template for your MedRec API project. On the left you will see the API Swagger definition. On the right, Apiary will display a graphical representation of your Swagger API. On this graphical representation you can test and validate your API endpoints, as we will cover later.
- At this moment you can start using the left side of the Apiary tool to define your APIs using the Swagger definition. The tooling is going to provide templates and real-time validations that will guide you to quickly build an error-free swagger definition that matches your API requirements. For now, we are going to use the existing MedRec Swagger definition that comes in the MedRec GitHub repository. For this, go back to your GitHub forked MedRec (ankimedrec-apis) repository. This can be on web or locally on your hard disk if you decided to clone it locally.
- If you decide to go to your forked repository on web, search for the API Swagger definition file named “anki-medrec_prod.yml” and click on it, so that you can copy its content.
- Once you clicked on this file, you might even want to click on the edit icon (pencil) . This will make it easier to copy the whole content without weird HTML tags.
- Now, go back to your Apiary browser window and replace the whole content on the left with the content that you copied from the Git repository.
- Apiary is going to help you validate that your API Swagger definition is correct. It is very important that you confirm that there are no errors or warnings at the tope right of your Apiary page. In this case, you should see “Valid”, otherwise, it will tell you what is wrong and the place to go and fix it.
- Make sure you click Save before you get too excited and forget about it.
- If you decide to login via your GitHub account, I would recommend that you do go into your Account Settings and set a password. This will ensure that you can also login directly without using GitHub. This is crucial when linking your APIs from the API Manager Portal into Apiary documentation.
Congratulations, you have a working API Swagger definition, now let’s move on to the next section to learn how to validate the API endpoints and test them against mock services and the real service end-points.
API Design – Validating and Testing the API endpoints with Apiary
Now that you have a working API Swagger definition, you can use tooling in Apiary to help you validate the different API endpoints configured in the definition. In this section we are going to explore how Apiary can help creating mock services to quickly test and validate the viability of your APIs, even before spending time implementing them. Finally, we are going to learn how to easily switch from mock services endpoints to point to the real backend services and be able to validate and test the APIs routing to real backend services.
- In Apiary, go to your new MedRec API project and on the right, you will see a graphical representation of your API Swagger definition. Notice that it gets refreshed as you type or modify the source code area on the left.
- Scroll down and see all the different API endpoints that are configured around Physicians and Patients. There is so much you can do with these APIs. For now, let’s play with two API endpoint to create (POST) and retrieve (GET) Physicians. Click on Finds All Physicians
- Notice how you can switch across Production, Mock Server, and Debugging Proxy. For now, simply select Mock Server
Notice how the endpoint changes to something like: . That is a public endpoint and you can share that with people who need to validate these APIs.
- Click on the “Try” button. This will open a testing page.
Also notice that this API endpoint does not require any body input, it is just a GET /physicians – Simply, click on the green button “Call Resource”
- Make sure that you get a 200 Response. That means that you were able to successfully invoke Apiary mock services configured according to your API definition.
- Now analyse the response. Notice that it contains mock data for the values, but it complies with the API schema/definition structure that you designed in your Swagger definition. That means that mobile developers could be potentially already free to start hitting these APIs against a working API definition that complies with certain contract (inputs/outputs). Pretty cool, huh?
- You can go back to other API Home page and test other endpoints. When you are comfortable with the concepts that we have covered, go back to “Finds All Physicians” API endpoint, but this time, select Production instead of Mock Server. You will find this option close to the “Call Resource” button.
- Once again, notice that the endpoint changes. This time it is not directed to hit a mock service anymore, but the real backend service endpoint configured in the API Swagger definition.
Once again, this API endpoint does not require any body input, it is just a GET /physicians – However, since this time we are hitting the real backend service and since this is a public demo where we want to provide some level of multitenancy without requiring authentication, simply create a new header that is going to give you some unique space or privacy on the records that you create, so that you can only see your records and not end up in a pool of records from other people. For this, we decided to slice the backend store by using an incoming header in your API request.
- Click on the Headers area:
Then, create a new header. Call it x-app-key and try to give it a unique value that will only identify you, for example use your email.
Then, just to save some time in the future, also hove over the last options on the right of your new header and click on “pin header”. This will make sure that this header is used across your other API endpoints (you will see how that gets useful in a bit).
Once you are done, click on the green button “Call Resource”
- Make sure that you get a 200 HTTP response code. That means, it all went well. Then, scroll down and review the response payload. Notice that this time it is empty, that is assuming that you or anyone else has not used that unique header value to create physicians. If you already see records and you shouldn’t, feel free to go back to the input header and choose something a bit more unique, then click on “Call Resource” again.
- Ok, now let’s go back and create a couple of Physicians using the “Adds new Physicians” (POST Physicians) API endpoint.
- Notice that the x-app-key header is already set, that’s because you pinned it before, which is good as you don’t have to set it again. However, since this is a POST operation, you have to setup the Request Body Payload that contains the body of the Physician that you want to create. For this, click on Body.
Notice that Apiary will already pre-populate the request payload as best as possible using mock data. This is based on the API definition/schema structure that you configured in your Swagger definition. At any time, you can modify it on the left area, save your API definition and you will see how this request body changes. That’s cool.
Modify the mock values to create a few new Physician, enter whatever names and specialisation that you prefer. When you are done, click “Call Resource“
- Confirm that you get a nice 200 HTTP code response and that you get echoed your request with some _ids, that basically means that it all worked
- However, to be sure, go back and retrieve all physicians again. This time, the list should not be empty, it should be populated with the records that you created.
- Congratulations, you just learnt how to interact with some of the key Apiary tooling to simplify the design of APIs. In the next section, we are going to learn how to define, secure and deploy APIs into the Oracle API Gateways. We will also publish these APIs into the API Developers Portal (API Catalog) and hooked them with this Apiary Documentation.
API Management – Configuration, Policy enforcement and Deployment
In this section we are going to login as API Managers into the API Manager Portal and configure the API that will be running at the API Gateway, with the required policy enforcement. We are also going to learn how to publish APIs into the API Catalog, accessible via the API Developers Portal with the right access level, so that APIs can only be accessible by the right communities.
If you still don’t have access to Oracle API Platform, simply go to https://cloud.oracle.com/tryit and request a new trial, they are free. Otherwise if you are an internal Oracle employee, you can request an API Platform environment via GSE UCM with Classic.
- Login to the API Manager Portal, it will be something like: https://[IP]/apiplatform
- If this is the first time that you login, you will not see any API being created, otherwise all the APIs that you have access with your username will appear. We want to create a new API configuration for MedRec, so, simply click on Create API button.
- Give it a good name, version and description. Then click Create.
- Click on your new API to configure it.
- Hove over the API Request area and click the Edit button that appears on the right end. This is to define the API external endpoint that you want your end-users to follow to get access to your APIs
Enter some Comments if necessary and select the Protocol that you want, it can be HTTP, HTTPS or both. Then enter the API Endpoint URL that you want your consumers to use to access your APIs. You can make it very specific or generic.
For example, if you make it very specific, you can define completely an API endpoint, such as: /api/medrec/patient/[id]/history – This way, for example, only when the full endpoint uri is invoked, that’s when this API will grab it. Otherwise, like in this case, you can define a more generic base URI that you want to grab any sub-request, for example: /api/medrec – This time any request that starts with /api/medrec will be attended by this API configuration, including: /api/medrec/patients, /api/medrec/physicians, /api/medrec/patient/[id]/history/20017/march/15 – There is not right or wrong approach, it basically depends whether you want to apply the same policies to a bunch of API requests or if you need to apply fine grained level set of policies to different APIs. It’s up to you.
Then click Apply.
Now, click Edit on Service Request. This is to define the back-end service endpoint that this API will point to when invoked.
In the Service Request window, you can either select an existing service or enter a URL. The service comes handy as you can define your commonly used services and simply reuse the endpoints when definition your APIs. In this case, for simplicity reasons we are going to enter the URL directly. Select “Enter a URL” option and enter the location of your backend service, in this case, it is the MedRec application that we have been working on and that we deployed since the previous blog.
Also notice how you can define a Service Account at the bottom of this window. This is to define credentials to get access to the backend service. In our case we don’t need to configure them as the MedRec service is anonymous and does not require authentication for simplicity reasons.
When you are done, click Apply.
Make sure to Save all Changes by clicking on the top right “Save Changes” button.
We still need to attach security policies to our API, but for now, let’s just deploy it and see if it works well. On the left vertical menu, click on the Deploy option.
- Click on Deploy API button. A list will appear with the API Gateways that you have deployed (whether locally on-premise hardware or in any cloud provider) and that your account has access to. In this case I can only see Production Gateway, which is the one I want to deploy my API into.
- Once you have selected your API Gateway to deploy your API into, click on Deploy. After a few seconds you should see that your API goes from status Waiting into Deployed.
That’s it, your API is running on the API Gateway. Let’s test it. Use your API Test tool of choice to test your APIs. I personally use Postman from Google. It is pretty good, but you can use whatever you want.
In your API test tool of choice, create a new test using the endpoint that was give to you at the moment of deploying your API. In my case it was something like: https://[domain]/api/crimedrec – Then to easily test you new API running in the API Gateway.
Let’s try to retrieve (GET) the list of physicians that we inserted earlier in this tutorial. Just try to remember the value that you entered as part of your x-app-key header and use it again… All your data should still be available.
The complete set of settings that you need to configure are:
- Method: GET
- URL: https://[domain]/[URI_Request]/physicians – Adjust accordingly to whatever you configured in your API Manager Portal.
- Header: x-app-key -> set to the value that you used previously. If you followed the instructions, it was your email.
- Assuming you used the same x-app-key that you defined when creating (POST) the physicians in Apiary, you should get all them once again.
Congratulations. Your API is working properly. Now let’s continue and enforce some security policies.
- In order to add some policies, go back to the API Manager Portal and go inside your MedRec API again. Make sure you are in the “API Implementation” tab.
- On the right-hand menu, you can see all the Available Policies that you can attach to your Request and Response. Still at the Request tab, expand Traffic Management and hove over the API Throttling and click Apply.
Add some comments if you like and then click on the Next arrow.
Notice that you can configure multiple conditions. For example, here I am saying to throttle by 300ms if there are more than 100 requests per hour. When done, click Apply.
Similarly, you can add any sort of out-of-the-box policies, around Security, Traffic Management, Interface Management, Routing or create any type of custom Policy based on a rich SDK. For example, notice that you can rate limit based on Applications or even across applications based on the overall API consumption.
Let’s add another one, this time let’s add API Key Validation, within Security
- Key Validation allows you to ensure that only API resources can be accessible to requests using authorised keys. This also ensures that such API consumption can be fully monitored and managed. For now, let’s add a new header-based key validation. In order to avoid clashing with the MedRec APIs, let’s called something like my-header-validation. When done, click on Apply.
- As of now, you should have 2 policies configured in your API Request.
As we did previously, save your work and re-deploy your API so that we can test it.
- After redeployment, make sure your API moved from Waiting state into Deployed.
- Now, using your REST API tool of preference re-test your endpoint as you did previously. What happened? It worked previously? How come no we are getting a 401 Unauthorized HTTP error message?
- Well, as you might be guessing, you have enforced a header-based Key Validation policy. So we have to add header my-header-validation, but using what key? For that we need to register an application to consume our API. That will be covered in the next section.
API Management – Subscription and Consumption
In order to be able to register to use our MedRec API, first we need to publish it and make it available via the API Catalog that lives within the API Developers Portal. So, let’s start by publishing our API.
- Similarly, as we deployed the API in the API Manager Portal, we are going to publish it, but this time click on the next option called “Publication“.
- Now, click on [Enter Portal Name] and type in the way you want to name your API within the API Catalog.
- In my case I just gave it the same name as the API. When done, click the tick button.
Make sure to Save all your changes.
- Then, click on “Publish to Portal”.
- Congratulations, you just published your MedRec API.
Next, we could link this API to the Apiary documentation that we created previously. This would allow that the same level of Apiary tooling that we experienced earlier, can be available to test and validate your MedRec API during the API Catalog (within API Developers Portal).
Note: From recent changes in the Oracle API Platform, only Apiary Team accounts can be used as links from the API Manager Portal. As far as I understand, in cases where customers buy API Platform, Apiary comes included as part of the API Platform. In that case you should use your Apiary account to host your API Definitions.
- Assuming, you are using an Enterprise Apiary account, click on the Apiary button.
Then, click on Continue to Sign in. It will then ask you for your username and password.
- Find the API that you want to link from Apiary. Then, click Connect.
Save your changes and click redeploy project to ensure the latest link goes into the API Catalog within the API Developers Portal.
- Notice that you can adjust the Granting privileges of who will have access to your API in the Developers Portal. For that, simply go to the next option on the left menu “Grants“.
Notice all the different ways you can grant control access to your API, for example some important access worth mentioning are:
- Manager API: Select the user or group of users who can Manage this API, for example to control its life-cycle.
- Register: Select the user or group of users who will be able to subscribe to use your API without further approval.
Request registration: Select the user or group of users who will be able to request subscription access, but then, it’s up to the API Manager to approve it or reject it.
Make sure you add the right privileges to the user or group of users that you want to use to login to the API Developers Portal. Then if you change any privileges in the API Manager Portal, make sure to Save your work and re-deploy.
- Now open a new browser window and go to the API Developers Portal. It will be something like: https://[server]/developers – Use your username and password that will give access to the API that you just published above.
- Once you login, on the APIs tab, you can see all the APIs that your username has access to. In this case at least you should see your API, otherwise it means that the Grant access needs to be tweaked.
- APIs are subscribed via Applications. For this, we need to go into My Applications tab and create a new Application. That is going to give us the Application Key that we desperately need to access our API once again.
- Fill in the blanks with information about your Application that will be used to consume your MedRec API. When done, click on Save.
- Search for your new Application and click on it. You will then be able to see the Key.
Now, what is needed is to use this Application to subscribe to consume our MedRec API. Click on the APIs tab and click on the MedRec API.
Click on the Subscribe top right blue button.
- Select your Application on the left and click on Subscribe API.
- A message will appear saying that your Application has subscribed to consume this API.
- Click on your Application again.
- Copy your Application key.
Go back to your REST API tool and create a new header with the name that you chose before e.g. my-header-validation and paste the Application key as the value and test it again.
You will notice that this time it will work and will return the expected response.
Congratulations your request now complies with your policy enforcement!!!
Finally, notice that back to the API Developer Portal – API Catalog, when you click on your MedRec API, you can see below the Apiary documentation as you were able to do before. Feel free to play with it and launch a few tests against Mock Services and Production end-point as you did before. As you have probably realised, the API Definition is governing the API Catalog, which means that if the API definition changes, it will be always in-sync to the documentation visible via the API Catalog. How cool is that?
API Management – Analytics
The last area that I want to show you is around API Consumption Analytics.
- Login to the API Manager Portal and go to Analytics, which is the last tab on the left menu. Feel free to browse across the different charts and graphs.
In here you can see a lot of out-of-the-box analytics that are surfaced into the Web UI.
Out of the box, the API Platform provides rich real-time and performance related metrics, including:
- How, when and why APIs are being used,
- How often and why requests are successful, errored or rejected,
- Monitor real-time data trends,
- Request volumes, response times, request/response failure rates,
- Summary information for a specific period (time controls slicing),
- Request/response payload size chart analysis,
- Error-rate charts, error distribution charts, gateway analytics charts:
- Policy Rejection Rate (Inbound/Outbound, By Policy)
- Policy Rejection Distribution
- Error Rate (Service Errors, Policy Errors, By Error)
- Error Distribution
- Monitoring filter by API resources (volumes, percentages, projections, etc.),
- Subscriber/Application based charts:
- The volume of requests sent from each application.
- The percentage of requests sent from each application.
- The volume of rejected requests sent from each application.
- The percentage of rejected requests sent from each application.
- The error volume for requests sent from each application.
- The error percentage for requests sent from each application.
- Rich filtering-based analytics,
Also bear in mind that the API Platform comes with system APIs that allows fine grained level scrutiny, way deeper than what the Web UI offers. This allows creating very sophisticated automated capabilities to slice and dice API consumption, producing perhaps a nice invoice at the end of the month.
I hope you found this blog useful. If you encounter any question or problem in this tutorial or if you have further comments or areas that you would like to discuss, feel free to contact me via LinkedIn (https://www.linkedin.com/in/citurria/).
Thanks for your time.