Exploring GitHub Docker Hub and OCCS Part 4

In my previous post in this series I covered linking GitHub and DockerHub and configuring the environment such that a build of a Docker image was triggered on updates to GitHub. In this final post of the series I will take you through the steps to pull the image from Docker Hub into OCCS in order to run the application. It should be noted that the image built on Docker Hub in my example is only the web tier that contains my Node.js project (APIs and SwaggerUI). The MongoDB component of my OCCS Stack is pulled directly from Docker Hub when my Stack containing the Web Tier and Database Tier services is deployed to OCCS. Continue reading “Exploring GitHub Docker Hub and OCCS Part 4”

Exploring GitHub, DockerHub and OCCS Part 2

In my previous post I detailed how I Dockerised the MedRec app. In this post I will show how I added MongoDB and defined a stack using Docker-Compose.

Add MongoDB layer using Docker-Compose

According to the official docker documentation ;

“Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a Compose file to configure your application’s services. Then, using a single command, you create and start all the services from your configuration. ”

A single command to create and start all the services in a configuration sounded pretty good to me. I definitely was keen on exploring docker-compose.

Add a docker-compose.yml file

Having proved that my web application runs up, I now need to address the persistence layer. The above Dockerfile contains the steps to create the required runtime platform for my node app, and installs the node application and package dependences (as specified in the package.json) file by doing the npm install. However if I tried to do a GET or a PUT my app will fail as it won’t find a MongoDB inside my container. I therefore still need a MongoDB somewhere in my environment to hold my application data. Continue reading “Exploring GitHub, DockerHub and OCCS Part 2”

Exploring GitHub Docker Hub and OCCS Part 1

In my previous post in this series I provided an Introduction describing the high level steps I planned to take.
In this post I will walk through the detailed steps to Dockerise the MedRec application.

Dockerise the MedRec APIs

Git clone the project repository

I used my Windows Surface Pro-4 with Oracle VM Virtual Box installed to host my development VM. I managed to source a VBox image that already had Ubuntu 16.04 and Docker installed so that helped get me started. In my development environment on my laptop, I created a directory under my home directory named gitprojects.
I cd into that directory.

cd gitprojects
Continue reading “Exploring GitHub Docker Hub and OCCS Part 1”

Exploring GitHub, DockerHub and OCCS – Introduction

As part of our MedRec API playpen initiative we had already developed some REST API’s using Node.js and leveraged MongoDB running on Oracle IaaS as the persistence layer. . This post describes what I did to dockerise the MedRec API application and eventually run it on the Oracle Container Cloud Service (OCCS).

The APIs have already been made available for interested parties to interact with via SwaggerUI. Of course developers could develop their own code (or any REST client) to interact with them. As a team we used a combination of the Oracle Developer Cloud Service (Git repository, Issue Tracker, Build Server etc) and also the public GitHub to provide public access to our project code. As the source code for the Node.js project containing the API’s was pushed to GitHub I simply did a clone of the Node application code in order to download and run it locally (MedRec API tutorials available here).

The application ran well enough on my local laptop which running Ubuntu 16.04, however I really wanted to be able to try to run the app and MongoDB as a Docker image/stack on my laptop. After I had the application successfully “Dockerised”, then I planned to deploy my application stack to the Oracle Container Cloud Service. I also wanted to explore the use of the GitHub / Docker Hub integration to build my image on Docker Hub, and then from within the Oracle Container Cloud Service (OCCS). With the application image available on Docker Hub, I could then pull my image from that source in order to run it up on OCCS.

A good blog can really help bring you up to speed quickly and help overcome inertia to get you started and I would like to acknowledge the help that Mauricio Payetta’s blog provided me.

In this series of blog posts I plan to retrace what I did during my self-learning. Continue reading “Exploring GitHub, DockerHub and OCCS – Introduction”

Teaching How to simplify building NodeJS APIs with Loopback Framework

In this blog, I am going to show you how to get started with the Loopback framework to easily auto-build REST APIs in NodeJS and persistence layer in a variety of options, including relational and non-relational databases e.g. In-memory DB, MongoDB, MySQL, Cassandra, Oracle, etc.

In terms of API design and development, Loopback allows you to work “top-down” or “bottom-up”. I am going to cover both approaches in this blog.

First, we are going to create an API model definition in place, as we are building the REST APIs, this exercise will give us a Swagger-based API definition. Alternatively, we are going to start from an existing Swagger definition and use it to implement NodeJS REST APIs pointing to a persistence layer of choice (in-memory DB, MongoDB, MySQL, DB2, Oracle, etc.). I personally prefer the “API First/Top Down” approach, as it gives me the option to properly design and test my APIs first and then, simply move to implementation phase, but this ultimately depends on situations, preferences and requirements.

Continue reading “Teaching How to simplify building NodeJS APIs with Loopback Framework”

Teaching How to use Nginx to frontend your backend services with Trusted CA certificates on HTTPS

Now days with the adoption of Serverless architectures, microservices are becoming a great way to breakdown problem into smaller pieces. One situation that is common to find, is multiple backend services running on technologies like NodeJS, Python, Go, etc. that need to be accessible via HTTPS. It is possible to enable these internal microservices directly with SSL over HTTPS, but a cleaner approach is to use a reverse proxy that front ends these microservices and provides a single HTTPS access channel, allowing a simple internal routing.

In this blog, I am showing how simple it is to create this front end with Nginx and leveraging “Let’s encrypt” to generate trusted certificates attached to it, with strong security policies, so that our website can score an A+ on cryptographic SSL tests conducted by third party organizations.

Continue reading “Teaching How to use Nginx to frontend your backend services with Trusted CA certificates on HTTPS”

Teaching how to use MongoDB and expose it via NodeJS APIs

 

Hi, we are starting to use different technologies to build complex scenarios around APIs. It’s getting common to use popular NoSQL DBs like MongoDB.

For this reason, I decided to build this very easy to follow blog that will help you get started with MongoDB and build simple REST APIs using NodeJS via the Express module.

The use case is simple, we are going to build a MongoDB to store users information that eventually (in a future blog) we are going to use to send SMS and Voice call notifications… But for now we are keeping things simple, we are starting by demystifying the use of MongoDB with NodeJS.

The code that sends the SMS/Voice call notifications in NodeJS via Twilio is out of the scope of this blog, but if you want to use it, you can find it here.

Pre-requisites

 

This blog is about simplicity, so we are going to build a simple HelloWorld sample that starts from scratch and interacts (GET and POST) with a MongoDB via Express APIs.

 

Installing and playing with MongoDB

 

In this section, I am going to show you how to install and get your MongoDB up and running. This is going to be the DB that we are going to use in a future blog to send SMS and voice call notifications to people.

Note: I am using Ubuntu, adjust if using other OS (e.g. yum if using OEL/RH):

sudo apt-get install mongodb

  • Validate the installation by running mongo client – This should connect to the running MongoDB Server

  • In the mongo client prompt, create a simple test database called myTestDB:

    use myTestDB


  • The database doesn’t really exist until we add some data. The best thing of MongoDB is that you store JSON payloads into it. This makes it very friendly when interacting with APIs.

     

    In our case, we want to build a notification service and we want to store the contact details of the recipients. Something like:

     

    {“name”:”Carlos”, “mobile”:”+615550505″, “msg”:”Hello World”}

     

    Note that we are not defining ids, mongoDB will take care of it.

     

  • Still within the mongo client db type the following inserts each followed by ab enter:

     

    db.usercollection.insert({ “name” : “Carlos”, “mobile” : “+615550505”, “msg” : “+Hello World”})

    db.usercollection.insert({ “name” : “Dave”, “mobile” : “+616660606”, “msg” : “+Hello World again”})

    db.usercollection.insert({ “name” : “Serene”, “mobile” : “+617770707”, “msg” : “+Hello World once more”})

     

     

  • Now, simply retrieve the inserted collection

    db.usercollection.find().pretty()


  • You can also add a JSON array of JSON payloads, so that you can insert a bulk of entries to MongoDB. For example:

     

    users = [{“name”:”Callan”, “mobile”:”+618880808″, “msg”:”More stuff”},{“name”:”Alessia”, “mobile”:”+612220202″, “msg”:”Stuff plus stuff”}]

     

    db.usercollection.insert(users);

     

  • Once again, retrieve the inserted collection:

    db.usercollection.find().pretty()


  • You can also use the usercollection object to remove individual entries by _id:

    db.usercollection.remove( {“_id”: ObjectId(“ID_GOES_HERE”)});

  • To completely clean up all your entries, you can simply use:

    db.usercollection.remove()

     

  • If you only want to use mong client to switch to a MongoDB and print a specific collection you can do this:

     

    • From Linux shell:             mongo [db_name]
    • Then, from within the mongo client:

      c = db.[collection_name];

      c.find().pretty()

  • To list all Collections in current db:

    db.getCollectionNames()

  • If you are using a MongoDB that requires authentication, you might need to create a specific user(s) under specific databases. For this follow the next steps:

     

  1. Using mongo client, login to admin db as administrator:

     

    mongo admin –username root –password [GIVEN_PASSWORD]

     

    Note: If using Bitnami, [GIVEN_PASSWORD] is the same as the OS root password.

     

  2. Switch to your MongoDB database:

     

    use [DATABASE]

     

  3. Create a new user(s)

     

    db.createUser(

    {

    user: “USER-GOES-HERE”,

    pwd: “PASSWORD-GOES-HERE”,

    roles: [ “readWrite”, “dbAdmin” ]

    }

    )

     

    Note: Substitute USER-GOES-HERE and PASSWORD-GOES-HERE with your own values.

     

  4. That’s it, you can now use that username and password to authenticate and grant read and write access to your db. If using monk in NodeJS as shown below in this blog, make sure to add the credentials in the connection string. A simple way to achieve this is by adding “USER:PASSWD@” before the server name.

     

    For example:

     

    var db = monk(USER + ‘:’ + PASSWD + ‘@’ + MONGODB_SERVER + ‘:’ + MONGODB_PORT + ‘/’ + DB_NAME);

 

 

For more information: https://docs.mongodb.com/manual/reference/mongo-shell/

 

Ok, our MongoDB is behaving well. Let’s run it together with our Express NodeJS code.

 

Building a simple Express JS API that interacts with a MongoDB

 

Now that we have a running MongoDB server running, let’s build a simple NodeJS code that will expose simple Express REST APIs to interact with your MongoDB.

I am going to explain below those snippets that require explanation, however before continuing make sure that you have downloaded/cloned the NodeJS project found here: https://github.com/barackd222/s2v-iia-nodejs-mongodb-crud-demo

The bits that you need to pay special attention from the NodeJS code:

  • app.js – This is a very standard Express JS code that starts a service listening in a specific port. In terms of MongoDB, notice the following snippets:

     

    • You need to require mongodb module, as well as monk.

  • Monk is going to give you the ability to connect to a MongoDB running somewhere. Think of it as a JDBC connection, in the relational database world, if you like.

  • Configuring the Express app to add the Monk db as part of it, so that it can be accessed within the request variable in each REST API.

 

  • The rest is just adding the right headers for the requests and starting the listening service.

 

  • mongoDBCrudAPIs.js – This file is where I define my REST APIs. At the time of writing this blog I just added 2 APIs, one that GET all Users in the MongoDB and the other that POST a new user.

     

    • GET /users – This is a very
      typical Express GET API. Inside the body though that’s where we:
      • Retrieve the monk DB connection that points to the MongoDB
      • Builds the Collection object as we did in the MongoDB console before
      • Runs a Find ALL function on the collections object that it built in the previous step
      • Sends back the resulting JSON payload as the API response. Notice that you can iterate and do whatever you want with this result

  • POST /user – Once again, this is a very typical POST Express API that retrieves some JSON body elements from the request. In terms of MongoDB, the bits that are worthwhile showing are:
    • Retrieve the monk DB connection that points to the MongoDB

  • Builds the Collection object as we did in the MongoDB console before
  • Uses the collection to insert the new user in a JSON format
  • Then it just handles errors if any. Otherwise return a successful response


  • Package.json – Make sure to add the right modules, so that you can run your NodeJS Application without having to manually install dependencies. In this case I am using:
    • express – To build the REST APIs
    • mongodb – To bring internal MongoDB libraries
    • monk – To help establish the MongoDB db connection
    • body-parser – To parse the incoming JSON requests

 

Let’s test our NodeJS Application

If you have been following this blog, at this point you have a few users in your “myTestDB” MongoDB, that you enter via the mongo client command line earlier.

  • I am using Postman to call the APIs, you can use your preferred test REST client. Let’s first try to retrieve these users.

  • Great job, now invoke your POST API to create a new one. In my case this is the JSON that I am sending:

{“name”:”Jason”,

“mobile”:”+6115791010101″,

“msg”:”Hi. This message comes from POSTMAN “}

 

Note: Make sure to add your header Content-type set to application/json

 

 

  • Now call the GET
    /users API again and make sure your new user is retrieved

Congratulations, you just managed to do very basic, yet powerful tasks using MongoDB as the backstore and NodeJS to expose interactive REST APIs via Express JS

 

If you have any comment, feel free to contact any of the authors of solutionsanz.blog – We are here to help you!

Thanks,