With the work that I’ve been doing with Open Street Map (here), I’ve been provisioning Pelias (here) – an open-source implementation of geocoding. This architecture is not small (consisting of 10+ docker images, and potentially 100+GB of raw geo data) especially if you are looking to geocode the whole world. The workload (or pipeline) had 4 main stages – download, prepare, import and query.
Download – to get the raw data sources
Prepare – to get the raw data into a format that can be easily imported
Import – to import the data into the elastic search (which is the backend)
Query – to accept geocode queries
Each of these stages have different performance characteristics and required different resources. The main thing that I’m looking at here is the use of compute. The need for compute during the prepare and import stages is significantly different from the download and query stages. I’m also not confidently in terms of when or how much I need.
And this is why I configured a burstable instance.
Here’s a couple of things to know …
There is a baseline utilisation OCPU. Consider this as a the minimum compute you want. For my scenario, it was primarily how much compute that I needed for the download and query stages.
There is full utilisation OCPU. Where this is can be 2x or 8x the baseline utilisation. (in the terms of the documentation – the baseline utilisation can be either 12.5% or 50% of the full utilisation OCPU). For my scenario, it was primarily the prepare and import stages that needed the additional compute.
The increased capacity is based upon the CPU utilisation metrics to determine whether to burst.
The average CPU utilisation for the month needs to up to the baseline utilisation OCPU.
Burstable Instances billing is known. It doesn’t come with Bill Shock.
You can find out more about Oracle Cloud Infrastructure burstable instances (here). If you want to try this out yourself or work on your own application, sign-up (here) for the free Oracle Cloud Trial. I’d be interested to hear your experiences and learn from others as well. Leave a comment or contact me at firstname.lastname@example.org if you want to collaborate.
Oracle Cloud Infrastructure provides a ton of useful services for automating and orchestrating behaviours in your cloud environment, and while they are often pretty handy on their own, leveraging them together gives almost complete flexibility on what you can achieve. Want to trigger a backup using a command in slack, then have a message get sent back when it completes? Sure! Want to periodically poll a log API and archive the results? Easy. Oracle Cloud Infrastructure provides a number of inbuilt capabilities, as well as the ability to jump into arbitrary code to build elaborate automation flows, and this blog post will focus upon the security constructs around this, looking at how services can be authorised to invoke one another, as well as how they authenticate themselves, while avoiding storing sensitive data in insecure ways. This post is intended as an overview of the concepts, and will be referenced in more concrete ways in future.
Over the past couple of weeks, I was getting back into the normal life of Cloud Engineering (the #BuildWithAI global hackathon isn’t the only thing that I focus on – check this article out #BuildWithAI Announces Winners). And something that I was doing was actually less about technology but more about budgeting – Cloud Estimations.
This is an interesting puzzle because of a couple of different elements.
Cloud is supposed to be elastic. But budgeting is typically not. Nor are project estimations and costs. Nor are approval processes. Nor are procurement processes. There are so many things in a business that are not elastic.
The people provisioning are not necessarily in charge of the costs. And I know as a developer, these overarching cost discussions aren’t necessarily the one you get invited to.
APIs are becoming the window to the digital assets of the modern business. Well documented, well governed and easy to use APIs are key to their successful uptake, longevity and associated business success. Yes, I did say well documented. In this instance I am talking about the documentation required to describe the APIs capabilities in a manner that is meaningful for your ultimate audience, the “API Consumers”, however it will also provide the template for the API Developer to develop their code from. In the modern business climate, we probably don’t want to produce War and Peace, we simply want to take a minimum viable approach to our API documentation. But where would I find a capability that will simplify our task as API Designers, capture the design documentation for our APIs, allow us to do some initialise testing to validate the usefulness of our design before any code is cut, and also have the documentation ready for consumption by team members and interested parties using a standards based approach. Where indeed ! Look no further than Apiary.io. Continue reading “Apiary designed APIs tested using Dredd”