But when you start to really think about that concept, there's a deep dive you can take, navigating various programming and computer science constructs and applying those to your infrastructure.
I've been working pretty heavily on getting the first API stable release of Noah out the door. It's been a challenge with the schedule I have to work on it - which is essentially "when everyone else in the house is asleep and I'm awake'. Last night, I came to a fork in the road where I needed to make a decision. This decision would lock me into an API path that I was unwilling to change for a while. Nobody wants to use a service or tool with a constantly changing API. I needed to shit or get off the pot, to use a creative euphemism. With the announcements of both Doozer and riak_zab, it was clear that I wasn't the only person attempting to tackle the ZooKeeper space.
Since Github lacks any facility for soliciting project feedback (hint hint, @github), I decided to create a Wufoo form and tweet it out. I don't have a very big audience but I was hoping it would at least get to the people who were likely to use Noah. The form was fairly simple with one question on something that I had pretty summarily dismissed early on - HATEOAS (hypermedia as the engine of application state).
A small HATEOAS diversion
The HATEOAS debate is a lot like Linux vs. GNU/Linux. It's fairly esoteric but there's some meat to the matter. My problem with it was simply that, despite what Roy Fielding and others intended, REST had taken on a new definition and it wasn't the strict HATEOAS one. Additionally, I found it VERY difficult to map HATEOAS concepts to JSON. JSON is a great format but a rich document structure is not (rightly so) part of the format. It's intended to be simple, easily read and cleanly mapped to machine readable format. It also felt like extra work on the part of the API consumer. The concepts that we use when reading a website (click this link, read this list, click this link) are simple not necessary when you have a contextually relevant (or descriptive) URL scheme. True, as a human I don't make changes in the URL bar to navigate a site (I use the links provided by the site) but when it comes to dealing with an API, I don't exhibit the same usage patterns as a web browser. I'm making distinct atomic transactions (DELETE this resource, PUT this resource) at a given endpoint. These simply aren't the same as filling out forms and are only tangentially related. I'm simply not willing to force someone to parse a JSON object to tell them how to create a new object in the system. The API for Noah is fairly simple as it is. Objects in the system have only two or three required attributes for a given operation and normally one of those attributes is directly inferable from the URL.
But based on the poll results thus far, I wanted to give the idea fair consideration which led me to think about what types of objects Noah had in its system.
Primitives
For those who aren't familiar or simple don't know, there's a term in computer science and programming called "Primitive". It essentially means a basic data type in a language from which other complex data types are created. A building block if you will. Some easily grokable examples of primitives are Characters and Integers. Some languages actually have ONE primitive like Object and everything is built on top of that. You could get into a semantic argument about a lot of this so I'm going to leave it at that.
But back to the phrase "Infrastucture as code". If we start looking at how we "program" our infrastructure, what are the "primitives" that our language supports. I inadvertently created some of these in Noah. I've been calling them the "opinionated models" but really in the infrastructure programming language of Noah, they're primitives.
When this hit me last night, I immediately pulled out the tablet and went to work on a mind map. I laid out what I had already implemented as primitives in Noah:
- Host
- Service
- Application
- Configuration
I then started to think about other concepts in Noah. Were Ephmerals really a primitive. Not really. If anything Ephemerals are more similar to ruby's BasicObject. The only real attribute Ephemerals have are a path (similar to the object_id).
So what else would be our modern operational primitives? Remember that we're talking about building blocks here. I don't want to abstract out too much. For instance you could simply say that a "Resource" is the only real operational primitive and that everything else is built on top of that. Also consider that languages such as Python have some richer primitives built-in like tuples.
One interesting thought I had was the idea that "State" was a primitive. Again, in the world of operations and infrastructure, one of your basic building blocks is if something is available or not - up or down. At first glance it would appear that this maps pretty cleanly to a Boolean (which is a primitive in most languages) however I think it's a richer primitive than that.
In the world of operations, State is actually quaternary (if that's the right word) rather than binary. There are two distinct areas between up and down that have dramatically different implications on how you interact with it:
- Up
- Down
- Pending Up
- Pending Down
Currently in Noah, we simple have Up, Down and Pending but something that is in the State of shutting down is grossly different than something in the state of Starting up. Look at a database that is queiscing connections. It's in a state of "Pending Down". It's still servicing existing requests. However a database in the state of "Pending Up" is NOT servicing any requests.
So I'm curious what other thoughts people have. What else are the basic building blocks of modern operations when viewed through the lens of "infrastructure as code"?
For the record, I'm still pretty confident that Noah still has a place in the Doozer, riak_zab, ZooKeeper world. All three of those still rely on the persistent connection for signaling and broadcast whereas Noah is fundamentally about the disconnected and asynchronous world.
