But when you start to really think about that concept, there's a deep dive you can take, navigating various programming and computer science constructs and applying those to your infrastructure.
I've been working pretty heavily on getting the first API stable release of Noah out the door. It's been a challenge with the schedule I have to work on it - which is essentially "when everyone else in the house is asleep and I'm awake'. Last night, I came to a fork in the road where I needed to make a decision. This decision would lock me into an API path that I was unwilling to change for a while. Nobody wants to use a service or tool with a constantly changing API. I needed to shit or get off the pot, to use a creative euphemism. With the announcements of both Doozer and riak_zab, it was clear that I wasn't the only person attempting to tackle the ZooKeeper space.
Since Github lacks any facility for soliciting project feedback (hint hint, @github), I decided to create a Wufoo form and tweet it out. I don't have a very big audience but I was hoping it would at least get to the people who were likely to use Noah. The form was fairly simple with one question on something that I had pretty summarily dismissed early on - HATEOAS (hypermedia as the engine of application state).
A small HATEOAS diversion
The HATEOAS debate is a lot like Linux vs. GNU/Linux. It's fairly esoteric but there's some meat to the matter. My problem with it was simply that, despite what Roy Fielding and others intended, REST had taken on a new definition and it wasn't the strict HATEOAS one. Additionally, I found it VERY difficult to map HATEOAS concepts to JSON. JSON is a great format but a rich document structure is not (rightly so) part of the format. It's intended to be simple, easily read and cleanly mapped to machine readable format. It also felt like extra work on the part of the API consumer. The concepts that we use when reading a website (click this link, read this list, click this link) are simple not necessary when you have a contextually relevant (or descriptive) URL scheme. True, as a human I don't make changes in the URL bar to navigate a site (I use the links provided by the site) but when it comes to dealing with an API, I don't exhibit the same usage patterns as a web browser. I'm making distinct atomic transactions (DELETE this resource, PUT this resource) at a given endpoint. These simply aren't the same as filling out forms and are only tangentially related. I'm simply not willing to force someone to parse a JSON object to tell them how to create a new object in the system. The API for Noah is fairly simple as it is. Objects in the system have only two or three required attributes for a given operation and normally one of those attributes is directly inferable from the URL.
But based on the poll results thus far, I wanted to give the idea fair consideration which led me to think about what types of objects Noah had in its system.
Primitives
For those who aren't familiar or simple don't know, there's a term in computer science and programming called "Primitive". It essentially means a basic data type in a language from which other complex data types are created. A building block if you will. Some easily grokable examples of primitives are Characters and Integers. Some languages actually have ONE primitive like Object and everything is built on top of that. You could get into a semantic argument about a lot of this so I'm going to leave it at that.
But back to the phrase "Infrastucture as code". If we start looking at how we "program" our infrastructure, what are the "primitives" that our language supports. I inadvertently created some of these in Noah. I've been calling them the "opinionated models" but really in the infrastructure programming language of Noah, they're primitives.
When this hit me last night, I immediately pulled out the tablet and went to work on a mind map. I laid out what I had already implemented as primitives in Noah:
- Host
- Service
- Application
- Configuration
I then started to think about other concepts in Noah. Were Ephmerals really a primitive. Not really. If anything Ephemerals are more similar to ruby's BasicObject. The only real attribute Ephemerals have are a path (similar to the object_id).
So what else would be our modern operational primitives? Remember that we're talking about building blocks here. I don't want to abstract out too much. For instance you could simply say that a "Resource" is the only real operational primitive and that everything else is built on top of that. Also consider that languages such as Python have some richer primitives built-in like tuples.
One interesting thought I had was the idea that "State" was a primitive. Again, in the world of operations and infrastructure, one of your basic building blocks is if something is available or not - up or down. At first glance it would appear that this maps pretty cleanly to a Boolean (which is a primitive in most languages) however I think it's a richer primitive than that.
In the world of operations, State is actually quaternary (if that's the right word) rather than binary. There are two distinct areas between up and down that have dramatically different implications on how you interact with it:
- Up
- Down
- Pending Up
- Pending Down
Currently in Noah, we simple have Up, Down and Pending but something that is in the State of shutting down is grossly different than something in the state of Starting up. Look at a database that is queiscing connections. It's in a state of "Pending Down". It's still servicing existing requests. However a database in the state of "Pending Up" is NOT servicing any requests.
So I'm curious what other thoughts people have. What else are the basic building blocks of modern operations when viewed through the lens of "infrastructure as code"?
For the record, I'm still pretty confident that Noah still has a place in the Doozer, riak_zab, ZooKeeper world. All three of those still rely on the persistent connection for signaling and broadcast whereas Noah is fundamentally about the disconnected and asynchronous world.
7 comments:
Dependency (or Relationship) might be worth considering extracting from Configuration (assuming it was intended there)?
I hadn't seen Noah before (or Doozer or riak_zab), but we built something internally to solve basically the exact same problem. Ours is based around Fabric (with Chef solo) for actual interaction and uses Amazon's SimpleDB to store all of the configuration data and state.
I hadn't thought about it, but we also used a kind of "state" primitive. We only use up/down though and a separate "health" state (healthy/unhealthy). Pending up and pending down seem like useful concepts though.
I can't wait for the time when rolling your own tools in this area will be like rolling your own web framework. Redundant :)
That's an interesting idea. Currently in Noah,relationship is handled either via tagging or linking. Tags are exactly what they sound like. Linking is a bit more flexible in that it let's you aggregate a bunch of disparate objects in the system under a custom hierarchy.
Dependency is something I hadn't yet considered because it's a rabbit hole I'm not ready to head down just yet.
As for being primitives, I'll have to give it a noodle.
Wes,
I think that rolling your own will be a rite of passage for every programming SA person. Kind of like every developer writing his own cms or blogging engine.
Your tool sounds pretty damn sexy too. Would love to hear more about it. I also hadn't considered health as being distinct from state. Interesting take. I want to think on that one a bit more.
I haven't enabled it yet but I'm adding metadata support to all the objects in Noah. That would be one place and enduser could store fuzzy information like health.
Does "Application" encompass "Process"? I've racked my brain good now, and that's the only other thing I can think of.
Looking through the wiki, it seems like Application is a bit higher level than Process. An Application would be something like "My Blog." The Service primitive might map closer to something like a mysql process that your blog pulls the blog posts from. Services seem to have a Host object and a state where as Application's are just basically a place to store configuration information.
What I'm a bit unclear on is how you tie Application objects and Service objects together. Let's say I've got one simple application with a loadbalancer, 2 apache app servers and 1 DB backend. Since they're separate servers, each would have a different Host and each would seem to be be a separate Service, but how does my Application object coordinate which services it cares about?
I'll comment on both you and Wes here. The ORIGINAL intention was this:
Hosts have Services
Applications have Configurations
Hosts are obvious (I think). The difference between Applications and Services is subtle but important. A service is "apache" where as Application is the web app running on Apache.
I like the think of it in terms of the OSI model:
Host L2
Service L3
Application L7
Recently I removed the binding of Configuration from just Application in the Noah primitives. As for tying it all together, there's no inherent relationship in Noah between the 4 primitives as a whole. In fact, I could just as easily remove all the relationships and have them be distinct models with no inter-model deps. It's something I've considered.
So Wes, in the case you're describing, the way I might model it in Noah like this:
https://gist.github.com/9a12afcde524198a5b1a
As I said, there's no real inherent relationship between anything in the system. You could just as easily model all of the above under the Ephemeral objects with custom JSON as the data. In the system you described, The Application object sort of stands alone. There's no mechanism, though I've considered it, for creating a Watch on an endpoint that actually updates another object in Noah. So your registered watch endpoint might be noah://noahserver:port/path/to/object?field=blah. That would actually be pretty handy to have.
Where watches really shine is in the same places with ZooKeeper. Who is my current database master? Who are all the hosts available to service X request? What is the value of the Configuration object? When any of those items change, a Watch can fire to callback into your application saying "hey, this changed".
Post a Comment