Designed for Developers - Why people keep asking you to use Github

I'll be the first to admit that I'm a Github fanboy. The shocker is that my love of Github has nothing to do with the DVCS underneath. While Git plays a major part of what makes github so great, the bigger reason github is so successful is this:

Github is designed for developers

What do I mean by that? Let's compare a series of screenshots from various code hosting sites:

 

Code Hosting Solutions comparison

I want you to take a look at the screenshots very carefully especially the "project" pages. What's the one thing you notice about Github compared to the others (excluding BitBucket). What's the focus of the project?

It's all about the code

You'll see quite clearly that with all the sites except for BitBucket, the focus of the project is the code itself. Not only is the focus of the project the code but everything about the code is about the community. I can "watch" a developer or project. I can easily see from the first page how to download the codebase. However the biggest part of what makes Github a success is one button:

Fork

From the start of a project page, not only can I easily browse the code and am provided with the information I need to checkout the code but I'm invited with a single button to become a contributor to that project. Immediately, I'm a potential contributor to that project. If I change something and push the code back to my fork, I can push one button and send a message to the project maintainers asking them to merge the changes back in. As a project maintainer, I have an easy way to evaluate the impact of the change and communicate with the requester and other team members about said change. At the bottom of the pull request page, I'm provided the information on how to easily merge those changes into my main tree.

Designed for Developers

I've been on a bit of a tear lately about usability in developer-targeted products. The latest target of my ire has been Atlassian. Let me clarify that I think Atlassian makes some wonderful products. Confluence is one of the best wikis out there. JIRA is a great issue tracking system for Developers.

However, Atlassian has some "duds" in my opinion. The biggest thorn in my side these days is Bamboo. Bamboo is Atlassian's Continuous Integration server. Like most Atlassian products, its primary target is Java developers. Everything about Bamboo is designed around the Java development toolchain - Maven, Ant and the like. But I don't have a problem with that. What I have a problem with is the over-complication. I grabbed the latest beta of Bamboo at the recommendation of one of the Bamboo developers who heard my rant on Twitter one day. He asked for some feed back and I provided it in a very detailed email. I'm happy to say that the new interface for adding build plans in Bamboo is much simpler than previous versions. I can't do screenshots of our company Bamboo install but previous versions had a VERY complicated multitab build plan configuration.

One point I mentioned in my email is that Bamboo felt like it lacked a focus. Jira was very clearly about Issues. That was the "unit of work". Confluence was very clearly about being a wiki. That was its "unit of work". Bamboo didn't have a singular focus. It was a CI server but what was the unit of work? A build plan? Test results? Fisheye integration? It wasn't clear.

Compare that with Hudson which had a very clear focus. The strength in Hudson is that it performs tasks. Those tasks are typically centered around CI but they don't have to be. In Hudson I can define a job that does nothing more than list directories. I don't even need to back it with a VCS. Bamboo, sadly, in the beta version still hasn't gotten this part right. I can't define a build plan without having a repository somewhere. It still assumes that I want to define all my work inside of an ant script. Using the "shell" builder is still VERY limiting. 

You can see some sample comparison shots between the two here. I'll try to actually setup a repo that Bamboo can use and do a deeper comparison later. 

So what's the focus of Google Code, Launchpad...

Going back to code hosting and comparing Github to the others, I think it's clear that they lack a focus. They try to do too much. They "feel" like they were designed by project managers and targeted at them. Maybe it was a faulty assumption that to effectively manage a large project, you had have all of the extra stuff. I don't know. Launchpad and others DO some things better than Github. Issue tracking is one. Github issue tracking is a pretty weak area for them. However here's where Github understands its focus and strengths.

Where Github lacks, it makes up for in integration. Github doesn't TRY to be the project manager's tool. It doesn't try to be a good issue tracker. What it DOES do is say "I suck at this. My focus is on the code and making working with and contributing to the code dead simple. I'll add hooks for the other stuff"

And they do. Github has a boatload of service hooks for everything from issue tracking to project management to irc and IM. They even have a "generic" hook that will submit JSON to a url for you so you can write your own receiver.

About BitBucket, backend technology and focus

I haven't mentioned much about BitBucket. The main reason is that at this point, BitBucket is simple attempting to feature copy from Github except using Mercurial in the background. Sadly, this isn't enough I think. If my only reason for using BitBucket is the DVCS tool then I honestly might as well use Github. I'll get more engagement there. See this quote from Mark Philips from Basho about why the moved from BitBucket to Github:

Why? There are several reasons, the primary of which is that GitHub,

the application, lends itself to more collaboration when developing

open source software. Again, this was a decision made on the basis of

community development; technically-speaking we were satisfied with

what Bitbucket offered.

The issue wasn't the technology. Mercurial and Git are pretty much at feature parity (as is Bazaar). One thing mercurial doesn't do out of the box is cherry picking but it's supported with extra configuration. Mercurial has hg incoming which let's you see what people are working on. Git has staging. Mercurial has better Windows support than Git. It's really six in one, half dozen in the other.

However what BitBucket DOESN'T have is the community. You see, BitBucket was playing catchup to Github. Simply copying the social aspects of Github isn't enough. Github has too much momentum precisely because they had the focus right from the start - code is king.

As a developer, my key focus is my code. It's what says the most about me. As a developer who wants to attract other developers, the best way to do that is showing the code and making that contribution as easy as possible. Github gets that.

That's why people keep asking you to switch to Github.

Potato Candy - A family recipe

With Halloween right around the corner and Thanksgiving beyond that, it's getting about the time of year when I get to make Potato Candy. Yes, candy made from potatoes.

I don't know the real story behind it. Ever since I was a little pile of baby fat, it's something the kids in my family have eaten. My uncle only made it for Thanksgiving and I think Christmas get-togethers. I've tried to find a bit of history about it over the years but nothing ever concrete. My uncle's family is Irish so that's as stereotypical of a reason as any. What I did seem to track down is that it's pretty unique to the Southeast. We do things weird here, ya'll.

Not long after I married my wife (a Michigan native), her aunt was putting together a family cookbook. Now that I was part of the family I got to contribute a few things. I had my mom and step-mom provide a few entries but I reserved one for myself - Potato Candy. Since the "secret" is out and because freaking @jtimberman got me thinking about candy, I figured I'd add it here for all my interweb friends.

Ingredients

• 1 Potato about the size of your fist. Seriously. Don't get it any bigger. If you've got big hands, find someone with normal sized hands and compare.
• 1 jar of peanut butter. Creamy not Crunchy. The last thing you want to deal with when making this stuff is nuts. Trust me.
• 2 bags of powdered sugar. Yes, you will probably use ALL of it.
• Wax paper and plenty of counter top space

Peel and boil the potato as you would to make mashed potatoes. When it gets sufficiently soft, mash that bastard up. No lumps. Again, trust me. As smooth as you can get it.

Dump it into a large mixing bowl and reach for the strongest and sturdiest spoon/stirring instrument you can find. Start folding in the first bag of powdered sugar.

This is where it gets fun. As the powdered sugar gets mixed in, this thing is going to get thick and heavy very quickly. It's going to be VERY hard to mix. Did you trust me on the sturdy spoon part? You should have. Don't even think about putting this in an electric mixer. It will burn out the motor. I've literally broken 1/4 inch dowel wooden spoons in this stuff. Your arm is going to hurt. You're going to have to put your back into it.

When you physically can't mix it ANYMORE put it aside for a minute. Spread out a nice sized area on the counter with wax paper and cover it in powdered sugar. This crap is sticky and you're going to need to manipulate it. Once you've gotten the workspace ready, start spreading the "mash" on the wax paper. Usually about 1/4 to 1/2 inch thick is good. You'll probably screw it up the first time around. I did.

Open the jar of peanut butter and start spreading it on the mash. Peaks are okay but you really want to get a good layer on there.

Now, the hard part

Somehow you're going to need to roll from one end of this beast to the other. Like a jelly roll. It's really hard and don't feel too bad if it isn't pretty. The end result is still good. You'll probably want to cover your hands in powdered sugar.

Once you've got it rolled up, flatten it back out. Stick it in the fridge overnight. The next day, cut it into smallish 1.5inch slices and enjoy.

As I said earlier, I've tried to do some research each year. The best picture I can find outside of making some myself is this one.

You probably won't be able to eat more than one or two pieces. It's REALLY rich and really thick. If you give it to kids, do it early in the day so they have time to burn it off.

Enjoy ya'll!

PyCon DevOps piggy back

So I had a random idea the other night and like any other random idea I immediately sent it to Twitter.

This of course brought feedback which is the whole point, right?

The idea was to have a Velocity style conference in the South East. We all know my love for Atlanta and my half-disdain/half-jealousy of the West coast. So I threw the idea out on twitter and immediately got my first reply from Joe Heck with a bit of reality thrown in:

@lusis nice idea. critical mass with either be easy or impossible to get. You might consider riffing on existing conferences ... PyCon2011

Awesome idea so I headed off to to read up on how PyCon does that kind of thing. I shot off an email to the pycon-organizers mailing list and got some really nice responses. I also got a private tweets from people on the list as well.

The end result is this. If I want to hitchhike on the back of PyCon for a devops-related conference, here are the requirements/suggestions:

• Involve Python in some way
• Will need to take advantage of the Open Spaces system

This essentially means unless I (or someone else) is giving a full blown talk on Python and DevOps, it will be an ad-hoc thing. We can't reserve the spaces until the day of the conference. I'm also not sure how big the spaces are. I think this is the same place LISA was held years ago so you might be able to snag a dividable room segment?

So what does everyone think? I'm considering giving a talk on the state of devops toolchains in Python (func, cobbler, fabric, kokki, overmind, whatever else) but I don't know that I'm ready for that yet after a single LUG presentation ;)

I know that Mitchell H. of Vagrant fame was considering heading into town for it. Vagrant isn't just for Rubyists ;)

I'm open to ideas. I'd love to just have the conference I sent the tweet about but when I really think about it, I don't think I can pull something like that off in this amount of time.

Many thanks to the pycon-organizers folks for the input - Doug Hellmann, Vern Ceder and Jesse Noller. Also to Dean Goodmanson for his feedback via Twitter.

Latest Vogeler update - MongoDB, protobufs, Riak and war!

I wanted to take a minute to post an update about Vogeler to those who are following the project. Let's get the easy stuff out of the way - it's not abandoned. Far from it.

There have been several reasons why I haven't made any commits lately, the least of which is both kids have been sick recently and I haven't been able to get a good solid block of time to work on it.

Technical Hurdles

Another reason is that I almost went down a rabbit hole with regards to swappable persistence. In the process of refactoring the persistence backend, I realized it should be fairly easy, using the model I put into place, to go ahead and implement MongoDB and Riak support. I started with MongoDB when I hit a wall. MongoDB does not allow dots in key names. When I ran into that issue, I realized that I made some dangerous assumptions based on the fact that I started with Couchdbkit as the interface to CouchDB:

I was using an ORM when I should have used a lower level driver. You see couchdbkit does some nice stuff like translating native Python datatypes to the appropriate datatypes. If I define a row as having DictProperty(), couchdbkit converts that into the commensurate CouchDB JSON datatypes. If I use ListProperty(), the same thing. This is really evidenced in Futon and makes using Futon as your interface to Vogeler very appealing. However this is VERY couchdb specific.

The pymongo driver, however, didn't like my strategy of dumping execution results that way. You can see the "gist" of what I'm talking about here:

;

I brought the issue up on the MongoDB mailing list here. I opened an issue for myself to braindump my thoughts. One of my biggest goals (data transparency) was starting to fall apart for me. I decided to shelve MongoDB for a moment and look at Riak. I wanted to make sure that I at least thought about how a generic model would work across multiple document stores. That's when I ran into the biggest cockblock:

protobuf

I'm almost firmly convinced that protobuf is a piece of trash. Google has some smart people but protobuf is something that quite obviously came out of the mind of someone who was sent off to "solve the RPC problem". There are quite a few issues I have with protobuf:

• Despite being a "universal" format, it works well in exactly TWO languages - C and Java. Everything else is an afterthought. Don't get me started on Python support. The one guy at Google who supports protobuf on Python can't make it work on anything but Python 2.5 because that's all Google uses. He's unwilling to cut a new PyPi package just to fix all the 2.5 assumptions because he doesn't want to bump the version number. You can't even install it on anything higher than 2.5 without hacking setup.py.
• You have to precompile your protos before use. I understand what Google is trying to accomplish but seriously? So I have to build the protobuf compiler to compile protos to ship with my code. There's a reason why people like FFI folks.

There are alternatives like Apache Avro that have promise but they also have their own issues. However, Basho has committed to using protobuf which does make sense. Write your own serialization framework or use an existing one? Easy answer when Google wrote one for you.

So I started to noodle out what route I wanted to take when something else came out of left field.

Sgt first Class Lance Vogeler

I have a search setup in Tweetdeck on my Droid for Vogeler. It was nice to stay on top of people mentioning the project. The name for the project came out of me pretty much immersing myself in the latest S.M. Stirling Emberverse books. One of the characters was named Ingolf Vogeler. I really enjoyed the books and liked the name so I picked it. I'm also considering using Ritva for another project.

So one day my phone starts going nuts with Vogeler alerts. I was already getting the occasional history tweet about the real Ingolf Vogeler but it turns out a soldier from Georgia, of all places, was KIA in Afghanistan. He did 8 tours in Afghanistan and 4 tours in Iraq. Politics aside (I'm personally entirely against these campaigns), I didn't want to "pollute" the twitter stream. Regardless of what I think of the current military climate in my country, I have the utmost respect for most of the members in our armed forces.

However what struck me most is that SFC Vogeler left behind a wife. A wife carrying his unborn child. That pretty much did me in. As a father myself, I was pretty torn up thinking about this happening to my wife. Yes, it was a known risk but that doesn't make it any less sad. I decided, in addition to making a donation of my own to his family and holding off on Vogeler till it wasn't alerting on Twitter so much that I would think hard about what my software means.

Steve Jobs asked a guy who emailed him this question "What have you created lately?" Someone else recently said that entrepreneurs are busy creating the next social media app that means fuckall when they could be affecting change with the software they write. That got me thinking. Could I help affect this family somehow with my project that happened to share a name with them? The best I could come up with is this:

If you use Vogeler, are interested in it or just feel like a random act of kindness, please make a small donation to the Vogeler family. My wife and I agreed that should I ever make ANY money off of the project in any identifiable form that I would donate what I could to the family. Vogeler is just a small project. I have no grand aspirations of getting integrated into some mainstream project. I'm just trying to scratch an itch - a niche itch at that. One reason I'm so gung-ho about DevOps is that, as a family man, I don't WANT to be dealing with stupid shit taking time away from my family. I've done it and I'm done with it. If my phone goes off, it's not going to be from some stupid mistake that I made editing a config file or lack of metrics causing an "oh shit we're out of space" moment. I'm past that in my career and I'm past having to work places where that's the norm rather than the exception. My family is first and foremost and anything I can do to keep it that way, I'm going to do it.

So I'm taking the Vim approach to Vogeler. I'm not going to ask anyone to go against his conscience. If you feel like a small donation to this family implies consent to the stupidity of my government then I fully understand. But if you think that open source software and the broader open source community can make a difference in more than just writing software, throw a small donation their way.

I'm going to be starting back up on Vogeler now. I've decided that for now, I'm going to attempt to keep things as generic as possible but continue to code against CouchDB. I'll keep revisiting MongoDB and Riak support but the primary target is CouchDB. If any Basho folks are reading this, if you can remove the hard dep in setup.py on protobuf, that would be awesome. You can't even install from PyPi with it in there anyway. If the MongoDB folks take a gander at this, can you do something about the dots in key names? Thanks!

Why I hope BankSimple succeeds but fear they won't have a choice in the matter

My current political affiliation is not the most popular with people right now. That's understandable. I tend to approach things from a rational and unemotional perspective. Call me callous. Call me a dipshit. I'm consistent if nothing.

But the issue that's been bugging me lately transcends party or political affiliation. It's one that touches me as a technologist, an IT worker and a citizen.

DevOps and the changing IT landscape

I've blogged/tweeted/talked/rambled enough about DevOps to make people sick. In some regards we risk crossing the line with our enthusiasm and killing interest in the topic. Ignore for a minute that much of what makes up "DevOps" is not new ground. That's not to say that the "movement" hasn't introduced some amazing things.

The point is that aspects of DevOps as a philosophy are changing how people approach IT holistically. Automation. Rapid iteration. Frequent changes. Breaking down organizational silos.

So how does this play into BankSimple?

It actually plays into it in two ways. The first is that, like DevOps, BankSimple isn't doing anything "new" per se. If you're talking "old" business, banking ranks right up there with prostitution and in shares more in common with it than most would want to admit. What's different is how BankSimple is doing it. The BankSimple guys want to make money, no doubt but they're approaching it from a different focus. It's been formed, not by the traditional cadre but by "techies". It's almost "Agile Banking".

You can guarantee that because of that perspective and the history of the cofounders, the business and by extension the IT aspect will be run differently.

If you can't beat 'em, buy 'em.

I'm a hands-off kind of guy when it comes to my government. Many people thought that, based on concepts like "hope" and "change", the government was going to turn around and focus on its customers - the citizens. It hasn't yet turned out that way.

Someone asked an interesting question at the last DevOps meetup in Atlanta.

"Do you think traditional companies feel threatened by startups who use more agile methods to beat them to market?"

The answer:

"Not really. If they get to be too big of a threat, just buy them out.".

That, of course, assumes they're willing to sell. Some people might have that luxury but when the VCs start to chime in that choice may be stripped from you. On the flipside, some might believe that by selling what they've worked so hard to build up will be destroyed yet they feel an obligation to the people who started with them to sell.

But let's assume that, in the case of BankSimple they have the goals of REALLY changing banking and finance (which I think they do). Assume they don't have to sell (which I don't know).

If you can't buy em, buy a lobbyist

I was talking to an individual a few weeks back about the idea that traditional companies would get eaten alive by leaner more agile startups. We were talking specifically about the financial sector but his comment applies. So I, with a head full of steam raving about agile operations and time to market, was hit head on with this reality in one comment:

"Yeah but do they have lobbyists?"

Every company I've worked at, save one, has had lobbyists on the books. At one company we actually had different colored checks that we printed for lobbyists vice expense checks. Some people think lobbyists just "lobby" for things like deregulation and "leave my business" alone but that's not the case.

Without going into too much detail, at one company we actually LIKED a certain amount of regulation. You see a certain amount of regulation is just enough to restrict entry into the market by new players. We liked that each state had its own set of regulations. That in one state we were classified under one set of laws while in another state we were classified differently. This ARTIFICIALLY raises the barrier to entry for competitors. It keeps them small. It relegates them to one of two positions:

• Small enough for us not to care because they can't afford to expand into new markets
• Big enough to justify buying them to eliminate the competition.

Sure, sometimes a company will succeed and grow big enough to be REAL competition but that's fine too. Helps prevent antitrust investigations. It's a win-win all around.

Not all lobbyists are bad. I would argue that the ACLU is a "good" lobbyist but they can also exist to help preserve an existing business either buy demanding hands-off or demanding more regulation under the guise of "consumer protection".

Sometimes the tree of innovation must be watered with the blood of failed business models

I sent this out as a tweet earlier tonight. With no disrespect meant to the original quote that inspired it, I firmly believe this to be true. Without even discussing politics or regulation or monopolies, the fact of the matter is that to progress as a society some things have to die. Horse and buggy. Media distribution. Publishing. All of these changes help move us to greater things. Many a business is built on convenience and inefficiencies in the supply chain. FedEx and UPS would probably have not been needed had the post office not sucked so bad. In an age when information is passed slowly, traditional news organizations made sense. BankSimple is part of that progression just as DevOps is part of that transition in IT.

But I hope that Alex and the gang aren't as naive as some people (myself included) have been about DevOps. Just as DevOps working its way into traditional organizations will have to deal with the Boogymen of HIPPA, SOX and PCI DSS, so will BankSimple have to deal with an establishment that, if threatened, sadly has the power to essentially make them "illegal".

We need them to succeed so we can move on to better and brighter things.

 

.plan

TODO

• Work through Vogeler issue queue (http://github.com/lusis/vogeler/issues)
• @padrinorb test suite enhancement - functional/integration (http://github.com/padrino/padrino-framework)
• Integrate libvirt into @fog (http://github.com/geemus/fog)
• Finish integrating Riak support into @padrinorb (http://github.com/seancribbs/ripple)
• Create a distro that uses LFS for the explicit purpose of hosting dynamic language applications (http://lusislog.blogspot.com/2010/09/distributions-and-dynamic-languages.html)
• Do all of the above while maintaining my marriage and raising two kids ;)