Tuesday, November 24, 2009

Saving the newspaper industry

I debated whether or not to post this but I figured what the hell. If somehow it gets read by my employer and causes issues, I'll cope.

I've recently become employed as a contractor with a large newspaper in the Atlanta area. I was and am really excited by the opportunity. The technologies are cool and it's a market I never thought I'd be in.

This particular paper has been hit pretty hard by the declining revenue in the industry because they are so large. They're currently in the process of moving out of the cornerstone building they've been in downtown since (I think) 1972. It's a gorgeous building and I would have love to been there during the hey day. These days the building is like 30% occupied or something absurd. One of the paths I take to get lunch takes me through the back past all the old printing and sorting equipment. It's very ghostly and I plan on getting some pictures before we move.

Anyway, I was talking to a coworker about a project they're working on. He needed some feedback about solutions from a system engineering side. During the conversation, we got off on a tangent about the company and recent history. It was legitimate discussion because the information helped me understand the purpose of the application and what it needed to do, how it was being accessed and such.

One thing he mentioned to me was that at the last publisher's meeting, they pointed out that digital was only 10% of the revenue and that they would be focusing more on how to increase print revenue.

I was flabbergasted. I mentioned this to my wife to gauge her thoughts and she came to the same conclusion I did.

Why would you scale back from the growing market to try and preserve the dying one?

Here's the deal. I'm pretty typical (I think) of the modern reader (and I LOVE to read).

Sidebar: I remember when this paper was actually TWO papers - one the conservative publication and the other the liberal one. Up until about 9 years ago, I had a subscription to the paper including Sunday. I got great joy out of sitting down and reading the newspaper. I still do.

Anyway, as I said I'd like to think I'm pretty representative of the transition in society. I'm 35 years old. I'm old enough to remember pre-internet and young enough to be hooked into the new stuff. Some of that might have something to do with IT but that's becoming less and less of a gap. Technology is now so pervasive and accessible that what used to be the domain of the geek is now the domain of the mom. Everyone, for the most part, has access to the Internet from any number of devices not the least of which is the cell phone.

So when I hear a statement like I heard above I have to wonder how anyone could make such a statement about trying to "life-support" the print side of things. In addition to the AMAZING post mortem from John Temple, former editor/president/publisher of the Rocky Mountain News. The market for that is slowly dying - literally. I'm not trying to be grim or hyperbolic. According to this, the largest percentage of newspaper subscribers is in the 65+ range. My age range only accounts for 17%.

What I'm trying to say is that once that subscription base starts passing on, things don't look to good. By the time I'm at that range, there simply won't be any subscription base left. From my perspective (and I'm going to be somewhat crass here), why would I buy a newspaper subscription when I can read the same news from 10 different sources on my cell phone in the morning while I'm taking a dump? I wouldn't. These days I find myself not even bothering to pick up a copy of the paper at the doctor's office or mechanic. I'm either tweeting, facebooking, posting on forums or reading RSS feeds of the aforementioned 10 different sources. I'm pretty damn efficient at it too. The generation below me is even BETTER at it. They'll be doing things when I'm 64 (cue music) I won't understand and I'll be bitching about the good old days of the tubes and the propensity of children to walk across my grass.

Newspapers cannot compete on news. At least as far as my "outside" eyes can see. News is a commodity in a sense. Word travels fast these days and it's only going to get faster. From the perspective of a print paper, the news is already old as soon as it happens in this modern age. 500 people have already Twittered, iReported, blogged , Facebooked, Youtubed and done interviews with newspapers overseas before it's been sent to the presses. Using any random bit of aggregation services and software all of that information has been compiled and presented to me in a way that I wish to consume it*. If it hasn't been, I can easily do so with a single Google search when I want to know more. I can easily participate with a hashtag, url shortening service and a cell phone if I want. I become a part of it. I'm a member of the community around that bit of information.

So what can be done? Where should newspapers go? I thought about this a lot on the way home from work tonight. I had a plan all devised in my head. It was all so clear. Too clear in fact. Obviously someone else has to have seen this as clear as I did. Maybe I got some of the ideas from John Temple's speech. I wasn't sure.

When I finally got downstairs in front of my computer, I did a quick search (on Google of course). I pulled up the copy of the Temple speech. I also searched for one phrase - "How to save the newspapers". The fourth result down** was this article on truliablog.com. There were all of my grand ideas laid bare and done so almost a year ago. This almost entirely sums up what I had in mind and plan on repeating here in condensed form.

So what was my idea that obviously isn't just my idea?

Community. The Eastern Standard Tribe kind.

Back "in the day", some of us geeks were identified by the web communities of which we were a part. We're you a Slashdotter or did you read Kuro5hin? Maybe you spent your time digging and burying. If tech wasn't your thing, were you a Deviant? These days it's much of the same but the communities are even more diverse. People like, I think, having a sense of belonging and it's much more if not entirely acceptable to belong to a community where you've probably never met half the people. You're known by an avatar and a nickname. I have people I consider friends who call me by my character name in World of Warcraft and yet we all know each other's real names and personal information just as much as some of my friends here in town. I went to parties years ago with people I only knew from IRC and was called by my nickname the entire night. I addressed others the same way.

Newspapers can establish that kind of community in the modern age. When there were two separate newspapers in town, you might have an idea about a person based on which one they read. These days, I find myself building relationships and communities with people around the world based on the most niche of ideas - a mailing list for an open source project, a coding language or a forum dedicated to an author I might like. If the author is reasonably current, I might even be able to follow them on Twitter or become a fan on Facebook. It's no less of a community than the one I actually live in which has an equally vivid online presence.

So my plan was for the newspapers to be breeding grounds for those communities. Create the tools. Forums. Aggregation. Whatever is appropriate for making it easy for people to build those communities. They will still be providing the same service they were before which, at the core, is delivery of information. Ebay is nice but Craiglist is better when I want to go local. Building local communities online can even help spur the local economy which can always use help regardless of where "local" is.

Take advantage of the personal capital you have built up in your existing personalities. If you have a reporter that people trust, that person should be right in the thick of the community. Clark Howard is someone who has an amazing community of like-minded people from everywhere that he's broadcast but I would wager he's loved nowhere else in the world more than he is right here in Atlanta where we've been listening to him locally for years. Modern news is interactive. It's no longer a matter of writing an editorial and walking away from it. You'll have to be accountable for what you say because if you aren't, you'll lose trust from the community and, as in any relationship, you may never be able to earn that trust back.

Be open. This somewhat ties to the previous statement but I'm thinking more along the lines of "open to ideas". I'm amazed by how much CNN has integrated iReport into the main "product". After the President's speech to Congress months back, CNN had two people on who represented each side of the issue. The amazing part is that those two people were from the community CNN had provided and weren't talking heads. They were real people that best represented the viewpoints of their community at CNN. Mind you I don't think the person representing one of the sides was particularly articulate but he wasn't a pundit or a news guy so I can't be too critical. Make the community part of the product. When someone feels a sense of ownership, it becomes more personal.

Don't be afraid of Google. Don't buy into the paywall tripe being spread by some business men. Don't cut off your nose to spite your face and all that. While your focus may be the traditional concept of local, there are expats all over the world who would belong in that "local" community. Those people might even be your greatest source of revenue. I can see me living in Michigan years down the road and wishing I could get an Frosted Orange. You get the idea. I still need to know about the community and if Google doesn't know about it, I won't know about it.

Commit to the community. Don't be halfway about it. If you're going to do it, do it. Nothing is more frustrating that getting involved with something (an online game or whatever) only to see it die on the vine because of lack of support from the people who put it out there in the first place. While people will gladly contribute more than you can handle, they'll need some nudging to maintain focus. If you have a sub-community built around Foodies in Atlanta, maybe you should have some prominent Foodies actually be involved in some officially recognized capacity. Moderators are a valuable thing. A forum is only as good as its signal to noise ratio. If appointing community managers is out of the question, at least provide the community with the means to self-police. Don't discount Whuffie in self-moderating environments.

Accept organic. While the communities will need some nudging to help get started, realize that they WILL take on a life of their own. They may grow beyond what you might originally envision. They'll merge. They'll split. The Atlanta Baklava community might realize that while anyone who DOESN'T like baklava is obviously deficient in some capacity, maybe they're better suited in merging with the Atlanta Pastry community as a whole.

So where does the news come out of all this? Where's the news in this modern newspaper? The communities are built FROM the news. You'll still have your traditional reporting "model" but the communities will spring up out of that news. I started following a few new people on Twitter after the #atlflood event and every single picture I saw of the flood came from either Flickr, Twitpic or *ding-ding* ajc.com. There were people watching the #atlflood hashtag who had the same experience I did. I would hope that the newpaper was ready to capitalize on that.

I don't claim to have all the answers. I've only been on the job for two weeks now. The thing is that I REALLY like the job. I enjoy the people I'm working with and I want the company to succeed. Maybe that's what's needed. Less old-school baggage and a fresh perspective from the outside. It's also more selfish than that. I would love to be at the ground level of helping to engineer something as amazing as bringing the newspapers into the modern age.

* Think Manfred Macx in Accelerando
** The first result was from Time magazine. The second was from another newspaper. The third was an article on how to save newspaper clippings.

Friday, November 20, 2009

Exchange Web Services via Ruby

I realized as I went to post an update on this that I don't think I ever posted an actual blog entry about it.

While I was on contract at HSWI, the email was hosted on Exchange. The development team was all on Mac workstations but the front-side was all on Windows. Since I was running Linux full-time, Exchange access wasn't a big deal. I ran Mutt and used LDAP/IMAPS/SSMTP. Digging around the Mail.app on the OS X side however, I realized that the client side was using Exchange Web Services for much of the functionality.

Wanting to do some poking around with Ruby and SOAP, I figured it would be a fun exercise to talk to the Exchange server with Ruby. I also had an itch to scratch thinking I could use it as an address book source for Mutt. I got it working in a day or so after dealing with some broken functionality in either the WSDL from the Exchange side or how SOAP4R tried to translate it.

You can find the code here.

Anyway, I'm no longer with HSWI but over at the AJC, we're also using Exchange in a much greater capacity. Anyway, so I whipped out the "old" code and was depressed to find out it didn't work.

Every attempt gave me a 401 error. It made no sense since I could access OWA, ActiveSync with my DROID (awww yeah) and even access the EWS wsdls on the server.

As I started to poke around online, I started to realize that in some of the more complex configurations, the Exchange server is hidden behind an ISA server or something. I don't do the Microsoft world much anymore and I have no real access to the environment.

What I noticed though, was that the internal IP is different than the external one. That gave me an idea to make sure and test the code externally from home while NOT being on the VPN.

It worked!

As I poked around with THAT information, I remembered an interesting thing that happened the one time I logged. on to a Windows machine at the office. If I fired up I.E. and went to the OWA url, I was logged in directly.

What I'm guessing is that the Exchange server has a different set of criteria for authenticating internally than externally. I'm not up to date on current Exchange and AD implementations so I have no idea what the configuration is that's preventing me from using EWS but still allowing me to use OWA from Firefox under Linux and OS X.

If any MS guys out there have any idea what's causing this, I'd be interested to know so I can either work around it or document it appropriately.

I have a *FEELING* that it's somehow related to NTLM but I don't know how to force my EWS call to bypass it just yet.