Apache BarCamp DC - Join Us!

Registration is open and we're just over 3 weeks from the first ever Apache BarCamp Washington, D.C.! The [free] tickets are going fast and space is limited so if you haven't already, click on over to the registration and join us!

Like all BarCamp's, the agenda will ultimately be determined by who attends but we do anticipate some great discussion on some favorite Apache-related topics from Apache Mahout to Apache Lucene, who knows, maybe an Apache Accumulo  talk will slip in!  Also, come learn how the Apache Software Foundation works, what the Apache Way is, and how to contribute to open source software in general.

I hope to see you there!

A chess set, won

John and I have been playing chess since he was four.  I've made it a point over the years never to take it easy on him.  Ever.  In the early games, he really took losing to heart and Rebecca would shake her head disapprovingly at the heartless... er, competitive... way I played, so I made a deal that the day he wins, he gets my chess set.  

Tonight, at seven years of age, he has caused me to be in the market for a new chess set.  He beat me, and I didn't even see it coming.

I always knew the day would come but, for some reason, I assumed it'd be gradual, perceptible, and, frankly, years away.  I was focused on bringing my Queen to cover the eighth rank while he was seemingly goofing around with some pawns, then, in a swift Qg7->Qa1 it was done, "got you Daddy."  He says that a lot though, so it took a while for it to slowly sink in.  Sadness, pride, happiness, deflation, humility... he won. Fair and square, he won and got himself a pretty decent chess set tonight.  

I'm so happy for him and look forward to many more games on his "new" chess set...

Enlightened corporate leadership

In a world where corporate leadership is dominated by a pursuit of growth and profit, it's encouraging to see some examples of enlightened leadership.  I initially came across this post from Tim O'Reilly saying,

"It was at this time that I formulated an image that I've used many times since: profit in a business is like gas in a car. You don't want to run out of gas, but neither do you want to think that your road trip is a tour of gas stations."

 He references a Huffington Post article in which Steve Jobs has this gem:

"My passion has been to build an enduring company where people were motivated to make great products," Jobs told Isaacson. "[T]he products, not the profits, were the motivation. Sculley flipped these priorities to where the goal was to make money. It's a subtle difference, but it ends up meaning everything."

And back to Tim O'Reilly,

"What's so great about the Apple story is that Steve ended up making enormous amounts of money without making it a primary goal of the company. (Ditto Larry and Sergey at Google.) ...  Making money through true value creation driven by the desire to make great things that last, and make the world a better place - that's the heart of what is best in capitalism. "

This sort of thinking is also represented clearly in the book Firms of Endearment.  The world would be a better place if more business leaders would focus their corporate energy into something they are deeply passionate about and allow profit and growth to be a nice side effect.  Who knows, maybe Jobs' legacy will be to inspire a generation of leadership that has the courage and confidence to passionately believe in what their doing and stop with the "growth/profit as a primal focus" meme.

Boots

If you've been fortunate enough to take a SERE course there's an excellent chance you've had Rudyard Kipling's poem Boots branded on your mind.  It's a special poem to SERE graduates.  But simply reading it's words isn't particularly satisfying, you've likely yearned to hear it in a particular audio form.  I don't know if this is the exact audio but with it's emotion and poor audio quality, it's close enough.

This reading makes the Library of Congress' National Jukebox project worth every penny to me! (actual poem starts at ~34 seconds)

PicasaUploader to Facebook Problems

So, this is semi-tested theory that hasn't let me down so far.  I use the PicasaUploader App in Facebook to get my pictures from Picasa to Facebook's Photos with frustrating intermittent success.  Sometimes it would go through the process perfectly and others it would never open the browser to do the last step that confirms the upload.  After playing around a while, it seems that <= 9 pictures is the key.  Try uploading 10 or more and it fails, but 9 or below is fine.  I dunno, it sounds arbitrary and strange, but this has worked so far for me.  I post this here in case someone else is having similar problems maybe they'll find this and confirm it.  YMMV.

First class link relations

The original proposal had link types:

"it is useful for the system to be aware of the generic types of the links between items (dependences, for example)... without imposing any limitations." ... "Note that each link has a type ("includes" for example)..." (1989)

and later, it was made clear that link relations should be first-class - peers to the target itself - or, first-class link selection criteria:

"In this way, link relationships in HTML, and in future XML hypertext languages, should migrate to becoming first class objects." (1999)

I suppose that migration is finally upon us?  RFC 5988 - Web Linking

REST and self-descriptiveness

The REST Architectural Style has this constraint called the Uniform Interface. The Uniform Interface Constraint itself decomposes into four constraints and one of those, the self-descriptive messages constraint, decomposes into at *least* three other constraints. The analysis of what system properties are evoked was done at the highest level - the Uniform Interface Constraint. So, what happens when you modify one of the sub-sub constraints? Who knows, as with all the constraints, you should probably *think* about it with "your eyes wide open"*....

if you don't see a movie below, open outside your reader:)

* an unfindable Roy quote

RESTFest 2010

I just returned from RESTFest 2010 in Greenville, S.C (on a side note, I wasn't thrilled about the location at first, but I should say, the people in Greenville, S.C. are great and the downtown area rocks - I never imagined I'd say such a thing about Greenville, but... there it is... it rocks - if you have a chance to work or visit, take it!) - it was great to see the Raleigh crew again and meet some new folks. Starting with Mike's thorough Hypermedia Workshop the first day and some great extended and lightening talks Saturday.

There was Apache CouchDB well-represented with some cool presentations and hacking, some really bizarre stuff (I'm thinking Events:), and everything in between. I don't do .Net development but if I did, I'd be pretty excited to get my hands on the coolness that Glenn presented on that platform. Anyway, many thanks to Ben and Mike for all their hard work to pull this thing together... Here's everyone that made it all the way to the end:



And a HUGE thanks to all the great sponsors. They were:

• github
• CouchOne
• Microsoft
• O'Reilly
• gnoso
• telerik
• pluralsight
• Devexpress

Health through good-for-you food

Science and the medical community are finally catching up to our elders and "raw foodies" who have, for years, claimed that eating your [natural non-GMO] vegetables will keep you healthy.  I can't remember it now, but I watched a convincing movie several years ago where a collection of last-resort cancer patients went into seclusion on a raw food diet and most (if not all) returned cancer free.

I have to say, that the secondary benefit of healthy eating might be reducing obesity, gave me a "duh!" reaction.  Anyway, ridiculously interesting, you really should watch it.

Happy SysAdminDay Infra

Happy Sysadmin Appreciation Day to the Apache Infra team!  Thanks so much for all you do to keep us safely and securely computing.  We enjoy the fruits of your ridiculous talent and tireless labor, you guys rock!

Architecture of Governance Structures

When working on complex problems it's easy to get bogged down by stuff that just doesn't matter. In software, we address this through architecture. We define, up front, properties we'd like to see in the end system. Then, we carefully begin adding constraints that will evoke those system properties. Over time, we learn that we can snapshot certain coordinated constraints and refer to them by a simple name - as an "architectural style."

For example, we might endeavor to have a system that evokes the properties of re-usability, simplicity, and evolvability and this might lead us to a design based on the Pipe-and-Filters Architectural Style which is a set of contraints (e.g. Uniform Interface) known to positively effect those properties. But you can learn more about that reading Roy's dissertation.

Like software architecture, within organizational governance it's easy to get bogged down by stuff that just doesn't matter. This typically leads to more bureaucracy. I'm interested in thinking about applying the same architectural techniques to defining an organizational governance structure.

For starters, we'd need to think about the properties that are important to the organization. For this experiment, I'll use an organization with wide familiarity - the Apache Software Foundation. So let's define some [subset of] organizational properties that are important to the ASF:

• Participatory (Pa) - the characteristic of affording the opportunity for broad participation.
• Stability (S) - the characteristic of the organization and it's product to be viable over a long period of time.
• Protectiveness (Pr) - the characteristic of being protective of volunteers and end-users from liability.
• Frictionless (F) - the characteristic of achieving goals while minimizing friction.
• Modifiability (M) - the ease with which a change can be made to the organization and/or its products.
• ...

This is enough to get the experiment going. Now, we can begin adding constraints and reason about their effect on the organizations governance goals.

1. Open mailing lists. Open mailing lists shall be used to facilitate communication and decisions. By channeling communications over asynchronous medium we maximize participation (Pa). We further maximize participation by being volunteer friendly - they can get to it when they get to it. It enhances stability (S) by being both openly archived and self-documenting. It moderately evokes modifiability (M) allowing anyone to participate in decisions and direction of the project.

2. CLA/CCLA/Grants. All contributors must agree to grant copyright/license to the ASF. This has a positive effect on long-term project stability (S) and modifiability (M). The great positive effective also negatively impacts wide participation (Pa).

3. Release Process. All projects must have formal releases following a process that ensures license compatibility, export controls, etc. This has a positive effect on protectiveness (Pr) by ensuring that software release to the public is done under the Corporate entity (ie. the Foundation).

4. Decision making. When coordination is required, projects use simple, pre-defined voting procedures. These are simple votes (+1, 0, -1) for pre-defined approval types (e.g. lazy consensus, consensus, etc.) for pre-defined actions (e.g. releases, major code shift, new committer, etc.). Having a transparent decision making process limits friction (F). It also has a positive impact on long-term stability (S), participation (Pa), and modifiability (M).

5...

Hopefully that's enough to convey the idea. Once the properties are fully enumerated and the constraints fleshed out, we might dub this coordinated constraints the "Apache Organizational Style." I'm sure the analogy and, therefore, the framework breaks down somewhere, but it seems a better fit than the typical adhoc adaptiveness that many organizations currently suffer. I think the key value that this brings is focus. Every proposed governance change would need to be reasoned about through the lens of desired organizational properties. It also forces the organization to think about what properties it really thinks are important.

WWW2010 WS-REST Workshop

Day 1 of WWW2010 is complete. I spent the day in the WS-REST Workshop. The day, for me, started out a bit sketchy. Sam's talk was ridiculously interesting, but only tangentially related to REST. Most of the others were solid, predictable and safe. The couple presentations that did approach the edge of our current understanding of REST were, well, bizarre and, arguably, an impedance mismatch for the style itself.

Federico's talk (no slides) was pretty interesting. He suggested the need to describe and communicate new styles that were based on REST but not exactly REST. More generally, a practical model for visualizing and understanding architecture design rationale. He had a modeling tool (i forget the name) that manages a graph of constraint, rationale, and properties (or somesuch). This is somewhat analogous to the styles->properties mapping in the dissertation itself but instead of talking about styles, talking about the constraints directly. For other reasons, I've come to talk directly about constraints instead of REST too lately. It turns out that frameworks facilitate getting REST wrong just as much as they do getting it right, so instead of talking about a squishy style, I've found it more useful to enumerate the constraints explicitly. Having a modeling approach to reason about all this would be pretty useful.

Being hosted at a conference dominated by academia, potential "research areas" seemed a key focus. Some thoughts that I had during the day...

1) It seems that we need more research on getting from style to architecture. Practically, we end up with an unreasonable delta between where the style leaves off and additional constraints of the architecture. We need a framework for defining, sharing, and reasoning about those extended constraints. This doesn't get highlighted very often primarily because the folks that talk about REST do so from the perspective of a single service rather than an
architecture group constraining a set of services across an enterprise? For example, suppose a particular architectural style gives us 6 constraints and a specific architecture has 3 additional constraints. Those 3 constraints are, typically, hidden as implementation details but they should be highlighted and reasoned about in the same way that the style itself was.

2) Someone mentioned the need for the sales pitch for REST style. In other words, if a simple RPC-style solution could be hacked together in half the time, why spend time designing media types, changing the paradigm, etc. ? Intuitively, it seems that the answer is the system properties evoked by the REST style, but maybe that doesn't resonate with managers, etc.?

3) Media type design. Someone mentioned this too, but it's clearly a theme in the RESTful world. It seems that we have collectively grokked what Roy has given us in his dissertation so far and are all collectively struggling with the variety of media type issues. When to pick one over another; when to create your own; how to create your own; json:link types missing; when to use specific vs. generic representation control data (e.g. /xml or /myformat+xml); etc.

4) Link qualifications. Link relations are clearly the way to assert the semantics of the relationship between the "current" resource and some "target" resource. I think we need some clear way to qualify the "target" resource independent of it's relation to the current. Perhaps there isn't a particularly important link relation but you want to assert some link selection criteria anyway.

5) Transactions, of course, got token mention. Mike(i think) mentioned them as the "third rail" of REST:) I dunno, I don't do services on the wild internet so maybe it's a concern, but I got a gut feeling that we've got some problem-solution force-fit going on here. I see the same thing with messaging too - I haven't seen a lot of valid needs for a composite REST/Publish-Subscribe Style. I like REST. A lot. But there are plenty of other valid architectural styles appropriate to other problem domains.

6) Similar to #1 above where there needs to be a macro discussion of REST for the enterprise, I wonder if there's a place for REST style in the micro. It seems like the same REST principles could be applied to micro application architecture. Something like the Actor model of immutable message passing.

Anyway, my rough notes of Day 1 as I understood it...

Architectural Styles, Constraints, Desired Properties, etc.

REST gets all the attention, but I think the framework presented in the first half of the dissertation is equally, if not more, important.  There are many frameworks that attempt to understand software architecture but none that I've found that are reasonable.  REST itself is a derived by way of a worked example through that reasoning framework.  I'd like to start using that approach myself but doing so demands a common understanding which, as with REST itself, is rather elusive.  Trying to explain it, as Roy did, in technical terms is further burdened by preconceived notions of overloaded terminology.  So, I crafted a story that attempts to communicate the essence (constraints, desired properties, architectural styles, etc.) without being burdened by the baggage of assumptions.

The story goes like this...

Suppose a customer comes to your "information business" and says, "I have a need for all the information on organic gardening."  The customer travels a lot and needs the information available to him for reference when his customers ask gardening questions, but he's frequently in the fields and, so, isn't always technology enabled.

Your organization is quite large and information comes from a variety of departments.  To make matters worse, you're in the Solutions Architecture side of the business and so you have no real authority to dictate a precise solution - these are 'information engineers' after all, who need room to flex their creativity.  You are, however, allowed to define the solution architecture by way of  "constraints" on their solution.  These "constraints" take the form of:

1) All the information must be together.
2) There must be a Table of Content.
3) All information must have a reference back to the original source.

You have done this so often though, that you and these engineers have agreed that these constraints can be grouped together and referenced by a simple name, its architectural style name, instead of enumerating each one every time.  This is beneficial because you know that certain constraints, when grouped together, evoke certain properties that are commonly desired by your customer.  This allows you to quickly match up your customer's needs with some starting constraints.  Now, you've previously agreed that constraints 1+2+3 above will be referred to as the Compilation Architectural Style.

It turns out that the constraints of the Compilation Style are a good starting point but they don't evoke all the properties that your customer really wants. They want something that's lightweight because they travel, they want something that's easily readable, and they also want something that doesn't require electricity/technology.

So you begin with an instance of the Compilation Architectural Style and add some concrete constraints to get you from "style" to a real architecture and evoke some properties specifically desired by your customer.  Namely, you add the following:

A) Compilation Architectural Style
 - evokes all properties known by the style
B) Information must be on paper
 - evokes lack-of-technology property
C) Information must be printed in Times New Roman
 - evokes readability property
D) Must be in a thin plastic binder
 - evokes lightweight property

So, you pass along the customer order and your solution architecture to the engineers.  Because you've chosen to define your solution architecture in terms of "constraints that evoke properties," you're able to objectively reason about them.  So, when the engineers come back and say that they'd prefer Helvetica because, being sans-serif, it would save on toner cost, you can reason about how changing this constraint might effective your overall solution architecture.  In this case, that level of font-specificity was simply you trying to flex some control where you have none, so you acquiesce. Likewise, the engineers come back and ask that you change constraint D to heavy-weight paper since it'd be a bit lighter - you, again, agree that it still evokes your desired property.

You deliver your solution, which makes your customer happy.  But then you realize that you ought to capitalize on your latest back and forth with the information engineers.  So, you go to the engineers and agree to call constraints B+C+D the Paperback Architectural Style.  In future requests like the original, this allows you to simply refer to a hybrid (Compilation Architectural Style + Paperback Architectural
Style) solution architecture and know that the desired properties will be evoked.

iMac

One of the annoying bits of leaving a job is leaving behind a laptop that has accumulated mass amounts of data. I've spent a good amount of time this weekend sifting through what's mine/what's not and transferring to the new machine.

The good news is that the new machine is a beautiful new 24" iMac. I've wanted one for years but could never really justify the purchase for myself and Becca didn't want one as her computer. I purchased it through Amazon which allowed me to skip out on some hefty sales tax and overnight it for $3.99 (I love Amazon Prime).

The only semi-challenging part so far is getting a proper development environment set up. I've long used TortoiseSVN for version control with Forrest and I'm going to miss it for sure. I immediately began the hunt for a mac replacement for it. It didn't take long to turn up SCPlugin which hooks right into Finder. I had only one last hurdle to clear in using it. I attempted to check out the Apache Forrest trunk and immediately got a PROPFIND error:

PROPFIND of '/repos/asf/forrest/trunk': Server certificate verification failed: issuer is not trusted (https://svn.apache.org)

After poking around a while, it seems you need to use the command line svn client (included in Leopard) to make an initial connection to the Subversion server to let it know that you trust the certificate (e.g. "svn ls https://somelocation/path" then, when prompted, type "p" to permanently accept it). Why SCPlugin couldn't simply do that for me? I haven't a clue but whatever. It's a small nit.

Ok, as I write this I realize there were some other challenges as well:

• Quicken.  I was a Quicken user on Windows and it's not exactly easy to convert the data to Quicken for Mac.  It's odd that Quicken's internal data format is different for different operating systems.  It's even more odd that they seem unable to create a loss-less conversion from one format to the other.  Fortunately, they have at least documented the quirkiness.    
• Picasa.  This, I will miss.  I've been a huge fan of all things Google but Picasa has to be one of my favorite.  Folks seem to think iPhoto will fill the void but so far I have found it unsatisfying.  No more simple enhancements.  No more geo-tagging.  Oh well.  

Fortunately, the intuitiveness, responsiveness, etc. of the Mac easily make me forget these trouble spots.  

Curious?