The Post-Facebook Internet

Source: Wikimedia commons

The recent Facebook scandal—in which Cambridge Analytica obtained 50 million Facebook users’ personal data—really shouldn’t have been such a big deal. By no means was it the biggest breach of data, nor a breach of the most sensitive kind of data. It wasn’t nearly as salacious as PRISM nor any of the other secret programs the NSA designed to siphon phone and internet data (which remained covert until whistleblower Edward Snowden famously told the Guardian about them in 2013).

The most—if not the only—potentially unlawful act in this saga was the deal struck to share the data between Aleksandr Kogan and Cambridge Analytica, a potential violation of the terms of service for the survey app that Kogan created to harvest the data in the first place. The act of collecting the data, though no longer permitted by Facebook, was perfectly lawful at the time.

So why then was this breach such a big deal?

First of all, we can universally agree that Facebook has amassed a trove of personal data larger than that of any other company on the planet. Given the obvious value of these data, Facebook is constantly targeted. When such a breach occured on the world’s largest social network, millions of people became upset, rightfully so. Facebook got pressured to explain how it happened. Still seems like no big deal.

Their explanation, however, begged an important question: Why does Facebook need all these data in the first place? That in turn led to an interesting Catch-22: In explaining the data breach, Facebook had to draw attention to its business model, namely that it collects data, anonymizes them, and sells them—albeit indirectly—to their real customers. The real customers are not us, the Facebook/Instagram/WhatsApp users. No, Facebook’s customers are ad networks and advertisers, i.e. companies and people who pay to promote their products and services on Facebook.

This “revelation” should have come as no big surprise to anyone, or at least to anyone paying attention. Revenue through tailored advertising is the business model of Google, Facebook and nearly every media entity that operates online. Last year, Facebook reported that 98% of their revenue came from advertising.

Using your data (and mine and everyone else’s), Facebook built an incredibly powerful ad targeting platform, a platform we allowed them to build and deploy when we accepted their terms of service—all two (plus) billion of us.

It’s even possible—through Facebook’s publicly-available advertising platform—to target a 41-year-old man in San Francisco who speaks Spanglish, who has attended at least one Lindyhop event and who belongs to the Bay Area Esk8 group. In other words, I can target an ad so narrowly that it’s shown only to me. (I just tried this, and though the platform gave me a warning that my targeting parameters might be “too specific,” it didn’t stop me from setting up the ad.)

So this is how Facebook makes hay using our personal data. Along with paywalls/subscriptions (e.g. San Francisco Chronicle, Medium, New York Times) and donations (e.g. The Guardian, NPR, Wikipedia), selling ads targeted to people’s personal sensibilities is how hay gets made not just on Facebook, but all over the internet. If that means I get to consume ads for dance camps and wetsuits in lieu of celebrity plastic surgery disasters, then everybody wins. (Facebook infers, correctly, that I surf. O’Neill pays Facebook to advertise the wetsuit to me and other surfers, we buy the wetsuits from O’Neill. Repeat. Cha-ching.)

Somehow we got from selling wetsuits to throwing elections. To understand how our current internet failed us in order to frame to where the new internet needs to take us, it’s worth doing a shallow dive into internet history.

A Brief History of the Internet (and Cats)

The internet was never intended to be a money-making machine. In the late 60s and early 70s, large universities wired their computers together in order to share research, primarily through email (of all things) on an early version of the internet known as ARPAnet. Along the way, the DoD provided financing to create DARPAnet. In the 80s, I’m sure sharing cat pictures (uuencoded as streams of text) started to become a thing, if it wasn’t already. Even still, the internet’s only “business model” was government-sponsored academic propeller-spinning.

In 1994, with the advent of the Netscape browser, non-academics flooded onto the internet in droves. Ten years prior, I got my first email account and dial up access from AppleLink. I connected to and explored BBSs and started using protocols like Gopher and NNTP (Usenet). I read up on “netiquette,” learned how to keep my CAPS LOCK key off, how to spot an AOLer (hint: CAPS LOCK USUALLY ON) and how to construct some basic emoticons, something we once called “ASCII art.” |_|] ← That’s a coffee mug right there. Really, it is.

This early internet, on the precipice of becoming commercial, had the feel of a loosely-coupled collection of “expert communities”—for lack of a better term—scattered amongst BBSs, Usenet and AOL chatrooms. (Keep this notion of “expert communities” in mind as you continue reading; I’ll circle back to it later.)

From roughly 1994-2002, companies flocked to the internet to experiment with the web’s first “real” business model: ecommerce. For a few years, it seemed like every business needed a web storefront. However, when investors realized that selling cat food online wasn’t quite what it was cracked up to be, the bubble burst. The same market forces that quickly evaporated five trillion dollars of value also declared Amazon the clear “winner” of ecommerce, proving that centralized inventory (along with on-demand inventory) and centralized technology and fulfillment logistics were the best way—if not the only way—to sell cat food online and actually turn a profit.

After a brief moment of reckoning, from the second wave of the internet—what some call Web 2.0—emerged a new, more indirect business model, this one borrowed from traditional media companies. Like newspapers and magazines, “Web 2.0” sites and applications would also run ads, but insteading of hiring professional photographers and journalists, everyday users would supply the cat photos and write the heartwarming cat stories. Sites like these could save money by letting amateurs create the content—called User Generated Content (or UGC for short)—while they collected money for every cat food ad impression (CPM), every cat photo click-thru (CPC) and every action, e.g. signing up for a site’s feline marketing content or taking a cat survey (CPA).

Naturally, the sites with the most users and the most cat photos (predominantly Facebook and Twitter) could provide the richest ad targeting platforms. Facebook’s claim of making the world more connected belied another mission: to create the richest, most effective ad-targeting platform known to mankind.

(It’s worth nothing that I’m glossing over huge swaths of the ad industry, including search ads/SEO/SEM and scores of networks that serve up ads on third-party sites and mobile applications. I’m also neglecting to talk about the mobile web in general terms, the Semantic Web, the Internet of Things and a whole host of other topics, just so we can stay focused on UGC.)

User Generated Cats

While it has been part of the technology toolkit and lingo for at least 15 years, many—if not most—people heard about UGC for the first time during the recent fallout from the Facebook/Kogan/Cambridge Analytica scandal. Prior to a few days ago, people thought Facebook was free; in reality it’s not. We pay for Facebook by bartering our personal information in exchange for the Facebook features we enjoy.

Perhaps “used to enjoy” would have been better phrasing, since this latest scandal left angry mobs of people joining the #DeleteFacebook movement. In many ways, they’re doing so in vain, because we would literally need to stop using our smartphones and the entire internet, change our names, addresses, hair/eye color, purchase history and a thousand other things to escape the personal data collection happening everywhere on the web.

On Facebook and elsewhere, UGC greases the gears of an enormous machine designed to turn cat photos into cash. And it works, or at least it works for a few massive companies, which seems to be a theme as far as internet companies go.

In fact, at least three times in the brief history of the internet have we seen huge oligopolies create—and consume—entire online business models: Amazon (for ecommerce), Google (for search advertising) and Facebook (for UGC advertising).

Organic growth and the acquisitions by Facebook alone resulted in more than two billion people’s personal information, likes, preferences and social interactions gettings stored inside what is effectively one enormous database.

And that finally explains why this scandal is important: because it has caused people to start asking some really good questions, like: Was it a good idea to allow companies like Facebook to give everyone a free microphone in exchange for harvesting, storing and mining everything everyone says?

It’s Not the Cat Photos; It’s the Cat Distribution

Facebook may be the biggest collector of data, but they certainly aren’t the only one. Plus, they’re not going to delete their data, as it’s the lifeblood of their company. So instead of focusing on Facebook, I want to ask a more fundamental question, one that will surely ignite the ire of free speech advocates everywhere, but one that needs to be asked regardless: Was it even a good idea to give everyone a free microphone in the first place?

Put another way, when is it a good idea—in the real, non-digital world—for us to tell something, instantly, to everyone we know: family, good friends, co-workers, acquaintances, people we just met and immediately befriended? Before Facebook, this wasn’t easily possible. We used to hide our reading materials and journals under the mattress and only send things like baby announcements to everyone we know (even then selectively skipping creepers like Uncle Charlie). Now Facebook has flipped that notion on its head. Your cat photo has more likes than my baby announcement? Does this make any kind of sense IRL? Then why should it be possible online?

But, what about free speech? Yes, in this country we are all free to say nearly anything without fear of repercussion. In another sense, however, speech isn’t really free at all. Our precious free speech is utterly worthless without distribution. Without distribution, our posts on the internet are nothing more than trees falling in the forest with no one around to listen for the sounds they might make. Distribution costs money—and that’s why we strike a Faustian promise with every word and click on Facebook. We provide the content; they provide the distribution. And we pay for the distribution, albeit indirectly, by allowing Facebook to broker our data to advertisers.

Too often and too easily is distribution confused with truth. If something is “widely reported,” that doesn’t make it factual. Therein lies problem with the awesome distribution power of Facebook: It can be used to distribute facts just as efficiently as it can to spread, um, “alternative facts.” As a result, Facebook and Twitter and other UGC sites are heavily moderated both by people and by machines. The other day, Facebook’s censorship robots blocked my friend Tim from saying “trees cause global warming.” Many artists have had their work removed for showing a little too much nipple (or a little too much something). This introduces a whole new set of problems, the most of important of which is: Do we trust Facebook to arbitrate “good” speech from “bad?” Under what or whose standards?

I had a revealing personal experience in 2012 when I helped Miso—a Google-backed venture conceived as a social media site for videos—build an application called Quips. This app would allow people to use their phones to take still images from TV shows and movies and create memes from them by adding the chunky white text we’ve come to associate with such artifacts.

Long story short: we didn’t build moderation (a common internet euphemism for censorship) into the first version of the platform. Rather, we gave people unfettered access to tools they could use to create potentially viral content. What could possibly go wrong? Within weeks, Quips had degenerated into the most profoundly hateful cesspool I’ve yet to see on the internet—and I even (sometimes) read YouTube video comments! Who knew Miso was actually short for misogyny—and racism, homophobia, xenophobia and a million other kinds of hate speech?

It was easy for us to sunset Quips and bury the steaming pile of dreck that Quippers created. It’s not so simple for Facebook.

They certainly can’t delete everything without destroying the data vital to their business model. Meanwhile trying to censor posts is an endless game of algorithmic Whack-a-Mole certain to offend the sensibilities of moles on the far-right, the far-left and every mole in between, including my friend Tim (who doesn’t actually believe that trees cause global warming; it was just a joke).

So distribution without moderation/censorship leads to a cesspool. We technologists all knew this already, but it hasn’t stopped a host of really smart people from trying to build a better moderation/censorship mousetrap. Ultimately they will fail because of (what I can only hope is merely a few) creative individuals with a lot of free time producing a seemingly-limitless supply of garbage. Or art. Or jokes! Sarcasm, something nearly impossible to detect on the web, can often be mistaken for hate speech, especially when the whole point of the sarcasm was to raise awareness of the hate speech in the first place.

When faced with an intractable issue like “stamping out misinformation on the internet,” it helps to reframe the problem by looking at the actual root cause. The cause is not fake news per se, nor ad networks, nor Facebook, Cambridge Analytica nor even UGC. Rather, the naive ideology of the internet coupled with the worst traits in humanity formed ideal grounds for a Tragedy of the Commons: If you create something open and free, some people will eventually find a way to exploit it for their own benefit and thereby ruin it for everyone else.

Emerging from the Cesspool

Even though it’s likely a very small segment of “bad actors” who are ruining the internet for everyone, I’m proposing a radical shift: let’s leave the internet for what it is (a cesspool) and build a better one. What if we could start over with the same lofty goals—connecting the world by sharing information—but this time build an internet with failsafes that would prevent us from creating yet another cesspool of misinformation and hate speech?

I’m not suggesting that we shut down the internet, but instead that we build something atop existing protocols that helps the world organize information, validate claims, and establish fact; in other words, we need to build an internet that lives up to its early design considerations, which, obviously, did not include building a cesspool of falsehoods and hate speech.

A recent NYT article really drove this point home for me: “The downgrading of experience and devaluing of expertise can be explained partly by the internet, which allows people to assemble their own preferred information and affords them the delusion of omniscience.”

Note it said “partly.” The internet is partly at fault. Humanity bears responsibility for the rest.

So yes, humanity is a big part of the problem. But it’s also the solution. For every bad actor, there are thousands and thousands of good ones.

What if we could build an internet wherein good actors could drive out bad?

What if we could create an internet consisting only of factual information? An internet devoid of corporate interests? An internet of real people wherein everyone could only interact with the system using a proven identity?

What if we could finally draw the line between private and non-private digital communications, such that private conversations could remain truly private?

What if all information was organized into siloes, like the “expert communities” of the early internet, but codified into a meritocratic hierarchy where every claim needed to be vetted by an established community of experts? What if experts could delegate privileges to other experts who prove their worth through contributions? What if the information curated remained free to the consumer, but provided a basic income to its creators and gardners for the work they put into curating the information? What if this internet could remain completely read-only to everyone not designated an expert in a particular silo?

Web X.0

Much of the technology we need to build something like this already exists. Signal, Keybase and scores of other platforms offer peer-to-peer (serverless) encrypted messaging. StackExchange already provides a model for curated expert communities, entirely based upon Q&A. Modeling the new internet off of StackExchange (or Quora or WhySaurus), each question response could be stored as a block in a blockchain with experts from the appropriate communities recruited to validate the responses, much like block validation already works today for cryptocurrencies.

Every information silo would require a community of experts to curate it. But what good are these experts if we can’t check their credentials and contributions to validate that they really are experts? The missing piece here is global identity management, i.e. a way to prove that we are who we say we are. We need a biometric-seeded revocable cryptographic key that would allow us to conduct business using our IRL identities or with pseudonyms that the owners can prove are theirs (but not the other way around). The Human Unique Identifier (or HUID) described by the ambitious Cicada Project proposes a clever design for this.

Creating a secure, un-spoofable identity system is a fundamental challenge, but it’s surely not the only challenge. In building this new internet, our biggest enemy is what we don’t know—and what we won’t know until we we’ve already written oodles of code and tests, as is often the case with software projects.

But we can’t let fear of the unknown stop us. The time has come—in fact it’s long overdue—to create a new internet, an internet that can’t be defeated by Nigerian scammers, Russian fake news bots or that 400-pound kid in his bed somewhere. Let’s leave the existing internet intact but teach our kids that they should assume that nearly everything they read there is either bullshit or sponsored bullshit. If vetted, cite-able, factual information is what they seek: They need to consult Web X.0.

And yes, this new internet would be read-only for 99.9999% of the world’s population. This would leave about 7,000 experts in control of all the world’s public factual information, with the ability to delegate more experts as needed. No corporations would be allowed; no corporate interests would be tolerated. In this way, the denizens of the new internet would maintain all the world’s information much like the denizens of the early internet “expert communities” on BBSs, Usenet and chatrooms, but this time with HUIDs and block validation keeping everyone honest.

People could still interact with corporations on the “old internet,” but we could use the Web X.0 HUID to doll out Basic Attention Tokens (or something like them) to allow people to decide for themselves which revocable personal information they want to share with commercial entities—and get compensated with cryptocurrency in return. In other words, corporations would pay consumers directly for paying attention to their messages, eliminating the layers of ad network middlemen who get paid for matching companies to consumers.

The Cicada Project takes this a step further by adding a secure direct democracy component, which would allow populations small and large to self-govern. Direct democracies are notoriously disastrous (e.g. Athens) but given that two of our last three presidents took office despite losing the popular vote, maybe is an idea worth considering again.

Then again maybe direct democracy is biting off more than we can chew. Maybe we should start by building and deploying the HUID on the existing internet and then go from there.

Maybe this is all hogwash.

But maybe—thanks to Facebook, Kogan and Cambridge Analytica—we’re finally starting to ask the right questions.

An Unlikely Cure for Procrastination

“It always seems impossible until it’s done.” —Nelson Mandela

We all have tasks that—for whatever reason—we just don’t want to do.

They might be as mundane as organizing the garage or as grandiose as building the next Facebook. Small or large, easy or complex, self-rewarding or based on the obligations to others; regardless of what needs doing, I noticed something recently that consistently helps me break through cycles of procrastination and stay focused on the tasks that matter.

My “ah-ha” moment of introspection about procrastination came when a coworker said, “I’m addicted to working on this project.”

I didn’t doubt that he was telling the truth. People have been addicted to far stranger things than software projects. But the remark made me wonder: Can I improve my productivity by channelling my inner addict?

The answer was a resounding yes. I use and re-use “addiction training” (for lack of a better term) any time I find myself resisting some task that I don’t want to perform.

In order to understand why this works for me—and may also work for you—we need to understand how someone becomes addicted. The word addiction carries with it some serious baggage. Everyone knows how dependence on hard drugs or alcohol can lead to financial and emotional ruin, the destruction of relationships and sometimes even death.

Most people also know that addiction is not a character flaw; rather a person’s brain chemistry changes related to how “rewards” get processed. A shallow dive into neurology explains the chemical nature of addiction, beginning with the prefrontal cortex, a region of the brain associated with logic and decision-making. At first, we consciously set “goals” of getting drunk or high (or working out or having sex) because those things feel good. After a relatively short period of time—with some drugs, just a few doses or with “good” habits, some say 21 days—the motivation to continue the nascent behavior moves from a logical, conscious place to a more Pavlovian one. A new part of the brain takes over: the anterior dorsolateral striatum, wherein we process rewards-based learning.

“In rats seeking cocaine, additional evidence supports the hypothesis that seeking behavior is initially goal-directed, but after extended training becomes habitual and under the control of the anterior dorsolateral striatum (aDLS).” [source]

Once the aDLS has taken over, addicts will feed their addiction at all costs, even if they can knowingly reason that “smoking is unhealthy” or “alcohol is ruining my life.” It’s literally beyond their logical control.

The chemistry of addictive drugs, stimulants in particular, facilitates the transition of using drugs from “goal-based” to “habitual.” But how does this apply to my software project—or cleaning my garage?

Here’s what I do when I find myself procrastinating:

  1. Set up an extremely small reward challenge (to trigger the aDLS), e.g. “I’m going to install RVM/ruby and create my Rails project, then I’m going to have a bowl of ice cream.”
  2. Do the extremely small task. (Okay, that was easy and it took less than five minutes.)
  3. Eat the ice cream. (That felt good.)
  4. Go back to procrastinating.
  5. Repeat.

By associating the smallest level of effort with a reward, we can begin to trigger the reward processing module of our brain, effectively feeding our nascent addiction. (Bonus points for substituting “eat a bowl of ice cream” with “go for run” or some other healthy habit.) After repeating these steps several times, you’ll likely find yourself autonomously attracted to the work you logically don’t want to do. There’s a lesson to agile product owners here too: Stories reduced to the smallest atomic parts can give developers little “slam dunks” wherein the reward is baked into the process of moving the story along the agile board.

It’s important not to create additional negative addictions during this process—and equally important to keep the aDLS on its “toes.” Give yourself a huge reward for doing very little. Then give yourself a small reward for doing something huge. Sometimes, give no reward. Or flip a coin and if it’s heads, eat the ice cream; tails: Go back to work! This “random” nature of the rewards helps cement the working addiction using ideas from something (anecdotally) more addictive than cocaine: gambling.

This method for training an addiction might work better for some than others. One study claimed that 47% of the population carried a genetic marker for addiction. Even so, we all have an aDLS and we can all learn to train it to our advantage.

Having trouble exploiting your addictive tendencies to become more productive? What other techniques have you tried when you need to break out of a procrastination rut?

The Quest for Better Tests

Over the past twenty years, I’ve written my fair share of unit tests, mostly just covering the happy path and sending in some bogus inputs to test the edges. Typically following a fat-model-thin-controller method (often recommended by me), I failed to understand the point of integration tests. I tried TDD at the beginning of several greenfield projects, but I was never successful in making it sustainable. Similarly, with Selenium, it worked at first but quickly proved to be too brittle to keep up with rapidly changing UIs. (In retrospect, bad CSS architecture on those projects probably deserved the blame more than Selenium per se.)

Despite my somewhat lackluster attitude toward testing, my employers and customers knew me as a big advocate for test automation—who always insisted that we never release anything to QA without at least a layer of unit tests. Oftentimes I was overruled by more senior leadership. As expected, from time-to-time, we all got burned by bugs that would have been easily caught by more comprehensive tests. We swore we’d write better test next time. But for a million reasons—”speed to market” being the peskiest of them—testing never became a consistent priority.

In February of 2016, I joined Lab Zero. The very first observation I made after starting on a project—a financial services application three years in the making—was the sheer volume of test code. Nearly everywhere I looked, I found at least a 10:1 ratio of lines of test code to lines of “real” code. Shortly after starting on my first story, it became readily apparent that at least a 10:1 ratio of developer effort was required to continue this pattern. We joked about developers who reported their status during daily standup by saying, “I’m done with the code and just need to write some tests,” because we knew that was a euphemism for being less than 10% done with the story!

It didn’t take long before realizing how much catching up I needed to do. In fact, the project leader told me it would take me “a year” to learn how to test properly. After first thinking that he sounded condescending, I came to realize that he was just being realistic. Testing is hard; testing effectively is even harder.

Ten months into my Test Quest, here are some important lessons I’ve picked up about automated testing.

Note: I used Ruby, Rspec and Cucumber to create my code samples, but the lessons learned will likely apply to other ecosystems.

The myth of 100% code coverage

Sure code coverage an important metric, but one that only tells part of the story. Test coverage is not the same as good test coverage. It’s remarkably easy to write tests that test nothing at all, that test the wrong things or that test the right things—but in ways that never fail.

Consider the following example, wherein the remove_employee method has a glaring error, one that will easily be caught by a unit test. Or will it?
class Company
  def initialize
    @employees ||= Set.new
  end
  def add_employee(person)
    @employees << person
    @employees.size
  end
  def remove_employee(person)
    @employees.size - 1 #danger: incorrect implementation!
  end
end
RSpec.describe Company, :type => :model do
  let(:subject) { Company.new }
  describe 'managing employees' do
    let(:person) { double(‘person’) } 
    it ‘removes an `employee`’ do
      employee_count = subject.add_employee(person)
      expect(subject.remove_employee(person)).to eq(employee_count - 1)
    end
end

Because the test for removing employees naively compares only the outputs of the add and remove methods, it passes with flying colors even though the remove_employees method internals are totally wrong.

And this why it’s a good idea to…

Test internals instead of just inputs and outputs

In most—if not all—programming languages, there are many more ways to produce “outputs” than just the return values of method calls.

C/C++ developers can optionally pass primitives to functions by reference (e.g. int &param1), morphing those inputs into potential outputs. More modern languages restrict everything to pass-by-value, but most of the time what’s being passed “by value” is actually a reference to an instance of an object. As a result, it’s possible—and quite commonplace—to mutate the object instance itself in the context of a method, providing another sneaky way for methods to have unexpected “outputs.”

Unfortunately, testing internals can be challenging, but it doesn’t have to be.

Design and write testable code

A previous version of me believed that only a very limited set of circumstances should trump writing elegant code. I recently relaxed this constraint, adopting the belief that it’s okay to over-decompose code (and make other code design compromises) in order to serve the goal of writing code that’s more testable.

For example, I might replace a simple, elegant call to a setter with a method that wraps it, e.g.:

shape.color == :blue

vs.

def is_blue?(shape)
  shape.color == :blue
end

In the past, code like this would make my eyes bleed. However, it’s really easy now to stub out is_blue? so that it returns a mock object or performs some other test-only behavior.

This is a contrived example, but if figuring out if a shape is blue required a database read or a call to an underlying service object, then over-decomposition like this is small price to pay to make the code testable.

Test incrementally

I’ve found TDD (specifically a test-first methodology) to be overly prescriptive, usually leading to diminishing returns as the project gets more complex. If it helps clarify the specs and define edges more easily, then by all means, write tests first! However, I’ve found more productivity (and less head-scratching) comes from writing tests not necessarily first, but in short iterative bursts.

Every time I finish an “idea” in code (for lack of a better term), I switch over and edit the test, usually already open in a split-screen view next to the code. If the “idea” is too complex, I take a step back and flesh out more tests to help me clarify what I’m trying to accomplish in the code.

In the past I’ve also worked in a pairing setup where I wrote the code and switched back-and-forth with another developer writing tests. Though I haven’t done this recently, it’s another technique that’s worked well for me.

DRY code, wet lets

Don’t Repeat Yourself (DRY) is a great rule-of-thumb for writing code, but it can be disastrous  when memoizing test data, e.g. through calls to rspec’s let or let!

With the exception of some truly global concepts (e.g. user_id), all test data should be initialized in close proximity to (read: immediately before) the tests that use it and should not be reused between unrelated tests.

Thinking I was helping, I tried to DRY-up some lets, soonafter realizing that I had no idea what test data was getting passed to what tests. Even it feels cumbersome to repeatedly initialize the same data over and over before each test, it’s the right thing to do.

Re-use Cucumbers with Scenario Outlines

Unlike lets, some parts of the test ecosystem are actually designed for reuse. One example: Scenario Outlines. I recommend using these whenever possible.

With Cucumber, Scenario Outlines represent the “functions” in an otherwise functionless DSL. In addition to the obvious reduction in code bulk, thinking about how I can turn several tests into one test “template” helps me write more thoughtful, self-documenting tests.

Vary only what needs to be varied

It’s tempting to cut corners (and make tests run more efficiently) by favoring randomizing test data over creating different tests for different values. Often this practice is harmless, especially if the specific values—as long as they’re in range, e.g. a person’s age—are inconsequential. (If specific values matter, e.g. people 65 and over get medical benefits, they should of course get their own explicit tests.)

Randomizing test data can also be a trap. For example, a test for a get_birth_year method might start to “flicker” or “flap,” meaning that it passes and fails non-deterministically between test runs—all because of the decision to randomize ages.

To protect against this, it helps to treat each test as a controlled experiment, i.e. by keeping the scientific method in mind. Try to control everything that can be controlled and vary only the specific inputs getting tested. Of course, there are things we can’t control, like the system clock, the speed of the network and the availability and behavior of upstream systems. But whenever things can be controlled, control them.

Write meaningful, descriptive test names

Acknowledging the fact that I just recommended thinking like a scientist, I’m now going to suggest putting on a writer hat. When naming tests cases and writing Cucumber steps (which read like prose already), it’s super-important to be descriptive, concise and accurate.

In a place full of smart people like Lab Zero (#humblebrag), developers are not necessarily the only people looking at tests. Recently I had an agile product owner ask me how a certain feature handled different types of inputs. To answer the question, I walked him through my rspecs, reading each test name aloud and describing the expectations.

Writing coaches always say “show, don’t tell.” There is simply no better way to show—and prove—that a feature works than reading through the tests, which serve as the closest link between the specs and the code.

Putting the Science in “Computer Science”

One of my professors in college said that any discipline that has the word “science” in it is actually not a science. This is especially true for computer science, something that at some schools classify as a fine art (making it possible get a BA in CS). Writing code is a certainly a form of communication, at least to peers and future developers. Of course, they are not the customers. And the best way to “communicate” with customers is to provide something for them that works as designed.

How do we ensure that? With well-written tests.

Tests really put the science in computer science. Think of them as a series of carefully controlled experiments. The hypothesis is that the code implements the spec.

Without tests, there’s really no way to know if it does or not.

* * *

Originally published on Lab Zero’s blog.

I Made my Wife a Bot for Valentine’s Day

This morning I rolled out Tink, a simple interactive chatbot I wrote for my wife as a gift for Valentine’s Day.

Every few days, Tink will text my sweetie a randomly-selected yes-or-no question from a list of questions I wrote, e.g. Would you like to take hip-hop classes? At different random times, it will also text me random questions from the same list. When we both reply “Y” to the same question, it will notify us of that happy coincidence and suggest that we, say, finally enroll in those hip-hop classes.

Basically it’s Tinder, but for couples. But not in the way you’re thinking (you dirty dawg).

Instead it’s a fun way for two romantic partners (or just friends?) to discover shared interests they didn’t know they had. I suspect Tink will also become a motivator to actually do the things it suggests. (We’ve been meaning to sign up for hip-hop classes for months, but haven’t yet.)

The questions I wrote for Tink’s inaugural run mostly revolve around ideas for fun dates, outdoor activities, new restaurants we want to try, etc. However, there’s no reason why Tink questions couldn’t cover religion, politics, sex—or even topics actually fit for the dinner table.

With G-rated questions, Tink could serve families or even small friend groups, but right now it’s only a bicycle built for two.

Wanna take a peek under the hood? I made Tink opensource under the MIT license.

Five First Impressions of LabZero

I recently joined Lab Zero as a software developer. My friend Brien Wankel, one of their Principal Engineers, had been encouraging me to interview here for more than a year. I hesitated because, to put it bluntly: What’s so special about another boutique software development agency? There are hundreds—if not thousands—of them in the Bay Area. Plus, I was still trying to strike gold playing the startup equity game and I had already run my own boutique software development agency for a decade.

At long last I took the plunge, and I’m really glad I did. Ten weeks in, these are my first impressions of Lab Zero.

1. “We pay for every hour worked, no exceptions.” —The CEO

Lab Zero’s culture in three words: “Life, then Work.” Everyone here, myself included, is a W-2 hourly employee. To prevent people from worrying about using PTO when they’re sick (which eats into vacation time), we’ve done away with the concept altogether. We get paid for every hour we work—and we don’t get paid when we’re not working. That also has the side benefit of discouraging people from coming to work when they’re contagious. As a substitute for PTO, we accrue personal/family sick time, bereavement and jury duty time.

Employment here includes all the usual benefits, but without the attached expectation of working 60-80 hours/week (or more) and getting paid for 40. I surf every Wednesday morning (if the weather conditions cooperate) and I volunteer at my daughter’s school in the afternoon. I might only bill for 4-5 hours on a Wednesday. I might put in a few more hours after dinner—or not.

I haven’t put this to the test yet, but I may need to scale back my hours at Lab Zero by 50% or more to run tech for another political campaign or to get more involved in the farm-to-table movement or maybe to start a side business—or not.

2. “We follow software best practices.” —Everybody

So we put life first and work second. But does that mean that we don’t care about what we do? Hells no!

Lab Zero embraces a documented set of methodologies that make great software development possible, if not pleasurable. We have 100% or near-100% test coverage on all our projects; we write unit tests, functional tests, automated UI tests—to the tune of roughly ten lines of test code for every one line of “real” code. We practice continuous integration; we have a stringent pull-request review process and we reject pull requests for even the slightest blemish, e.g. a typo in a commit message.

This culture of doing things right at all costs may sound too onerous to be practical, but what I learned after a couple weeks here is that the effort we put into rigorous testing pays us back in spades, measured by the very small number of issues that slip through the cracks, eventually needing to be caught by QA or found in production. Plus, as long as I can keep the test suites passing, I can refactor without fear that I’m going to break something.

And if I do break something incidentally, it usually just means I need to write a better test, which in turn will help overall quality in a virtuous cycle.

3. “We do Agile really, really well.” —Our Customers

Agile prides itself on being agile, per se. (How deliciously meta is that?) Take what you want, leave the rest. As a result, there are infinitely many ways to do agile well—and an equally-indeterminate number of ways to do it badly.

Last week, I heard a senior executive at one of our customer sites tell us (in front of a room of twenty people) that we were the gold standard for agile projects at their organization. Enough said.

4. “We care about having a beautiful, functional workspace.” (And it shows.)

We have top-shelf coffee, great snacks and drinks, a loaded kegerator, automatic standup/sitdown desks (each with four presets), Apple Cinema Displays, an office sound system, massive TVs, stylin’ chairs and Fluid Stance boards. If you need anything, within reason, it just shows up at the door.

In addition to that, Nicole Andrijauskas just finished painting an amazing mural spanning the entire south wall of the office.

We have catered lunch-and-learn sessions every other Friday. On the alternating Fridays, we descend in a hungry mob to a local restaurant (like Barbacco, this past Friday) and Lab Zero picks up the tab. In addition to Fridays and the regular bevy of snacks and beverages, there are also bagel Wednesdays, eclairs one day, coffee cake another, etc.

As much as I love our office, I also love my half-time Wednesdays working from home (and/or the beach). Which is totally fine, of course. I’ve even been finding a leftover bagel or two on Thursday morning for me.

5. “Diversity is woven into the very fabric of our culture.” —Me

The notion of full-time employment does not preclude hiring people who rawk at things besides their profession, but employers don’t explicitly benefit from it either.

At Lab Zero, where life comes first—and turnover is near nil—we’ve built an eclectic mix of developers, designers, writers, agile product owners and bizdev folks who double as parents, recovering chemists, musicians, surfers, teachers, artists, marathoners, photographers, LGBTQ folks, future real estate moguls and one of the world’s leading experts on tiki.

There’s no better testament to Lab Zero’s people than this: I could do my job almost exclusively at home. I could also bill an extra two hours instead of commuting to downtown SF from the North Bay. But I actually want to come to the office.

Ten weeks in. Zero regrets. Can you say this about your job? If not, maybe you should join us for lunch.

Lessig 2016: The Internet’s President

On the heels of Larry Lessig’s historic announcement of his Referendum Candidacy came another newsworthy item: Jimmy Wales announced that he is chairing LECEC, the Lessig Equal Citizens Exploratory Committee, which consists of me and scores of other volunteers and staffers working at a breakneck pace to make Lessig’s nascent campaign a reality.

At first blush, the Wales announcement might seem like a footnote on an afterthought, but I read something very different into it. Something that reminds me of Paul Revere. But no British are coming this time. Instead, Wales penned these nine simple words and so began the largest, most peaceful, and most desperately-needed democratic revolution in human history:

When you light up the Internet, anything is possible.

–Jimmy Wales, Chairman emeritus, Wikimedia Foundation

That’s right, Internet: He may have understated it a bit, but Jimmy Wales just asked us to mobilize and elect Lessig in 2016.

Let’s face it: No one has been kinder to the Internet than Larry Lessig. In the twenty-plus years I’ve followed his work, he’s rallied against outrageous software patents, fought copyright takedowns, drafted the contracts that legally protect and enable “open source” software (which powers most of the Internet), started the Center for Internet and Society at Stanford, fended off aggressive corporate and government entities in his quest for net neutrality and, naturally, he’s spoken at TED. Four times.

Larry is and has been the Internet’s dedicated steward for the better part of his career. His body of work enabled the Internet to become what it is today: a loosely-coupled network wherein heterogeneous data, applications, and systems play in an ecosystem with minimalist governance atop a tiny handful of protocols and specifications.

No one’s been a bigger advocate for the open internet than Larry Lessig.

–Cenk Uygur, The Young Turks

If Vint Cerf, Donald Davies and Bob Kahn are the “fathers of the internet” and if Aaron Swartz was “the internet’s own boy,” then Lessig is the Internet’s uncle.

What better way for the Internet to say thanks to Uncle Larry than to use its enormous catalytic power to hack him into the White House?

And who better to do the hacking than the hackers of our own generation? We were born into a world with basically no connectivity to information, services or people. Today, we have access to pretty much every other connected person and all of the world’s public digital information (so long as we can keep our phone batteries alive).

As software developers in the mid-90s, we didn’t just witness the explosive, hockey-stick growth of the commercial internet; we actually built huge swaths of that reality ourselves. My peers and their peers built out the data centers and server racks, the gateways and firewalls, the routers and switches, the firmware and software platforms, the web servers and middleware, the web services and mobile applications — that drew billions of people into a web of inter-connectivity, knit together so tightly that a single thread can be spun half way around the world and back again in just seconds.

As my peers and I built PayPal and Yahoo! and Oracle and Amazon and Google — and the millions of companies that weren’t as fortunate — we didn’t realize this tremendous side benefit:

We created the most powerful agent of social change in the history of humankind.

In 2012, when Lessig and Swartz “lit up” the internet to defeat the SOPA and PIPA bills, scores of sites — including Wales’ Wikipedia — “went dark” in protest of these hair-brained bits of legislation.

That was the battle. This is the war.

Today we are at war with a much darker evil, far more insidious than the foiled attempts to reign in and regulate the internet. Our government has become the handmaiden of the “funders” — billionaires, PACs, multinational corporations, labor unions and other special interests — and we are fighting to restore a representative democracy back to the citizens to whom it was promised.

We are fighting “the root of all evil,” the darkest evil with the deepest pockets. And we’re already in way over our heads.

But now, Jimmy, we’re gonna light up the Internet.

Darkness cannot drive out darkness; only light can do that.

–Martin Luther King, Jr.

If there exists one light great enough to drive the corrupting influence of money out of DC, it’s not burning torches and gleaming pitchforks. Rather, it’s the glow of a hundred million mobile phones, tablets and laptops, mobilized under a single, peaceful mandate. With every Facebook like, every share, every re-tweet, every blog post, every comment, every volunteer effort and every donation, the light spreads and burns ever more strongly, driving out the darkness.

Wales said:

When you light up the Internet, anything is possible.

I say:

Lessig for President in 2016.

Wales said anything. And it’s our job — as denizens of the Internet — not to prove him wrong.

On Open Letter to the Java Community

java_oracleIn the wake of the Sun acquisition by Oracle, the much-lambasted Oracle vs. Google lawsuit over Google’s alleged JavaME patent infringement, and the rumblings I’ve been hearing at Oracle Open World / JavaOne / Oracle Develop 2010, I have a message to the Java community:

Quit your bitching and moaning and start doing something productive!

Now that I’ve offended all the Java fanboys/girls out there, let me explain:

  1. Why I’m qualified to give you all one big collective kick in the ass, and
  2. Why this collective ass-kicking is coming from a place of love, not hate.

My first experience with Java was in 1994/95, when Stanford started switching its Computer Science curricula from C/C++ to Java. After struggling with memory management, segmentation faults, horrific concurrency problems and the other ways I kept shooting myself in the foot, Java was a breath of fresh air. My first corporate experience with Java was working as a summer intern for JavaSoft (a former subsidiary of Sun) in 1997 porting Patrick Chan’s Java 1.0 sample applications (remember Hangman?) from JDK 1.0 to JDK 1.1.

I went on to join Plumtree. Originally, they were a Microsoft darling. I helped lead the charge to switch them from COM/DCOM, ASP 1.0 and SQL Server to Java and Oracle.

In 2002, I started a Plumtree-focused consulting firm, helping 50+ customers install, maintain and grow their Plumtree deployments. In all but a precious few of those accounts, I wrote all of the code in Java/JSP.

Since about 2008, we’ve been using Ruby on Rails for most of our software. When Rails hit the scene, I had a similar “breath of fresh air” moment similar to when I first encountered Java.

But this letter is not about Ruby or about Rails; it’s about Java. A language I’ve used since it’s very first iteration in 1994/95 and up to the present day. A language wherein I’ve written at least half a million lines of code, most of which still run in production today inside Plumtree/AquaLogic User Interaction/WebCenter Interaction, at major customer sites in the corporate world and in the federal government.

So, fast-forward to today, this is what I’m hearing about Java, in a nutshell:

  1. Oracle’s going to kill/close-source/fuck up Java
  2. Life’s not fair!
  3. Blah blah blah

Twitter _ Jock Murphy_ @oracle I love Java, I do .All of this bitching and moaning starts right at the top with Java grandfather and CEW (Chief Executive Whiner) James Gosling, who is showing incredibly poor leadership, lousy judgment and massive immaturity with his totally irrelevant, outdated and hateful anti-Oracle bitch-fest.

Twitter _ Marcello de Sales_ Solaris 11 to be contI’ve heard people whining about everything around them that’s not running on Java: mobile applications, web sites, conference tools, Twitter, Facebook, etc.

Twitter _ Paweł Szymczykowski_ @dendro Awesome thaI even saw someone complain on Twitter that the Black Eyed Peas, who Oracle paid an undoubtedly handsome sum of money to entertain your sorry asses last night, gave a shoutout to Oracle and not “The Java Community.” Seriously? Give it a rest, folks!

There are lots of choices of development stacks and people are free to choose the one that works best for them.

Embrace that freedom; don’t fight it.

And the word Oracle doesn’t mean “database” anymore. It is an umbrella term that could refer to thousands of different products.

Let’s take a look at some of the advantages of Oracle owning Java.

With respect to OpenWorld, the Java Community got:

  1. Your own conference with around 400 sessions
  2. Your own tent
  3. Your own street closure (Mason Street)
  4. Invited to OTN Night, one of the best parties at OpenWorld

More importantly, with Oracle Corporation, the Java community gets:

  1. Cemented into the infrastructure of nearly all of Oracle’s products, meaning that nearly all of their customers — most of the Fortune 1000 — are now Java shops (if they weren’t already)
  2. Stability, stewardship, thousands of really bright engineers and nearly unlimited resources
  3. One of Corporate America’s most powerful legal teams backing you up
  4. A secure and promising future, including a just-announced roadmap for JDK 7 and 8

And, with all that being said, guess what?

Java is still open source.

Do you know what that means?

Let me answer that question with another question: what brilliant phoenix rose from the ashes of the debacle that was the AOL acquisition of Netscape in 1998?

It was Firefox, a free, open source-based browser that literally revolutionized the massively screwed up browser market and gave the dominant browser (IE 5, and later, IE 6) a true run for its money. From wikipedia:

“When AOL (Netscape’s parent) drastically scaled back its involvement with Mozilla Organization, the Mozilla Foundation was launched on July 15, 2003 to ensure Mozilla could survive without Netscape. AOL assisted in the initial creation of the Mozilla Foundation, transferring hardware and intellectual property to the organization and employing a three-person team for the first three months of its existence to help with the transition and donated $2 million to the foundation over two years.”

IBM’s symbiotic relationship with Eclipse is another great example.

So, dear Java community, to ensure your own survival, please, in the name of Duke, stop complaining and start thinking strategically about how you can “pull a Firefox” here. You’re all brilliant engineers, so start putting all the effort you’re wasting in complaining toward something productive.

I love you all and I love all your passion and energy, but I hate your bitching — use that energy to go save the world, Java style!

Shameless Self-Promotion

panel_picker_voteAt last year’s SXSW I said to myself: “Self, you need to be speaking at this conference next year.”

Help me fulfill my self-fulfilling prophecy and please take a minute to vote for one (or both) of my proposed talks! Unlike the SXSWi Web Awards last year, you don’t have to vote every day — once is plenty.

The first is a solo presentation on the future of RSS.

The second is a panel discussion on whether it’s better to have one horizontal social network like Facebook or loads of smaller, niche social networks.

Thanks for your support.

Chris Bucchere Speaking at the NovaRUG on June 18th, 2008

Calling all local Rubyists! I’m speaking about modular page design in Ruby on Rails at tomorrow night’s NovaRUG. The title of my talk is “To Portal or Not to Portal: How to Build DRY, Truly Modular Mashups in Rails.”

The meat of my talk is going to come from these two recent blog posts:

Modular Page Assembly in Rails (Part 1)
Modular Page Assembly in Rails (Part 2)

I’ll be followed by Arild Shirazi of FGM giving a presentation entitled “CSS for Developers.”

Get all the details here.

P.S.: Free pizza!