Thursday, April 2, 2015

Who will own the virtual network?

There are two network virtualization products being pushed by heavyweight vendors. In this corner: VMware NSX. In that corner: Cisco ACI.

The debate is ongoing how to add agility to networking. The overlay model, like NSX, says that the network can be simple as long as the apps are smart. The controller model, like ACI, says that the network should be aware of the apps.

On the overlay side, the strength is in its deployment simplicity: end nodes communicate with each other, but the network itself (the "underlay") doesn't require any adjustment. Its weakness is the flip side: there's no way to tune the underlay, so there's no way around bottlenecks and other potential artifacts of the architecture.

On the controller side, the strength is the potential of the network to adjust to support whatever flow patterns the app requires. There are two weaknesses: first, if the app doesn't integrate with the controller API, then the controller has to interpret the data it's carrying and make a best guess on optimal configuration; second, the "smart" abilities generally require new hardware, which greatly slows down the rate of adoption.

In the short term (1.5-3 years), it's likely that NSX will be deployed much more than ACI. NSX doesn't "technically" require the server team to involve the network team in the roll-out, and the agility that's grown from virtualization into modern devops will lead both to an increased tolerance for risk and a demand for better network delivery. (As a network guy, I'm used to hearing the network always getting blamed for any problem.) The fertile ground will be large applications hosted either classically or in private cloud, especially where the app is multi-tiered (with lots of local east-west traffic), but may grow as NSX support increases across data centers.

The question is whether ACI will take hold, and where. The fertile ground will likely be single-owner data centers with monetized services, where ACI will be seen as an investment to increase capacity, and API integration can be pushed top-down into the hosted systems. However, the Internet Giants (Google, Facebook, etc.) have demonstrated that they're willing to create their own network technology rather than purchase big-ticket vendor products, and analysts and pundits are declaring that everyone else will move to cloud hosting. Even though that prediction is hyperbole, it does put downward pressure on the ACI target markets.

Keep in mind that 3 years ago, the "future of networks" was OpenFlow, so any future predictions that aren't wildly inaccurate will likely be due to blind luck. :-)

What do you think? That's what the comments are for down below.

Monday, August 5, 2013

A sonnet about the TSA

This was my entry for the Liquid Matrix Podcast contest for a pass to BSides Las Vegas. I didn't win, but wanted to share it with you.

The Iron Wisdom of the TSA
inspired by a Vendor DERP from Liquid Matrix episode 0x2C

The airline tells us that their onboard staff
are there for safety, not for in-flight ease.
And yet SouthWest will brief us with a laugh,
and once airborne, serve beverages to please.

But safety on the ground from TSA:
before you soar, they make your spirits sink.
Remove your shoes, your liquids throw away!
No wonder on the plane you need a drink.

Your courtesy is answered with their scorn,
suspicion is delivered without tact.
So cover up, the scanners churn out porn!
There's theater security to act.

The mission of the TSA thus laid:
"Security" keeps passengers afraid.

Monday, June 3, 2013

Want innovation? Reduce the cost of education!

Today's post is inspired by the discussion over US visas for foreign workers, and the argument from high-tech companies that there aren't enough new US grads in STEM (Science, Tech, Engr, Math) to fill the open positions. Specifically, I'm reacting to the "data" in this article, which says what we've all suspected for the last 15 years: US companies can't find qualified people who will are willing to work for what the positions pay.

My gut reaction (no data to back this up other than personal experience) is that new US grads have a disadvantage that a lot of foreign grads historically haven't: student loan debt. If a new US grad leaves school with $50k-$150 in student loans, that's a strong disincentive to take risks. Instead, a monthly loan bill will demand that students quickly find a reliable source of income. Additionally, cost of living in traditional areas where STEM companies congregate is high. In my case, it was the Silicon Valley in 1996, when I was "lucky" to pay $1200/mo for a 2-bedroom apartment.

The "mythical" Silicon Valley startup mentality had the motto, "work any 80 hours a week you want," and didn't care what your education level was as long as you got the job done. Now, startups are less work-intensive, but early stage positions still offer very little income. If these companies want to attract the "best and brightest", one way to do it is to increase applicants' openness to risk.

A relatively recent idea has been the "hacker hostel" model, in which startup incubators come with not only office space, but sleep space. While that reduces the monthly expenditure for a new hire, it has its own disadvantages, including being appealing primarily to early-stage companies (read: the co-founders are the ones sleeping on the couch).

A more long-reaching solution would be to drive the cost of education back down. Forbes reported in 2012 that education has risen 500% since 1985, while the consumer price index was only 115%. There's a pretty picture here (note: data from same source). While there was some re-structuring a couple of years ago, in which some colleges switched from mixed financial aid to grant-only, that story has fallen out of the media, and costs haven't really been affected. Furthermore, the global economic downturn has reduced spending on eduction (at least in the US), driving tuition & fees higher, even in state schools.

A reduction in the cost of education is a likely driver to innovation specifically because it will reduce the p2p driver that killed so many startups. In business, p2p for a VC-backed firm was "path to profitability", insisting on rapid repayment of investment by bringing products to market early. In education, I'm using the term as "path to payment", repayment of student loans. If we want graduates to work for less money, then let's make it possible for them to do so.

Tuesday, January 22, 2013

Urgency and activity

I've noticed that I tend to get work done best when I'm nearly out of time to do it. Presumably, I'm not the only one who works like this, but it definitely causes... let's call it "friction" with the people I work with who seem to work at a constant pace.

I wonder sometimes if my life would be different if I could maintain that constant pace. Given my irregular output, I see myself as a miracle worker who would thrive in an environment of being called in to get stuff done in an emergency. However, my co-workers tend to see me as unreliable, which has occasionally been career-limiting.

A few years ago, the team I was part of was introduced to Scrum, an Agile methodology. At the time, we were told in training that one of the goals was to even out the crisis humps, replacing the "mad rush" at the deadline with a series of smaller deadlines, so there'd be no all-nighters to get things done. Looking back now, I wonder if it was a compromise to try to get "bursty" workers to increase their output, or at least make their output more predictable to schedule-keepers.

Driving home today, I wondered about what a crazy frenetic work environment would be like, in which goals for the day were handed out in the morning. After a while I know it would be maddening, and that very few people would survive working there. However, if they could put the right team together, imagine the output!

Some of this thought is driven by just having read The Phoenix Project, a novel about DevOps. It's a great read, and if you're an Amazon Prime member with a Kindle, you can "borrow" it for free. That's what I did before buying it. :-)

What it got me thinking about is how to overcome "impossible" challenges. In the book, one such challenge is going from a deploy every 9 weeks to a deploy of 10 times per day. It's called "impossible" by some characters, but, since the novel is a polemic about DevOps, of course they figure it out quickly. The key is to challenge your assumptions about what's possible. At my current job, we recently had a semi-successful meeting where some team members argued that what was being asked for was "against the laws of physics". That to me is a sign that you need to approach the problem differently, set aside the "impossible" label, and look at what obstacles are in the way. If you can re-route around those obstacles, or wave a magic wand to streamline them (by several orders of magnitude), then the solution becomes possible, and the new challenge is driven by the excitement of doing what you knew yesterday was impossible.

I guess the challenge for me now is to figure out how to find something impossible every day... make each day a mini-sprint, and see what happens to my output.

Wednesday, November 28, 2012

Blogging: inconsistent quality or frequency?

One of my tasks at my company is to create blog entries, both on internal and external blogs. (See some of my other posts for links to what I've written.) I've noticed an interesting difference between writing for myself and writing for my company:

When I write for myself, my output is determined by when I get an idea that I want to share. Results are created by inspiration.

When I write for my company, my output is determined by our schedule. If I don't have an idea, I get one from our marcom organization, and I work at it until it makes sense, even if I personally don't see a lot of value in the content. Conversely, if I have an idea which is burning inside my head, I have to fight to get it included in our publishing schedule, with the occasional exception as a special post.

There are good reasons for scheduled output: when trying to generate a follower base, predictable frequency makes it easier for readers to know when something new will go up. It also means that, for infrequent readers, there will almost always be something new to read.

However, it's hard to train my brain to get inspired on demand. A lot of us know this feeling, and there's a fabulous write-up about it from The Oatmeal. In short, content on a schedule leads to inconsistency of output quality.

It's tempting to put together a bogus formula which asserts that quality ideas are a non-linear, which can lead either to inconsistent output frequency or to inconsistent output quality. I'll leave that as an exercise to the reader.

However, I'll also point out that there's a disconnect in perceived quality between the information producer vs the information consumer. Neil Gaiman expressed this eloquently in his Make Good Art speech. This also appears to be a common experience: I write something that I think is awful, and other people love it, and vice-versa.

Given this disconnect, I feel comfortable for now working with a content production frequency schedule. My internal discomfort at writing can spur me to produce output that's much better than I'd anticipated. It's also useful to get ideas for technical blog posts from people who have a different understanding of technology that I do, because it triggers the instinct to correct the misinformation - which leads to output on a schedule.

That being said, I still don't like the process, but I'm okay with the results.

P.S. Some readers of this post will notice that the title of this post is a question. When I started writing, I had a completely different title, but I changed it part-way through, with the implication that it's an either-or. Now, I look at it and realize that I'm compliant with Betteridge's Law, and the answer to the question is in fact "No".

Monday, August 13, 2012

IPv4 "Offshore Account" Predictions

IP addresses are a necessary resource for using the Internet, especially for service providers like web hosting companies.  Given that unused addresses are becoming scarce, I predict that we'll start seeing businesses invest in Latin America and in Africa specifically to acquire IP addresses there.

The setup

On July 31 this year, ARIN adopted a new policy to allow inter-region transfers of IP address allocations.  That may be news to some people, that IPv4 addresses aren't like normal property that can be bought and sold at will.  According to ARIN CEO John Curran, this is because "how we use [...] IP addresses affects all networks". Interestingly, as of August 8, the only other regional internet registrar (RIR) with a compatible policy is APNIC, which reached address exhaustion on April 19, 2011.

A quick note on address exhaustion: this means they have less than 1 /8 block left, not that they are completely out of addresses.  A /8 potentially contains 65536 /24 blocks - although that number will be smaller if an organization can convince the RIR to allocate a larger block than a /24.  APNIC currently has 0.9183 /8 blocks, which roughly translates to about 60000 /24 blocks.

Why is it important that APNIC and ARIN have compatible regional transfer policies?  That means that it's possible to move IP address allocations between them.  Right now, the obvious motive is to ease the IP crunch in APNIC from ARIN, which has 3.4561 /8 blocks.  However, on the FAQ page for this new policy, ARIN states that "There are inter-RIR transfer policy proposals in several other regions at the moment".  Assuming that other RIRs have similar mentality to ARIN, it's likely that ARIN is establishing a policy now to allow for inbound IP transfers once ARIN reaches exhaustion in early 2013.

The sources

Current projections for remaining IP blocks are available in this nifty gadget courtesy of INTEC, although its numbers differ from the data provided by Internet guru Geoff Huston.

The big question is what ARIN will consider a necessary part of the policy to be "compatible".  ARIN's transfer policy imposes a 12-month before-and-after waiting period on transfer sources within the ARIN region: the source must have had the IP addresses for over 12 months before the transfer, and can't receive any more addresses from ARIN for another 12 months after the transfer.  However, the policy also states that "Source entities outside of the ARIN region must meet any requirements defined by the RIR where the source entity holds the registration."

The possibilities

1. Direct IP exporters

If another region has a much less restrictive policy, there's the possibility of a new business model for a company in that region to apply for IP address blocks, then sell them.  It's a centuries-old practice for a developing nation to sell its raw resources to overseas buyers.

2. Foreign shell companies

If other regions are planning to adopt strict restrictions like ARIN on source organizations for IP transfer, the logical step is for global organizations to found shell companies right now in the LACNIC and AfriNIC regions.  Then, once the hold-down timers expire - e.g. the shell company has had its IP addresses for 12 months or whatever the local policy is - then the parent company would either initiate a transfer, or just acquire the shell under ARIN's mergers and acquisitions policy.

3. Foreign providers

If an address transfer isn't feasible, the next likely business model would be for "boutique" hosting companies to spring up in the LACNIC and AfriNIC areas.  Bandwidth prices there are high but falling, leading to the potential for hosting companies there to rent out destination IP addresses, or, more importantly, blocks of IP addresses.  

To reduce bandwidth usage (and reduce costs), there are techniques available both at the IP layer and at higher layers.

Within IP address advertising, there's no current technical restriction to prevent geographic relocation via BGP advertising, in effect becoming a semi-legitimate use of IP Hijacking.  Even anti-hijacking technologies like RPKI could be co-opted, either via disabling (like the current status of the anti-spam SPF technology) or by simple delegation within the PKI.

Within higher layer protocols like HTTP, techniques like redirects are a time-tested method of providing a fixed landing point with dynamically located content.  The hosted site would contain just enough information to pull the real content from a CDN or other external high-bandwidth low-cost source.

The timeframe

Given the 12-month limitation from ARIN for transfers, plus the projection of ARIN address exhaustion only 6-12 months away, look for large organizations to start this kind of "IP address offshoring" very soon.  In fact, given that the newly adopted ARIN policy was first proposed in February 2011, it's likely that some global organizations have already started this process.

Just for grins, check out this map of Chinese investment in Africa - then remember that China is in APNIC, and has long been short on IP addresses.