Introducing Elixir - Introducing Erlang - Learning Rails 3 - XML Pocket Reference, 3rd - Office 2003 XML - Programming Web Services with XML-RPC - XML: A Primer - XML Elements of Style - XHTML: Moving Toward XML - Building XML Applications - Inside XML DTDs: Scientific and Technical - Cookies - Sharing Bandwidth - Dynamic HTML: A Primer

Articles - Projects - Biography - Identi.ca - Twitter - O'Reilly Network Weblog - Advogato diary   

March 18, 2015

O'Reilly Radar » Simon St. Laurent

Worship maintainers


Technology has had a cult of newness for centuries. We hail innovators, cheer change, and fend off critics who might think new and change are coming too fast. Unfortunately, while that drives the cycle of creation, it also creates biases that damage what we create, reducing the benefits and increasing the costs.

Formerly new things rapidly become ordinary “plumbing,” while maintenance becomes a cost center, something to complain about. “Green fields” and startups look ever more attractive because they offer opportunities to start fresh, with minimal connections to past technology decisions.

The problem, though, is that most of these new things — the ones that succeed enough to stay around — have a long maintenance cycle ahead of them. As Axel Rauschmayer put it:

“People who maintain stuff are the unsung heroes of software development.”

In a different context, Steve Hendricks of Historic Doors pointed out that:

“Low maintenance is the holy grail of our culture. We’ve gone so far that we’re willing to throw things away rather than fix them.”

That gets especially expensive. Heaping praise on the creators of new things while trying to minimize the costs of the maintainers is a recipe for disaster over the long term.

I’ve contributed to this disaster. I love starting new projects on open fields, full of visions for creating from scratch. I can choose my own tools, tackle problems from the angles I prefer, and, perhaps most important, choose the order in which I add new features. In the books I write, I assume the reader is starting from scratch, able to learn features in isolation and slowly develop a sense of how it all fits together.

Even as I’ve spent years telling the story of starting from a clean slate, though, I’ve spent most of my programming and Web time on maintenance, updating my own and other people’s content, code, servers, and networks. Maintenance constantly calls me out of my comfort zone, forcing me to remember things I did long ago or focus on parts that used to just work. Maintenance forces me to rely on other people — the real heroes — on a regular basis, often in emergency situations.

So how can we fix it?

This is a cultural problem, not a technical problem. More precisely, the problems have technical aspects, but those aspects vary wildly from project to project. Maintaining AngularJS projects is very different from maintaining COBOL projects. The cultural stories, though, may be more similar.

Someone at CSSDevConf last fall took the first step on that path, asking the audience how many of them moved from new project to new project and how many stayed on projects for a long time going forward. That prompted some blunt conversations about the differences among Web developers and their priorities, though it was clear that it was just a first step.

DevOps — I think because it values operations — takes maintenance seriously, insisting on covering the whole lifecycle of a product rather than the waterfall vision of perfection at the deployment of a project. Iteration and continuous development models put a gloss of new and fresh on what used to seem old and blocking.

DevOps eases the problem and gives some of us a place to talk about it. Most of us, though, work in places whose projects have not been and likely will never be under the DevOps umbrella. It’s also possible that projects that start in full DevOps mode dwindle and find themselves in more traditional maintenance. While I hope DevOps approaches will prove contagious, something smaller seems like it might be more important.

We need to place a much higher value on maintenance and maintainers. We need to worship maintainers with the same fervor we worship creators. We need to recognize that not having emergencies can mean more, despite being less visible than a well-handled emergency. We need to value continuity (in a bumpy world) as much as we value growth. We need to recognize that growth is pointless unless we provide it with a solid foundation.

Editor’s note: if you’re interested in finding out if a DevOps approach will prove to be contagious within your organization, you’ll want to check out the Effective DevOps training session at Velocity in Santa Clara May 27-29, 2015.

This post is part of an ongoing exploration into cross-pollinating Web communities.

Cropped switch image CC BY-SA 2.0 Andrew Hart via Flickr.

by Simon St. Laurent at March 18, 2015 07:35 PM

March 11, 2015

O'Reilly Radar » Simon St. Laurent

Full-stack tensions on the Web

Vista_de_la_Biblioteca_VasconcelosI expected that CSSDevConf would be primarily a show about front-end work, focused on work in clients and specifically in browsers. I kept running into conversations, though, about the challenges of moving between the front and back end, the client and the server side. Some were from developers suddenly told that they had to become “full-stack developers” covering the whole spectrum, while others were from front-end engineers suddenly finding a flood of back-end developers tinkering with the client side of their applications. “Full-stack” isn’t always a cheerful story.

In the early days of the Web, “full-stack” was normal. While there were certainly people who focused on running web servers or designing sites as beautiful as the technology would allow, there were lots of webmasters who knew how to design a site, write HTML, manage a server, and maybe write some CGI code for early applications.

Formal separation of concerns among HTML, CSS, and JavaScript made it easier to share responsibilities among specialists. As the dot-com boom proceeded, specialization accelerated, with dedicated designers, programmers, and sysadmins coming to the work. Perhaps there were too many titles.

Even as the bust set in, specialization remained the trend because Web projects — especially on the server side — had grown far more complicated. They weren’t just a server and a few scripts, but a complete stack, including templates, logic, and usually a database. Whether you preferred the LAMP stack, a Microsoft ASP stack, or perhaps Java servlets and JSP, the server side rapidly became its own complex arena. Intranet development in particular exploded as a way to build server-based applications that could (cheaply) connect data sources to users on multiple platforms. Writing web apps was faster and cheaper than writing desktop apps, with more tolerance for platform variation.

Web services added a new level of separation, letting server-side developers spend less time on templates and more time on data structures. They attracted a much more specifically programmer crowd. Front-end engineers emerged as client-side work became more like programming, often interpreting the data received from those services and transforming it into user experience in browsers and (web-based) apps.

For a brief period in my career, I knew the stack down to the IP networking layer and up through JavaScript, with enough Perl or Java and database skill to write a complete application. That was over a decade ago, though. More recently, Ruby on Rails let me pretend to be a full-stack developer, insulating me from a lot of details I used to know more about. Rails has offered a coherent, if ever-changing, home base for a wide variety of different components from databases to stylesheet and code pre-processors. As recently as 2008, full-stack seemed achievable.

Learning more about the pieces that interact with your work can broaden your horizons, making it easier to create new connections or figure out where in a project things have gone strange. Highly driven individuals who want to build complete sites or apps on their own will find it useful to be a full-stack web developer. Employers, especially at startups trying to minimize the number of conversations on the way to a product, still dream of hiring full-stack web developers despite the necessary compromises. Node may have encouraged some of those dreams, giving JavaScript a home on the server as well as in the browser.

The primary reason developers should know about the full web stack, though, is to be able to communicate better with their collaborators. Asking a designer to style a REST request isn’t likely to make sense without more context, and neither is expecting a NoSQL database admin to understand the details of formatting HTML tables or a content creator to understand JavaScript function call structure. You’ll be much more effective if you can combine broad understanding with specialized skills: be great at what you do while understanding what others do.

So what stack should you know about? The LAMP stack? The MEAN stack? The stack your projects use? Understanding the particular stack your projects use is a good idea, but stepping back a little further may help you deal both with change and with other people. The real web stack — and I’m sure there are multiple possibilities — looks more like:

  • Networking – DNS and TCP/IP basics provide a foundation.

  • Web protocols and communication styles – HTTP, REST, Websocket, and WebRTC connect us further. Caching and proxies often appear inside large applications as well.

  • Data and document formats – JSON, XML, CSV, HTML, and sometimes more options let us share information.

  • Web server configuration – Creating a host for information means knowing about web servers, operating systems, and sometimes more.

  • Data storage – Relational databases, NoSQL databases, file systems, and Object-Relational Mappers (ORM) give you places to put things.

  • Server-side programming – Handling requests and integrating information using your favorite tools and languages. Node and JavaScript? Ruby on Rails? Python and Flask? Or something else entirely?

  • Content management systems – Reusing existing approaches can save you lots of time, or create new integration problems.

  • Library and framework management – Which libraries to use, if any? How to combine them? How should you create internal libraries and frameworks?

  • Bundling and packaging – How do you distribute what you’ve created? Does packaging it as an app make sense? Or, in another context, do you need a Content Distribution Network? Or processing in the cloud?

  • Device capabilities – It might seem like there are more HTML5 APIs every week, but geolocation, local storage, web workers, and many more are constantly broadening what you can do.

  • Template design – How do you create easily reusable (and flexible) structures?

  • Content presentation – How should you structure the content of your site? What should it look like? HTML and CSS dominate here, but it’s also critical to understand the contexts in which content appears.

  • Interface programming – HTML, CSS, and JavaScript. Data binding has added new JavaScript-centric dimensions.

  • Performance – How do you make this all happen fast?

  • Resilience – How do you make sure this happens reliably?

  • Security – How do you make sure this only goes to the right people and places?

  • Navigation design – How should people (or other applications) find their way through what you’re building?

  • Graphics – Show, don’t tell, but with images, SVG, Canvas, or WebGL?

  • Reach – How do you make sure people (or other applications) can find what you’ve created?

  • Interface design – How can you create interfaces that people can use? That people want to use? That work in the device contexts of their choice?

  • Visual design – Making all this work and look good and probably look distinctive is a challenge.

  • Content – What are you trying to convey, or ask?

  • Product design – What are you building? What problems does it solve, and how?

That’s too much, but it all fits together.

Obviously, you don’t need to have specialist-level skills in all of this to build on the Web. The core technologies tend to be HTML, CSS, JavaScript, and HTTP, and a basic understanding of each of those can carry you a long way. If you only build services and just know your preferred programming language and environment, you’ll probably be fine. Visual designers and content creators need to understand the medium in a broad sense, but probably don’t need to know TCP/IP details.

Staying afloat means, to some extent, staying shallow. You need to know the basics of all of those things, at least what they mean, so you can follow the conversation. You need to specialize in your own space, and you need to be able to communicate well with your peers in adjacent fields.

Learning more is great — just don’t feel, and hope your employer doesn’t feel, like you need to know everything about every part.

This is the first post in an ongoing exploration into knowing a full stack, even if you don’t work it every day.

Image of the José Vasconcelos Library CC BY 2.0 Eneas De Troya.

by Simon St. Laurent at March 11, 2015 04:00 PM

March 10, 2015

Living in Dryden

Family Fun in Varna, March 14th

Family Fun in Varna

Saturday, March 14

We're raising funds for the new Varna playground!

  • 2pm - Kid-friendly music by Angie Beeler (Music and Motion)

  • 3pm - What's a community-built playground?? Refreshments, too!

  • 4pm - Classic rock songs you know you love from the band Encore

Everyone welcome! Suggested $10 donation per adult, children free. Donations above $10 will be matched by anonymous donors. Donations are tax-deductible and will help make the new Varna playground a very special place.

Varna Community Center, 943 Dryden Road (NY Route 366)

March 10, 2015 05:54 PM