What is an efficient web development process?

A few years ago, I answered this question on Quora which somehow, for a time, propelled me to the top of the Quora contributors on Test-Driven Development, ahead of the guy who basically invented it. I’ve been talking to a lot of companies lately about their development practices, and this question comes up. I’ve been sending them to this article, so I thought it warranted a revisit.

Honestly, this is still the development process I think is most efficient for a young company, but one thing that’s missing is the why, so before I repost my own work, I’m going to clarify some of that.

User stories hit the sweet spot of visibility and accountability for the management team, and give an appropriate amount of autonomy to the team. Speaking again about small, early stage companies, the goals of the company, milestones, and customer at the end of the day need to be owned by the CEO. But at the same time, the product and engineering teams were hired for their expertise. The movement of an idea from goals to user story to code puts each member of the company in the position to make the right level of decision at the right time — the CEO and other leaders at goal setting time, PMs and designers at defining the behavior, and engineers at implementation, while at the same time giving everyone a chance to participate all the way up to the goals-setting level, so they can bring their unique experience and expertise to the problem and actually have ownership.

This cycle of estimation, iteration, and velocity tracking also hits a sweet spot I believe. We all know software complexity is difficult to predict, and thus I believe there are diminishing returns to spending more time scoping, especially for small stories. However, especially at a startup, it’s not good enough to say “hey, we have no idea — we’ll be done when we’re done.” Estimation should take no more than 30–60 minutes out of your week, and doubles as a time to understand and agree upon the details of the work to be done. Basic velocity tracking can be used at a minimum to provide some sense of urgency/a goal to be achieved in terms of deliverables, that is both non-threatening and empirical (or as close to empirical as this can get.) At its best, velocity tracking can be used as a diagnostic tool for the team and process — “Why did X take longer than we expected? What are the blockers?”

I still believe in test-driving if you have the facility for it. In the giant landscape of tools and processes for making sure you don’t mess up the code, automated tests for me are the first and most important piece, if only because they allow you to do continuous deployment with confidence, which reduces the time you spend between writing code and putting it in front of users. There’s value to engineering focus, but there’s possibly even more value in terms of business and customer insight to having a tight feedback loop.

There’s one small thing I’m changing here, and that’s generalizing the post to be about any language and framework instead of just Ruby and Rails. Though I will admit, this process is messier in an SOA or any system with a complex architecture. This is possibly the strongest argument in favor of simple architectures at the beginning. I’d always favor velocity over scalability — until you hit scale that is.

I do think that this process is tailored for small companies, and will break down as the company grows. It’s important to evolve the process as the team finds new and better ways to do things, but I started a company today with 1 PM, 2 engineers, and a designer, this would still be how I structure the work.

I think it also bears noting that this is an answer to a question about efficiency. If there are other things you’re optimizing for, your process won’t look like this. But I would argue that any early startup looking to beat the odds is going to be primarily interested in moving quickly once they’ve identified the goal, and thus looking for efficiency.

1. Write down a list of goals, roles, and features

• Goals — what the goals of the whole project are — business and otherwise. This will help you decide what features are important
• Roles — who is going to use the site — visitors, logged in members, admins? Do different people have different views of the same information on the site?
• Features — what are the basic categories of interaction on the site? For example: Users: registration, using the forums, and blogging; Admins: moderating the user content

2. Write a list of stories

• A story is different than a feature because it represents a single thread of interaction from a particular user’s perspective.
• It’s common to express stories in form “As a ____ I want to ____ so that I can _____.” This forces you answer three important questions — Who is this for? What do they want to do? Why do they want to do it?
• If you can’t complete a story in this form, it’s likely that you don’t have an answer to one of these three questions yet, so you’ll need to do some thinking to get the answers before the story is actionable.
• Ex: “As an admin, I want to ban users from the forum, So that I can improve the quality of user-submitted content on the site.
• Write these stories down on notecards. This will help you in estimation and prioritization.

3. Estimate the stories

• Estimation is a huge topic in itself, but the basic idea is to associate a particular level of effort with each story.
• The most common scales are 0/1/2/3/4, 0/1/2/4/8. I don’t think this is incredibly important, but pick something and stick with it.
• Don’t get too hung up on the exactness of the estimates. Lots of things affect how long it takes you to finish a story, so small differences in story complexity tend to get lost in the noise.
• Your goal here is to differentiate things that are low in effort, like stories that will result in you creating a simple model with a REST controller, from stories that are high in effort, like interfacing your application with a challenging third-party API, or a story that will require you to use a technology you aren’t very familiar with.
• Write the estimate on each card.

4. Prioritize the stories

• Rearrange the cards in the order that you’d like to tackle the stories.
• Only the product owner can really make this decision. There are a lot of things that go into prioritization — deadlines, user testing, business value, etc. Estimation may have a lot to do with prioritization, because it illuminates opportunity cost. Maybe the product owner really wants that detailed Admin Dashboard, but if all the stories to make that work total 40 points, is it worth it to spend a month on just this feature. Maybe the product owner still wants the story
• Are there any stories that don’t fit into the very minimum viable product to launch? If so, you should move them down. Try to complete a functioning app as quickly as possible so you can put it in front of users.
• At this point, I usually move my cards into Pivotal Tracker, but I know lots of people who prefer pen and paper.

5. Test-drive the first story to completion

• Begin with acceptance tests Write a an acceptance test that covers the user’s interaction with the site from beginning to end. Define the undefined steps as you come to them, and when you hit your first failure, you know that there’s a behavior that you desire that your app doesn’t have (This will happen very quickly at first, because your blank app doesn’t have much behavior).
• Continue to unit tests Write the test for the behavior you wish you had.
• Write your code Write the code to make the spec pass. This is going to take you throughout your application from routing to UI, to models, to the database schema, to the controller. You will tackle these pieces of code in the order your tests direct you to.
• Repeat until the acceptance test passes, and you’re done with the story.
• Now is a good time to fix up the CSS styling assuming you have the design done. If I’m working alone or without a designer, I like to try to wireframe the UI either on paper or in something like Adobe XD before I even start coding the story.

6. Accept the story

• Is the story acceptable? Does it do what you wanted it to? If not, you need to go back and make it work the way it was supposed to. Writing acceptance tests in advance helps prevent this from happening.
• Do all your tests pass? You didn’t break the build, did you? If so, you need to fix what you broke.
• If you’re working alone, it may be helpful to have someone else do acceptance for you, as it may be hard to see your own work with objective eyes.

6. Repeat until done

This is how I do things. It’s by no means the only way to do things in all circumstances.

The advantages of this approach are numerous. You have something to put in front of users to gather feedback from your very first checkin. Your project plan is extremely flexible in terms of changing priorities and highly tolerant of changing requirements. If you’ve done any amount of engineering you know that changing requirements are not a matter of if, but a matter of when and how much. For those 3 reasons alone, you should approach your application development in vertical slices as opposed to horizontal slices.