I par­tic­i­pated in my first main­te­nance win­dow in 1999. Well, let’s say down­time. Or, if we want to be char­i­ta­ble, acci­dent.

A fancy new switch was deliv­ered to the office where I was work­ing at the time, along with a bunch of cables, and my boss said to me, “Plug that into our network!”

You might be able to guess what hap­pened next. I took down three sub­nets with a net­work loop. My boss walked in as I was pulling cables out and con­grat­u­lated me for being the fastest sprinter back to the data center.

And then we had a meet­ing to talk about how we could keep that from hap­pen­ing again. We talked about span­ning tree, run­ning con­fig­u­ra­tions by a coworker before plug­ging things in and testing.

What I came to respect my boss for was not just that he didn’t hang me out to dry for an ama­teur mis­take (and, hey, I was an ama­teur!). He took the time imme­di­ately to fix the process: edu­cat­ing me about what to do next time with a focus on testing.

At Emma, we have poli­cies in place to man­age main­te­nance like change man­age­ment tick­ets, and a weekly meet­ing among IT and engi­neer­ing staff to dis­cuss changes. With a grow­ing office in Nashville and data­base folks in Portland, OR, we recently real­ized we needed a lit­tle more, both to make it clear to folks when a change was com­ing and to plan for when things didn’t go quite right.

So I set out to doc­u­ment the exist­ing process and add a few extra steps for explicit com­mu­ni­ca­tion. Here’s what I came up with:

1. Create a ticket to track the change.
2. Get in touch with folks imple­ment­ing the change — who will make the change, who will restart, reload and test the affected services.
3. Loop in our Community team.
4. Choose a date suit­able for all involved.
5. Set up a pre-change meet­ing to dis­cuss the changes and roll­back plan with all team mem­bers — who might think of things that you forgot!
6. Announce change to departments.
7. Send an announce­ment to cus­tomers through Community.
8. Conduct the change!
9. Send an announce­ment that the change was com­pleted through Community.
10. Have a post-change meet­ing with Community to wrap up lose ends, and hope­fully cel­e­brate a suc­cess­ful change!

The things we added to the exist­ing process were “Loop in the Community team” and “Have a post-change meet­ing with Community.” The Community team includes all of Emma’s post-sales sup­port staff and our designers.

In the past, we’d just emailed folks in Community about what we were up to. Now, we’re mak­ing them part of our change team. This keeps them bet­ter informed, and it allows us to ask for their insight into test­ing and the impact we’re hav­ing on customers.

Our first crack at the new process was the sec­ond week­end in January. We imple­mented a few con­fig­u­ra­tion changes and upgraded our pro­duc­tion data­bases. Immediate feed­back was good. Internally, folks work­ing on the change reported a sense of calm, as well as sat­is­fac­tion that the change went smoothly. Customers reported appre­ci­a­tion with our com­mu­ni­ca­tion, but they would have liked a lit­tle more notice. Point taken.

And then we had the post-change meeting…

My next post will be all about that meet­ing. Hopefully, it will give y’all a peek into how we tend to han­dle sug­ges­tions for improve­ments at Emma.