Thursday, June 10th, 2010. About 5:45am. I had just climbed the stairs to the Blue Line El platform at Damen and North on my way down town. There was that intense early morning sunshine we get in Chicago near the solstice, and it wasn’t too warm yet. There weren’t a lot of people around, but I immediately saw and made eye contact with two Black gentlemen that didn’t appear to be from the immediate neighborhood. I’m the type of person who makes eye contact with everyone, and well, sometimes you get “caught.” Women will know what I mean. They did seem to have a positive attitude, with the smaller of the two smiling – beaming almost. I had little chance they would take much interest and form an attachment to me compared to a woman, so I wasn’t worried.

When I used “gentlemen,” it was out of initial respect for all my fellow citizens. They were not from the cream of society and bore scant similarity to the romantic English gentlemen of old. Little attention to fashion of clothes or style of hair. Mannerisms, etc. The smaller, smiling guy was the leader and clearly going to be the talker. Hygiene seemed normal, thank god. That’s passing a major test for interacting with strangers.

I could say they were “working class,” but I don’t thing these guys were gainfully employed. And, who is completely, 100% wide awake looking totally refreshed before 6am? Neither those who are out all night, nor those who are up early for work. Only these guys. In some third category I can’t fathom.

So the talking started. “How about those Blackhawks!?” he asked, beaming. The Chicago Blackhawks had just won the Stanley Cup the night before. Hmm, they weren’t the stereotypical hockey fans. Then, I did what you do when you are not sure if you want to be talking to strangers – say as little as possible in a very reserved way. This opening did, however, put me more at ease. “Pretty awesome,” I replied. Then, according to the standard back-and-forth protocol, the story began.

“I don’t know nothin’ about hockey, but when that dude got his teeth knocked out and kept playin’ I knew they was gonna win.” BEAMING SMILE. “I didn’t have any money [I can't keep up with the vernacular], but I made a huge bet with the bookie and won 1600 dollars.”  Duncan Keith did have seven teeth knocked out in the last game of the semi-finals and impressively kept playing.

“Cool.”

“Yeah, if they lost I wasn’t going to pay them. I was going to go hide out in Madison for a few years.” Madison in Wisconsin. (Suspected employment status confirmed.) That brought a smile to my face. Made my day.

Betting on sports is common in trading firms, but none of my friends risked stiffing Chicago street bookies as far as I’m aware. But, that’s why I talk to strangers. I plan to share other stories on this blog and hope that some of you find them interesting.

Sean

Rob Martin asked whether to use use other dev Ruby gems when developing new gems, or if it was better to use qa or prod version gems in the development environment. I’m going to consider that he means in individual coding environments. Ideally, you want to start all dev projects with a code configuration that is going to look like production when the dev project gets released. Any departure from this is taking on some amount of at least one type of software development risk. In this post I’m going to call out some of those risks in typical commercial environments to help us identify and avoid them in practice. (Currently I’m managing C++, Perl and database code and not a Ruru=Ruby Guru, but the question is broadly applicable to most common languages.)

For an app with few changes and static release inventory, there is no problem having a dev environment and configuration that can match production or the next qa release, but for a high change-volume application with dynamically changing release content, this can be a lot more challenging to achieve. I’ll illustrate with a few scenarios.

Let’s take a standard ‘dev-qa-prod’ lifecycle, and start with a case where each of the dev, qa and prod code configurations are identical. By code configuration, I mean they have all have the same exact versions of the same files. I now have a dev project to develop girrafe.gem. No problem with any dependencies giraffe.gem may have because my dev matches qa and prod. When I release giraffe.gem to qa, I can expect there to be test integrity between dev and qa. When giraffe.gem passes qa, I can be assured of test integrity between qa and prod, because those configurations match after releasing giraffe.gem. (In this case, I’m considering a delta release – the only change to qa and prod is the introduction of giraffe.gem). It’s good to use this simple case as a reference for more complicated ones, as a sort of Gedankenexperiment.

For the next scenario, we have a release called Managerie, that is currently in qa. A problem is found that requires a change to zebra.gem. I think we can all foresee issues if I start off my work on zebra.gem with a dev software configuration that matches prod and not qa. I’ll keep going, though because this is another ideal case that can serve as a reference. Here, I would be using versions of gem’s in development behind the one’s in qa. I could get lucky if I am lazy and feel omniscient and believe=hope there are no dependencies that affect my change to zebra.gem, but I would be taking on some risks at this point:

  1. Risk of breaking test integrity. Test results in dev may not match test results in qa – “It worked for me.”  This could happen even if my change request for Managerie involved a version of zebra.gem in qa that matched prod – in other words even if zebra.gem had not been part of the original changes required for Managerie. Because dependencies differ between dev tests and qa tests, the results could differ.
  2. Risk of non-productive time in development. This is ultimately tied to all the other risks. If I misled myself with my test results, then it’s another cycle through dev. If zebra.gem were part of Managerie, then the qa version would differ from the prod version. If I tried to get that to work in a prod-like dev environment, it might not even work properly wasting time.
  3. Risk of production defect. If qa doesn’t have perfect coverage (when does it?), an issue might not show up until released into prod. Now you have added a fair amount of stress and/or money to everything else.

Now we get to a more reasonable scenario, I think closer to what Rob was considering. OK, were intelligent and ambitious. We have multiple projects getting started that will be released at different times. I’m working on the zebra.gem again for the Savannah release. Mary is working on badger.gem for the Michigan release and Lucy is working on tiger.gem for the Zoo release. Menagerie is still in qa, but looking good and just waiting for the right time to move to prod.

Mary, Lucy and I each need to start our dev projects with the qa configuration, Managerie. It will be released next, then Savannah, then Michigan, then Zoo. If my environment has ideal order and predictability this ideal set of actions would follow:

  1. Dev, QA completes for Savannah.
  2. Mary and Lucy merge/sync their dev environments with the Savannah configuration.
  3. Dev, QA completes for Michigan.
  4. Lucy merges/syncs her dev environment with Michigan.

Now let’s replay this scenario, but throw a wrench at it. Mary has been given the common but unreasonable task of starting Michigan with a strong dependency on the version of the zebra.gem being developed for Savannah. Mary can’t do her work without a basic Savannah zebra.gem. Here’s a legitimate case where Mary is going to beg for, use and possibly contribute to Savannah’s zebra.gem. She may need to frequently merge/sync her version of the zerbra.gem and possibly here entire dev configuration (if zebra.gem has strong dependencies) with Savannah’s dev configuration. It could mean a lot of pain and inefficiency for Mary’s development, and this is a management issue, but sometimes business interests win out over optimal development. It’s also possible that Mary and I have some kind of awesome Agile synergy and power through these two releases together.

What about Lucy? Zoo has little dependency on Savannah and Michigan. I’d say she should stick with a stable release and avoid Savannah and Michigan like the plague until there is a stable release and it becomes important to avoid a negative impact on her development and testing. Let’s say Lucy feels omniscient and wants to keep tabs on the awesome stuff in Michigan and Savannah. With Git, it’s easy to keep those releases as separate viewable branches, and keep it out of the Zoo dev configuration, but let’s say she pulls those changes into her Zoo dev configuration anyway. What risks is Lucy taking on?

  1. Risk of non-productive development time in a number of ways:
    • The zebra.gem could be periodically inoperable, and though it is not a strict dependency of Zoo, it could still impact her ability to test.
    • Adds complexity and mental drag to much of her work. These changes aren’t part of zoo – which are Lucy’s changes, which are others’? What does the target release code look like? I can’t tell because there are so many other changes in my configuration.
    • Unforeseen dependency. Notice I’ve been using the word ‘omniscient’ a lot. Are you really omniscient? How well do you really know the code and dependencies? Ask yourself, “Do I feel lucky?” It can be good to identify unforeseen dependencies early in development, but there is also such a thing as “too early” for minor changes and this is not an argument to go Total Cowboy in development, Mr. I-Think-I’m-Agile. I admit waiting for a stable version to pass qa is likely too late. The purpose of syncing with stable Zoo and Savannah is to find those dependencies. If it were something major, there should be enough expertise on the team to find out early in the release planning. If this is not the case, then more effort needs to go into planning.
  2. Career risk. Lucy, what is your job? Are you fooling around with someone else’s project when you are supposed to working on the Zoo release? OK, hopefully you are on a cool, hip Agile team and that won’t happen, but beware of the organizational and even political impact of what you are doing.

So, I go with squeaky clean development configurations and invest in keeping it clean to avoid software development risk. I take only my planned changes against a code configuration that I think I’m going to release my changes into. MY_DEV = TARGET_RELEASE + MY_DELTAS. I also want to have an operable dev server that reflects my dev code configuration so I can test effectively. Keeping things simple makes me more productive.

HTH

Sean

Welcome Back, Me

May 22, 2013 — Leave a comment

It’s been a few years since I last blogged on software development operations over at OpenMake Software and it’s time to re-join the public community and share a wider set of knowledge, stories and views. This blog is going to be a bit of a challenge to pull off because there will be a few diverse lines of content including:

  1. Commentary and speculation on the direction and development of our “Information Age”
  2. Gritty tales from the edge of society that I’ve heard first-hand.
  3. Low-level technical how-to’s around Linux, application and web development.

I’ll try to keep a very small set of simple categories and maybe some portals to give direct access to one category or the other. At worst, it will be pretty obvious if you get a post in #3 and are not interested. Color and layout coming as well.

Cheers