Rob Martin asked whether to use use other dev Ruby gems when developing new gems, or if it was better to use qa or prod version gems in the development environment. I’m going to consider that he means in individual coding environments. Ideally, you want to start all dev projects with a code configuration that is going to look like production when the dev project gets released. Any departure from this is taking on some amount of at least one type of software development risk. In this post I’m going to call out some of those risks in typical commercial environments to help us identify and avoid them in practice. (Currently I’m managing C++, Perl and database code and not a Ruru=Ruby Guru, but the question is broadly applicable to most common languages.)
For an app with few changes and static release inventory, there is no problem having a dev environment and configuration that can match production or the next qa release, but for a high change-volume application with dynamically changing release content, this can be a lot more challenging to achieve. I’ll illustrate with a few scenarios.
Let’s take a standard ‘dev-qa-prod’ lifecycle, and start with a case where each of the dev, qa and prod code configurations are identical. By code configuration, I mean they have all have the same exact versions of the same files. I now have a dev project to develop girrafe.gem. No problem with any dependencies giraffe.gem may have because my dev matches qa and prod. When I release giraffe.gem to qa, I can expect there to be test integrity between dev and qa. When giraffe.gem passes qa, I can be assured of test integrity between qa and prod, because those configurations match after releasing giraffe.gem. (In this case, I’m considering a delta release – the only change to qa and prod is the introduction of giraffe.gem). It’s good to use this simple case as a reference for more complicated ones, as a sort of Gedankenexperiment.
For the next scenario, we have a release called Managerie, that is currently in qa. A problem is found that requires a change to zebra.gem. I think we can all foresee issues if I start off my work on zebra.gem with a dev software configuration that matches prod and not qa. I’ll keep going, though because this is another ideal case that can serve as a reference. Here, I would be using versions of gem’s in development behind the one’s in qa. I could get lucky if I am lazy and feel omniscient and believe=hope there are no dependencies that affect my change to zebra.gem, but I would be taking on some risks at this point:
- Risk of breaking test integrity. Test results in dev may not match test results in qa – “It worked for me.” This could happen even if my change request for Managerie involved a version of zebra.gem in qa that matched prod – in other words even if zebra.gem had not been part of the original changes required for Managerie. Because dependencies differ between dev tests and qa tests, the results could differ.
- Risk of non-productive time in development. This is ultimately tied to all the other risks. If I misled myself with my test results, then it’s another cycle through dev. If zebra.gem were part of Managerie, then the qa version would differ from the prod version. If I tried to get that to work in a prod-like dev environment, it might not even work properly wasting time.
- Risk of production defect. If qa doesn’t have perfect coverage (when does it?), an issue might not show up until released into prod. Now you have added a fair amount of stress and/or money to everything else.
Now we get to a more reasonable scenario, I think closer to what Rob was considering. OK, were intelligent and ambitious. We have multiple projects getting started that will be released at different times. I’m working on the zebra.gem again for the Savannah release. Mary is working on badger.gem for the Michigan release and Lucy is working on tiger.gem for the Zoo release. Menagerie is still in qa, but looking good and just waiting for the right time to move to prod.
Mary, Lucy and I each need to start our dev projects with the qa configuration, Managerie. It will be released next, then Savannah, then Michigan, then Zoo. If my environment has ideal order and predictability this ideal set of actions would follow:
- Dev, QA completes for Savannah.
- Mary and Lucy merge/sync their dev environments with the Savannah configuration.
- Dev, QA completes for Michigan.
- Lucy merges/syncs her dev environment with Michigan.
Now let’s replay this scenario, but throw a wrench at it. Mary has been given the common but unreasonable task of starting Michigan with a strong dependency on the version of the zebra.gem being developed for Savannah. Mary can’t do her work without a basic Savannah zebra.gem. Here’s a legitimate case where Mary is going to beg for, use and possibly contribute to Savannah’s zebra.gem. She may need to frequently merge/sync her version of the zerbra.gem and possibly here entire dev configuration (if zebra.gem has strong dependencies) with Savannah’s dev configuration. It could mean a lot of pain and inefficiency for Mary’s development, and this is a management issue, but sometimes business interests win out over optimal development. It’s also possible that Mary and I have some kind of awesome Agile synergy and power through these two releases together.
What about Lucy? Zoo has little dependency on Savannah and Michigan. I’d say she should stick with a stable release and avoid Savannah and Michigan like the plague until there is a stable release and it becomes important to avoid a negative impact on her development and testing. Let’s say Lucy feels omniscient and wants to keep tabs on the awesome stuff in Michigan and Savannah. With Git, it’s easy to keep those releases as separate viewable branches, and keep it out of the Zoo dev configuration, but let’s say she pulls those changes into her Zoo dev configuration anyway. What risks is Lucy taking on?
- Risk of non-productive development time in a number of ways:
- The zebra.gem could be periodically inoperable, and though it is not a strict dependency of Zoo, it could still impact her ability to test.
- Adds complexity and mental drag to much of her work. These changes aren’t part of zoo – which are Lucy’s changes, which are others’? What does the target release code look like? I can’t tell because there are so many other changes in my configuration.
- Unforeseen dependency. Notice I’ve been using the word ‘omniscient’ a lot. Are you really omniscient? How well do you really know the code and dependencies? Ask yourself, “Do I feel lucky?” It can be good to identify unforeseen dependencies early in development, but there is also such a thing as “too early” for minor changes and this is not an argument to go Total Cowboy in development, Mr. I-Think-I’m-Agile. I admit waiting for a stable version to pass qa is likely too late. The purpose of syncing with stable Zoo and Savannah is to find those dependencies. If it were something major, there should be enough expertise on the team to find out early in the release planning. If this is not the case, then more effort needs to go into planning.
- Career risk. Lucy, what is your job? Are you fooling around with someone else’s project when you are supposed to working on the Zoo release? OK, hopefully you are on a cool, hip Agile team and that won’t happen, but beware of the organizational and even political impact of what you are doing.
So, I go with squeaky clean development configurations and invest in keeping it clean to avoid software development risk. I take only my planned changes against a code configuration that I think I’m going to release my changes into. MY_DEV = TARGET_RELEASE + MY_DELTAS. I also want to have an operable dev server that reflects my dev code configuration so I can test effectively. Keeping things simple makes me more productive.