Wednesday, April 20, 2016

Sprint 2 - Retrospective

Work-Life-Music Balance

Completion of Sprint 2 took longer than expected. Partial to blame is a recent expansion of duties at work (i.e. putting out fires with the latest release) which has resulted in long hours at the office. Also partial to blame: my past few weekend were dominated with attending music events. Umphrey's McGee, Chick Corea and Bela Fleck, the New Orleans Opera: Puccini's Tosca, and several bouts of karaoke with friends (ohhhhhh, sweet child of miiiiiine) :).

For Goodness Sake; Document your Deploy Procedures!

Part of my increased duties at work includes overseeing a production deploy. At the moment our production deploy is a small feat, requiring half a dozen people during late evening hours (when there is least impact to our users). It has been very educational. I've had to embrace making go/abort judgement calls, making risk assessment, exercise kurt communication during crisis events, and improving procedures to minimize any 'developer gets hit by a bus' scenarios (i.e. better documentation).

On the movie making front, at the end of each sprint I cut an updated version of the film. Naturally I couldn't help but apply some of my new-found deploy discipline. I've learned that even the most seemingly obvious steps are exposed for all their complexities (and kludges) once written out. Even a procedure so seemingly easy as 'backup footage' requires a couple hours to properly document. As annoying as it might be, it's always an extremely valuable exercise. Pedantic and fool-proof.
Take a gander at the result;

Synchronicity Move Release Requirements:

  • Access to project JIRA (
  • Access to the production computer
  • Access to the backup drive
  • Full procedure should take about 45min
Synchronicity Movie Release Procedure:

  1. Compile list of scenes touched during this sprint 
    • In JIRA, reference the scene labels of all the tickets closed during sprint.
  2. Export each scene in the list
    • In Premiere, for each scene, select the corresponding sequence in the project panel, right click, the select "Export Media". The Export Settings dialog will open; leave all the settings as is, then click "Queue".
    • After all sequence have been queue, switch to the Adobe Media Exporter application and click green arrow button at the top.
    • This operation will take about 15 minutes.
    • Close both Premiere and Media Encoder when the operation is finished
  3. Move the exported clips into their own folder
    • Go to the output directory for the film clips (C:/User/.../Adobe/Premier Pro/9.0)
    • Create new folder called Sprint_##. (e.g. "Sprint_02")
    • Move to newly exported clips into the Sprint ## folder
  4. Backup files
    • Retrieve the back drive from its secure location
    • Plug in the USB external harddrive.
    • WARNING: Shortly after plugging in the drive Windows will prompt with a warning dialog suggesting that the drive is corrupted. DO NOT CLICK THE OPTION TO SCAN AND FIX. (This will ruin the drive). Instead click the option to continue without scanning.
    • The drive will appear in explorer as "Seagate Backup Plus Drive"
    • Copy the Sprint_## folder to the external drive
    • Copy the Adobe Premiere project file ("C:\Users\spike\Documents\Adobe\Premiere Pro\9.0\Synchronicity_sc19_sc20")
    • Note: Ignore the fact that the file is named "_sc19_sc20". In reality this file contains work for all scenes. 
    • Change the name of the copy to include the current date as a suffix (e.g. Synchronicity_sc19_sc20_2016-04-20)
    • Copy the premier project file from the previous step to the external drive.
    • Wait for the copy to finish; should take a couple minutes
    • Eject the drive (read
    • Unplug drive from USB
    • Place the USB drive back to its secure location
Misc Release Notes:

  • There is no need to re-export every scene for every release. Only the scenes that have been touched.
  • There is no need to re-backup the original raw footage; the raw footage is already backed up. Only the premeire project file is changed.
  • The only purpose of the backup drive is ONLY for backing up movie files. 
  • Because of the confusing error message windows give NEVER lend the drive to anyone.

Evaluating Risk

One of the benefits of documenting your release procedure is it helps expose areas of risk. I'm talking about 'sink the company' levels of risk. Naturally I had similar insights after documenting my movie release procedures:

Current Sources of Risk:

Sound effects are not part of the back up procedure
Risk: Medium
Impact: Medium

  • This release procedure needs to be updated to include new sound effects and new raw footage added during the sprint.
  • At the moment my file organization for sound effects is attrocious (some files are in Downloads folder, some are on desktop, etc.)
  • I'll need to select a common place for sound effects to live, and update the file dependencies in the premier project
  • This represents some technical debt that will only get worst with each release.
  • Worst case scenario: About a week's worth of would would be lost.
  • Action Items: Ticket SM-159

Power outage during back up
Risk: Low
Impact: Very High

  • The production machine does not have a battery backup. 
  • Worst case scenario, a power outage occurs when copying files to the backup drive and both copies of the files are destroyed (about a 5 minutes window).
  • Raw footage is also backedup on another laptop; but we would lose all the premiere files. 
  • Worst case scenario: Over a year of work will be lost.
  • In New Orleans, expect about 5 power outages per year. Almost always in conjunction with heavy weather; thus do not perform backup during heavy weather.
  • Action Items: Get a UPS power supply for the production machine; Ticket SM-160

All sources of files are colocated
Risk: Very Low
Impact: Very Low

  • In the event of a natural disaster, all sources of files could be destroyed.
  • The backup drive is stored in a fire proof safe; but it is not water proof.
  • The backup drive is small and portable. The backup drive is already stored as part of my 'go bag'. It would only take a few seconds to evacuate the building with the backup drive.
  • Action Items: None

Looks like I'll be shopping for a UPS power supply pretty soon...

No comments:

Post a Comment