01.04 Perform a program/project discovery

At the beginning of our implementations at Stelligent, we perform a project, program or organizational discovery so that we have the means to make a before and after comparison. There’s no point in making an investment in this kind of change without a way to demonstrate how the change impacted the team. We have a tool that we’ve created called a cloud delivery scorecard. The scorecard manages questions and answers, provides recommendations and provides a means of comparison across industries. Moreover, there’s an automated subsystem of the scorecard that we hook up to our customers’ AWS accounts and cloud delivery components to provide a real-time analysis of certain attributes that indicate their ability to perform cloud delivery. We provide this tool as a free service to our existing customers.

You can go to scorecard.stelligent.com to get a list of the questions we ask along with some example screenshots of the recommendations and the real-time indicators our customers see.

Scorecard (Source: stelligent.com).

Example project discovery questions

If you don’t want to use the Scorecard, we’ve provided some sample questions that we ask of project stakeholders.

  • Which version control repository do you use?
  • Is 100% of the sensitive environment data separated and in a secure environment such as a database, directory service, etc.?
  • Is the entire software system versioned at all times: environment, data, source files, configuration?
  • Is the protected configuration data versioned with the rest of the software system/application?
  • Is anyone able to spin up a complete new (demo) environment in less than hour using the infrastructure?
  • Which storage mechanisms are being used?
  • What types of files and/or data types are managed on these storage mechanisms?
  • Which ports must be opened inside your network?
  • Which ports must be opened outside your network?
  • Which firewall software is used?
  • How are switches used to restrict external access?
  • Which applications use SSL and what are the domains?
  • What is the process for requisitioning hardware?
  • When new hardware is required, approximately how long does it take to procure it?
  • How do you guarantee that OS and application servers are consistently configured across tiers?
  • Which applications are installed on the various hardware instances? Provide the application name, directory location, version, URL, etc.
  • Which software is used to run these applications?
  • Are teams able to perform centralized remote deployments to different target environments?
  • How do you handle duplicate dependencies between projects (i.e. dependency management) (Ivy and Maven are examples of tools that support centralized dependency management)?
  • How often do developers checkin all their code to repository?
  • Which language/technology platform?
  • How many times a year do you release software to users (on average)?
  • How many target environments per pipeline (i.e. project)?
  • Is there a suite of products released at once or do you release by product?Which development methodology do developers use?
  • Is there a process for requisitioning hardware and managing deployments?
  • What’s the preferred mode for sharing knowledge? (e.g. Wiki?)
  • Which build tools are you using?
  • How many projects are there in your organization?
  • Are you leveraging cloud computing? If so, how?
  • Does your build/deploy process automate 100% of all aspects of deployments?
  • Are all teams in the delivery process (dev, dba, qa, operations, etc.) using the same script or set of scripts?
  • Are 100% of the provisioning scripts to create environment instances checked into the version-control repository?
  • Which teams modify project build scripts?
  • Which teams modify project deployment scripts?
  • Which teams modify environment provisioning scripts?
  • Are your environment instances “disposable”?
  • Are teams able to perform centralized remote deployments to different target environments?
  • Are automated unit tests run with the automated build?
  • Are automated component/integration (e.g. database) tests run with build process?
  • Are automated functional tests run with the build process?
  • Are automated load and performance tests run with the build process?
  • Which Continuous Integration server do you use? (if applicable)
  • What kind of documentation does your build generate?
  • Are you using virtualization? If so, which tool(s) and how?
  • Are your builds utilizing any of the following static analysis reports?
  • When target environment are provisioned, does it require someone to configure each of these environments or is it one click?
  • Is there a QA team and/or are they paired w/ development teams?
  • Is there a separate Systems/Operations team that manages deployments?
  • Are DBA’s separate from development teams?
  • Are there particular security restrictions with certain environments (e.g. developers can’t modify certain machines, etc.)?
  • Is all the source code, configuration, data and host environment (everything to provision, deploy and build software) checked into the version control repository?
  • What’s the typical lifespan of a defect?
  • How many total hours are spent on deploying software internally (before it gets to production) for each release?
  • Is there a process of build promotion (i.e. from DEV to QA, etc.)?
  • Describe the process for build promotion (i.e. from DEV to QA to STAGE, etc.)
  • Describe the step-by-step process of provisioning one of the target environments
  • Describe the step-by-step process of deploying to one of the target environments
  • Are you able to automatically rollback (or click a button) a software release?
  • How often does the entire delivery team have retrospectives to discuss process improvements?
  • Do the deployments run an automated smoke test to verify a successful deployment?
  • Is any authorized team member able to perform self-service deployments?
  • Is the instance creation (i.e. provisioning) 100% automated and headless (can be executed without a human)?
  • Which database(s) are your teams using?
  • Does your automated build/deploy process perform database integration and upgrades?
  • Are ALL SQL scripts (DDL and DML) checked into version control repository?
  • How do you make sure all required database scripts are run?
  • How will you know the current state of the database to figure which scripts need to be run.
  • How do you make sure that scripts are not run more than once?
  • How do you make sure database scripts do not change between builds to different environments?
  • If applicable, which database upgrade tool do you use (e.g. dbdeploy, Liquibase or other)?
  • How do you connect to various hardware and software resources – passwords, private keys, etc.?
  • Provide the location for various credentials to access these hardware and software resources?
  • Are you able to track the precise cost of each hardware/virtual instance in use, in real time? If so, how?
  • Are you able to track the precise purpose of each hardware/virtual instance in use – in real time? If so, how?
  • Are you able to track the precise utilization of each hardware/virtual instance in use – in real time? If so, how?
  • Where do you manage your software licenses?
  • How do you manage your software licenses? Where do you track the relevant license information?
  • Which resources are backed up?
  • How often are these resources backed up?
  • What’s not being backed up?
  • Has there ever been the need to recover once something goes wrong?
  • What is the process for recovery when something does go wrong?
  • How often do you test this recovery process?
  • How do you track defects and other issues on projects?
  • How do you develop and share schedules or plans on projects?
  • What’s the preferred mode for sharing knowledge? (e.g. Wiki?)
  • Is your project manager tool/server managed in the cloud – Software as a Service
  • Is the provisioning of new environments on-demand?
  • Is the provisioning of new environments self-service (i.e. users (e.g. developers) do not need to go through a separate team to assist in the provisioning of the environment)?
  • Is any authorized user able to determine the cost of each instance, in real time?
  • How do you account for future capacity?
  • Can transient instances be terminated by an authorized user without contacting anyone else?
  • Show list of questions along with results and recommendations from these questions?
  • Are all provisioning processes the same, with the exception of environment-specific properties?
  • Are all provisioning processes the same across environments, with the exception of environment-specific properties?
  • How many non-development people are in the organization (e.g. QA, DBAs, Operations, etc.)
  • How many developers are there in the organization?
  • Use SaaS-based issue tracking and project management
  • Which language/technology platform? Java, .NET, Ruby/Rails, PHP, Other?
  • How many software applications are there in your organization? 1
  • How many non-development people are in the organization (e.g. QA, DBAs, Operations, etc.)
  • Did we receive an organization chart of my area of focus that includes titles, areas of responsibility, who reports to whom, etc.?
  • Are teams able to perform centralized remote deployments to different target environments?
  • How many developers are there in the organization?
  • How often do developers checkin all their code to repository?
  • How many times a year do you release software to users (on average)?
  • Is there a suite of products released at once or do you release by product?
  • How many target environments per pipeline (i.e. project) Do you have separate Development, QA, Stage and Production environments?
  • Are you leveraging cloud computing? If so, how?
  • Which development methodology do developers use?
  • Is there a process for requisitioning hardware and managing deployments?
  • What’s the preferred mode for sharing knowledge? (e.g. Wiki?)
  • Which build tools are you using?
  • How do you handle duplicate dependencies between projects (i.e. dependency management) (Ivy and Maven are examples of tools that support centralized dependency management)
  • Does your automated build/deploy process perform database integration and upgrades? An automated process that doesn’t include “export and import” of the entire database.
  • Are ALL SQL scripts (DDL and DML) checked into version control repository? Can you recreate your database using only DDL and DML scripts checked into your version-control repo?
  • If applicable, which database upgrade tool do you use (e.g. dbdeploy, Liquibase or other)?
  • Which database(s) are your teams using?
  • Does your build/deploy process automate 100% of all aspects of deployments? For example: Checking port usage; configuring ports, Modifying datasource connections, Configuring internal security attributes (e.g. JAAS, login-config, propertiesservices, etc.), Configuring LDAP (if necessary), Configuring SMTP server(s), Configuring web services, Configuring SSL, Other
  • Which teams modify project build scripts?
  • Which teams modify project deployment scripts?
  • Which teams modify environment provisioning scripts?
  • Are teams able to perform centralized remote deployments to different target environments?
  • Are automated unit tests run with the automated build?
  • Are automated component/integration (e.g. database) tests run with build process?
  • Are automated functional tests run with the build process?
  • Are automated load and performance tests run with the build process?
  • Which Continuous Integration server do you use?
  • Are you using virtualization? If so, which tool(s) and how?
  • Are your builds utilizing any of the following static analysis reports?
  • Is there a QA team and/or are they paired w/ development teams?
  • Are there particular security restrictions with certain environments (e.g. developers can’t modify certain machines, etc.)?
  • What kind of documentation does your build generate?
  • What’s the typical lifespan of a defect? That is, what is the duration between discovering the defect and fixing the defect?
  • Is there a process of build promotion (i.e. from DEV to QA, etc.)?
  • How many total hours are spent on deploying software internally (before it gets to production) for each release? Not creating the scripts; just the act of deploying from, say. Dev to QA to Stage
  • Describe the process for build promotion (i.e. from DEV to QA to STAGE, etc.) *
  • Describe the step-by-step process of deploying to one of the target environments *
  • Are all provisioning processes the same, with the exception of environment-specific properties?
  • Is the instance creation (i.e. provisioning) 100% automated and headless (can be executed without a human)? Are all steps scripted (or other) and checked into the version control system such that there are zero manual steps?
  • Are 100% of the provisioning scripts to create environment instances checked into the version-control repository?
  • Describe the step-by-step process of provisioning one of the target environments *
  • Are you able to track the precise purpose of each hardware/virtual instance in use – in real time?
  • Are your environment instances “disposable”? That is, is anyone on the delivery team able to perform a full deployment from a clean machine without installing other tools or performing manual configuration and generate working, testable software?
  • When target environment are provisioned, does it require someone to configure each of these environments or is it one click?
  • Are all provisioning processes the same across environments, with the exception of environment-specific properties?
  • Are you able to track the precise cost of each hardware/virtual instance in use, in real time?
  • Are you able to track the precise utilization of each hardware/virtual instance in use – in real time?
  • Is all the source code, configuration, data and host environment (everything to provision, deploy and build software) checked into the version control repository?
  • Is 100% of the sensitive environment data separated and in a secure environment (such as a database, directory service, etc.)?
  • Which version control repository do you use?
  • is the protected configuration data versioned with the rest of the software system/application?
  • Is the entire software system versioned at all times: environment, data, source files, configuration?
  • Are all teams in the delivery process (dev, dba, qa, operations, etc.) using the same script or set of scripts?
  • Is any authorized team member able to perform self-service deployments? Pattern: Self-Service Deployment. Select an environment, a revision number and click a button/type one command?
  • Do the deployments run an automated smoke test to verify a successful deployment? Deployment/Configuration Test, Host Environment Test, Data Test, Code Test
  • Are you able to automatically rollback (or click a button) a software release?
  • Is there a separate Systems/Operations team that manages deployments?
  • How often does the entire delivery team have retrospectives to discuss process improvements?
  • Are DBA’s separate from development teams?
  • What is the process for requisitioning hardware?
  • When new hardware is required, approximately how long does it take to procure it?

Deliverables

Below, you’ll find a complete list of the typical deliverables in a scorecard/discovery. You might choose to only focus on the current scores against the patterns. At a minimum, the idea is to assess the current state so that the team knows what improvement looks like when it happens. Think of it as the “before picture” prior to entering a weight-loss program. If you choose to provide detailed recommendations, then this becomes the action plan in how to go about “losing the weight”.

  • List of goals, findings, risks and recommendations
  • Current value-stream map
  • A score for each continuous delivery patterns (over 30 patterns) and how the organization ranks against other companies across industries
  • A roadmap to implement the recomendations

 

Leave a Reply