Talk:04 Deployment & Release Management Procedure
Appearance
1 Introduction
Purpose:
- Create a clear, repeatable process for deploying and releasing software to production, ensuring teams can follow it.
Scope:
- Covers all software projects (web, mobile, backend) across development, staging, and production environments in the company.
Key Terms:
- Release: A versioned set of changes (e.g., v1.2.3).
- Deployment: Moving a release to an environment.
- Rollback: Reverting to the previous working version.
- Artifact: A compiled, versioned package (e.g., .jar, .zip).
- Environment: Dev, staging, or prod setups.
2. Roles and Responsibilities
Key Roles:
- Release Manager: Plans and tracks releases, coordinates approvals.
- Deployment Engineer: Runs deployments, checks environments.
- QA Lead: Verifies test results before deployment.
- Product Owner: Approves release scope and timing.
Deployment Team Tasks:
- Set up and validate servers or cloud environments (e.g., AWS, Azure).
- Run deployment scripts and monitor logs.
- Execute rollback if deployment fails.
Release Management Team Tasks:
- Schedule releases in Calendar.
- Collect acceptance from QA and Product Owners.
- Update release notes in a shared repo (e.g., GitHub Wiki).
Team Collaboration:
- Developers push code to Git; QA tests in staging; Ops confirms prod readiness.
- Daily standups or Slack channels for updates.
- Escalation Contacts: List names, contacts and emails for Release Manager, Lead, and Product owner for urgent issues.
3. Deployment & Release Management Process Overview
Core Steps:
- Create a release plan in Jira with scope, timeline, and dependencies.
- Deploy features in staging for QA verification.
- Fix bugs in relative branches.
- Complete integration testing.
- Deploy and demo on Test environment for stakeholders' acceptance.
- On formal acceptance and product owners' approval, prepare for Production deployment.
- Verify functionality, and rollback if issues arise.
- Document every step in Jira
- Close Jira ticket with logs and release notes.
SDLC Fit:
- Agile: Deploy after each sprint (e.g., every 2 weeks); link to sprint tickets in Jira.
Change Tracking:
- Log all code changes in Git with commit messages (e.g., “Fix #123: login bug”).
- Link commits to Jira tickets for traceability (e.g., JIRA-456).
- Use Git tags (e.g., v1.2.3) for release versioning.
Exceptions:
- Emergency releases: Skip staging, deploy hotfix directly to prod with CTO approval via Slack.
- Tag emergency tickets in Jira with “HOTFIX” prefix and log justification.
- Notify all teams via Slack #releases within 1 hour of emergency deployment.
Version Control:
- Use Git branches: main for prod, develop for dev, release/v1.2.3 for releases.
- Require pull requests (PRs) with at least 1 reviewer in GitHub
- Include commit messages with Jira ticket IDs (e.g., “JIRA-456: Add payment API”).
- Tag releases in Git (e.g., git tag v1.2.3) and push to repo.
4. Release Planning
Release Types:
- Major: Big major features scheduled quarterly.
- Minor: Small enhancements or Bug fixes scheduled sprint wise.
- Emergency: Critical fixes deployed same or next day.
Scheduling and communication:
- Create a release calendar in Google Calendar or Jira, shared with all teams.
- Avoid deployments during peak usage (e.g., 9 AM–5 PM weekdays for e-commerce apps).
- Set maintenance windows (e.g., Sunday 2 AM–4 AM) for major releases if required.
- Notify teams via email and communication 24 hours before employment.
- Update internal Confluence page with release details
- Include version number, release date, and Jira ticket IDs.
- Notify stakeholders via email about potential user-facing changes.
Release Criteria:
- Code passes 90%-unit test coverage.
- Zero critical or high-severity bugs in QA tests, tracked in Jira.
- Performance tests (e.g., JMeter) meet thresholds and latencies.
- Documentation for the deliverable is updated and reviewed.
Risk Checks:
- Verify database migrations won’t break existing data (e.g., run in staging first).
- Test rollback scripts in staging to ensure they work.
- Confirm no conflicts with other scheduled deployments or maintenance.
Approvals:
- Product Owner approves release scope in Jira ticket (e.g., REL-789).
- QA Lead signs off on tests via Jira comment or email.
- Tech lead approves emergency releases via Email or Teams
- Log all approvals in Jira with timestamps and usernames.
Dependencies:
- List external dependencies if any in Jira.
- Check dependency versions in staging before prod.
5. Deployment Process
Pre-Deployment Activities:
Environment Setup:
- Check server health (CPU <80%, memory <70%).
- Verify database connectivity (e.g., MySQL, PostgreSQL) with test queries.
- Validate SSL certificates are active.
Build and Packaging:
- Run build scripts to create artifacts.
- Create database migration release script.
- Push artifacts to Artifactory with version tags.
- Tag build in Git (e.g., git tag v1.2.3) and push to repo.
Checklist:
- Confirm prod backups are complete.
- Verify network connectivity.
- Check disk space on prod servers.
- Ensure rollback artifacts are available in Artifactory.
Backups:
- Create database backups.
- Backup configuration files.
- Test and validate restore process in staging to confirm backups are usable.
Deployment Execution:
Checklist:
- Verify artifact version matches Git tag.
- Check dependencies are installed.
Execution
- Run deployment script or manual deployment.
Monitoring:
- Watch server logs.
- Monitor CPU, memory, and response times.
Post-Deployment Activities:
Verification:
- Run smoke tests (e.g., login, API health check) in prod within 10 minutes.
- Check performance (e.g., <200ms response time) manually or in Grafana dashboards.
- Confirm key features work via manual checks or automated scripts.
Rollback:
- Revert to previous artifact in Artifactory (e.g., app-v1.2.2.jar).
- Restore database from last backup.
- Test rollback in staging first if time permits.
- Notify teams if rollback occurs.
Notifications:
- Email clients with release notes for major releases (e.g., new UI features).
- Email specific stakeholders for hotfix.
- Notify stakeholders of any user-facing changes via email.
Documentation:
- Update release notes in GitHub Wiki with features, fixes, and issues.
- Record new environment settings (e.g., updated API keys) in Confluence.