Zero-downtime migration tooling; widen File.file_size to bigint (expand stage)#5986
Open
rtibbles wants to merge 3 commits into
Open
Zero-downtime migration tooling; widen File.file_size to bigint (expand stage)#5986rtibbles wants to merge 3 commits into
rtibbles wants to merge 3 commits into
Conversation
bjester
reviewed
Jun 29, 2026
bjester
left a comment
bjester
left a comment
Member
There was a problem hiding this comment.
Overall, the code looks good and makes sense. I have lots of comments and a few questions for deliberation. After that, I'll look at testing it
- Safe-DDL backend: lock timeout, fail-fast, runtime unsafe-op guard - CI linting of new migrations on pull requests - Declarative dual-write trigger decorator (mirror_field) - Reusable batched-backfill command (idempotent, resumable, throttled) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LfZvkigk8hdsKdEif3hzBi
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LfZvkigk8hdsKdEif3hzBi
Expand stage of the zero-downtime int->bigint widening: - Add nullable file_size_bigint shadow column and its index (built CONCURRENTLY). - Mirror file_size into it via the change-guarded @mirror_field trigger. - Wire the online backfill as a commented deploy-migrate step. - Stage the cutover and contract steps as comments on the File model. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01LfZvkigk8hdsKdEif3hzBi
rtibbles
force-pushed
the
widen_file_size_bigint
branch
from
4f79103 to
84f402d
Compare
Member
|
So I have a slightly older SQL dump with a bunch of data that I tried to use to test this out. While this won't occur on rollout because these migrations have already run, it does appear that the zero-downtime package aggressively blocks migrations that it deems are not zero-downtime. This likely explains why AI decided to put the clear documented note about not doing column renames: |
Member
|
So besides the aforementioned potential issue, testing this looks good. Migration ran 10k chunks in about 1 second each. I killed it half way through, and re-ran with |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Add tooling and playbook for zero downtime migrations for changing field types, e.g. int to bigint, or char to uuid - implement start of migration for file size on File object
Create reusable tooling for migration of file_size and other fields (like our CharField UUIDs to proper UUIDFields) - allow upload of files bigger than 2.1GB > int max value
Implemented with clear tooling to help ensure we implement with guardrails and clear repeatable steps.
References
Fixes #5973
Fixes #5974
Reviewer guidance
settings.py,production_settings.py): DB backend now routes throughdjango-pg-zero-downtime-migrations.0167: nullable shadow column + concurrent index + mirror trigger — all safe DDL (the backend forcesatomic=False, so the in-migrationCREATE INDEX CONCURRENTLYis fine).deploy-migrateruns the backfill this release. New writes dual-write via the trigger.AI usage
Used Claude Code to implement the tooling (safe DB backend wiring, the
mirror_fielddual-write trigger, thebackfill_columncommand) and draft the runbook. I reviewed the output across several local review rounds for correctness, tightened it to Studio's conventions, and ran the test suite.