-
Notifications
You must be signed in to change notification settings - Fork 80
feat: Clean cache command #1394
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Open
d3xter666
wants to merge
70
commits into
main
Choose a base branch
from
feat-clean-cache
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Open
Changes from all commits
Commits
Show all changes
70 commits
Select commit
Hold shift + click to select a range
8280272
feat: Clean cache command
d3xter666 1971750
refactor: Use single place for DB manipulation
d3xter666 637f2b5
refactor: Simplify cache clean
d3xter666 d302ef6
refactor: Position correctly the CacheCleanup
d3xter666 0c16994
refactor: Add confirmation dialog for the cache clean command
d3xter666 b0c3f3b
refactor: Rename cacheVersionDir
d3xter666 21e2e89
refactor: Restore location of CacheCleanup
d3xter666 329ba63
fix: Clean only current cache version
d3xter666 884d024
refactor: Simplify CacheCleanup
d3xter666 046c401
test: Improve coverage
d3xter666 b53c357
refactor: CLI package orchestrates cache cleanup
d3xter666 81c130a
refactor: Add skip confirmation option
d3xter666 21b5277
fix: Windows paths for tests
d3xter666 406058d
refactor: Use yesno package for CLI confirmation
d3xter666 5b099ce
refactor: Simplify cleanup meta structure
d3xter666 2f4e7a0
fix: Add guard to not accidently create a new DB
d3xter666 6b92cbd
refactor: Reuse meta from installers in cache cleanup
d3xter666 9558e61
refactor: Cleanup details
d3xter666 cfde118
fix: Respect datadir config
d3xter666 086c9e3
docs: Update cache clean --help CLI information
d3xter666 a51c16d
refactor: Cleanup confirmation revised
d3xter666 6fa610a
refactor: Revise actual cleanup UX
d3xter666 4ee4e5f
fix: Docs generation failure
d3xter666 5c2cac8
fix: Respect cleanup locking
d3xter666 5045721
refactor: Redundant cleanup summary
d3xter666 b38256f
fix: Cache locking race condition
d3xter666 132a7e2
test: Fix race condition expectations
d3xter666 84d590b
refactor: Cleanups
d3xter666 73cd772
refactor: Naming of locks
d3xter666 8c73b1e
docs: Update documentation to respect recent changes
d3xter666 6c9872d
revert: Redundant maven installer locks
d3xter666 0501f1b
refactor: Use pacote's internals for its own cleanup
d3xter666 cb1f1d4
revert: NPM Install sync. It's now redundant
d3xter666 6f02c79
docs: No mention of "incremental build"
d3xter666 6391071
refactor: Avoid hardcoded values
d3xter666 016bd18
feat: Add cleanup lock for "ui5 serve"
d3xter666 137360d
refactor: Move server lock into the server package and reuse in cli
d3xter666 f5e5d86
refactor: Data dir is resolved centrally via util
d3xter666 b7817b1
fix: Remove redundant lock cleanups after server close
d3xter666 970156f
refactor: Revise locks destinations and messages
d3xter666 de109a2
refactor: Lock full build, so that it does not run into timing problems
d3xter666 9fce7eb
refactor: Avoid cache cleanup + build race conditions
d3xter666 c2db60e
refactor: Cleanups
d3xter666 d258614
refactor: Use the common test/tmp dir
d3xter666 4e36bd0
refactor: Move server lock to the ProjectGraph.serve
d3xter666 2c534e3
refactor: Rename getDefaultUi5DataDir to resolveUi5DataDir
d3xter666 40a861e
refactor: Do not pass dir (cwd) as argument to resolveUi5DataDir
d3xter666 bbecbd6
refactor: Cleanup of redundant lock watchers
d3xter666 937acfb
refactor: Cleanup of ProjectGraph
d3xter666 596f898
refactor: Acquire locks in the project builder & server
d3xter666 873712e
test: Fix stubs
d3xter666 486340b
test: Update lock util tests
d3xter666 9b333b9
test: Server integration
d3xter666 6d87bf6
refactor: DB size optimization
d3xter666 0d446a6
refactor: Split cache command into logical chunks
d3xter666 49c560d
refactor: Consolidate common logic for CacheManager
d3xter666 40c19a4
refactor: Consolidate repetative code
d3xter666 c8b4484
refactor: Lock cleanups
d3xter666 e71b97d
test: Cleanup of repetative cases
d3xter666 df56779
refactor: Provide dataDir to the buildServer
d3xter666 abeb88d
refactor: Enhance locks
d3xter666 6b80d8b
refactor: Cleanup locks for wrong places
d3xter666 2276a08
refactor: Locks for ProjectGraph
d3xter666 07cffbc
test: Fix unstable tests
d3xter666 5cac4c7
docs: JSDoc updates
d3xter666 98389e5
fix: Separate async & sync dir creation
d3xter666 a31c263
refactor: Add ui5DataDir as required in ProjectGraph's constructor
d3xter666 0630e18
refactor: Give a name for the graph's internal locking method
d3xter666 747e1e0
refactor: Move locks to the specific lock points
d3xter666 db6b3de
feat: ProjectGraph's lock now has a tolerance to wat for the cache cl…
d3xter666 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,260 @@ | ||
| import chalk from "chalk"; | ||
| import path from "node:path"; | ||
| import process from "node:process"; | ||
| import baseMiddleware from "../middlewares/base.js"; | ||
| import {resolveUi5DataDir} from "@ui5/project/utils/dataDir"; | ||
| import {getLockDir, hasActiveLocks} from "@ui5/project/utils/lock"; | ||
| import * as frameworkCache from "@ui5/project/ui5Framework/cache"; | ||
| import CacheManager from "@ui5/project/build/cache/CacheManager"; | ||
|
|
||
| const cacheCommand = { | ||
| command: "cache", | ||
| describe: "Manage the UI5 CLI cache (downloaded framework packages and build data)", | ||
| middlewares: [baseMiddleware], | ||
| handler: handleCache | ||
| }; | ||
|
|
||
| cacheCommand.builder = function(cli) { | ||
| return cli | ||
| .demandCommand(1, "Command required. Available command is 'clean'") | ||
| .command("clean", "Remove all cached UI5 data", { | ||
| handler: handleCache, | ||
| builder: function(yargs) { | ||
| return yargs | ||
| .option("yes", { | ||
| alias: "y", | ||
| describe: "Skip the confirmation prompt, e.g. for use in CI pipelines", | ||
| default: false, | ||
| type: "boolean", | ||
| }) | ||
| .example("$0 cache clean", | ||
| "Remove all cached UI5 data after confirming the prompt") | ||
| .example("$0 cache clean --yes", | ||
| "Remove all cached UI5 data without confirmation (e.g. in CI)") | ||
| .example("UI5_DATA_DIR=/custom/path $0 cache clean", | ||
| "Remove cached data from a non-default UI5 data directory") | ||
| .epilogue( | ||
| "The cache is stored in the UI5 data directory (default: ~/.ui5).\n" + | ||
| "Override the location with the UI5_DATA_DIR environment variable or\n" + | ||
| "the 'ui5DataDir' configuration option (see 'ui5 config --help').\n\n" + | ||
| "Two cache types are removed:\n" + | ||
| " UI5 Framework packages Downloaded UI5 library files " + | ||
| "(~/.ui5/framework/)\n" + | ||
| " Build cache (DB) build data " + | ||
| "(~/.ui5/buildCache/)" | ||
| ); | ||
| }, | ||
| middlewares: [baseMiddleware], | ||
| }); | ||
| }; | ||
|
|
||
| const LABEL_FRAMEWORK = "UI5 Framework packages"; | ||
| const LABEL_BUILD = "Build cache (DB)"; | ||
| // Pad labels to equal width for two-column alignment | ||
| const LABEL_WIDTH = Math.max(LABEL_FRAMEWORK.length, LABEL_BUILD.length); | ||
|
|
||
| /** | ||
| * Format a byte size as a human-readable string. | ||
| * | ||
| * @param {number} bytes Size in bytes | ||
| * @returns {string} Formatted size string | ||
| */ | ||
| function formatSize(bytes) { | ||
| if (bytes < 1024) { | ||
| return `${bytes} B`; | ||
| } else if (bytes < 1024 * 1024) { | ||
| return `${(bytes / 1024).toFixed(1)} KB`; | ||
| } else if (bytes < 1024 * 1024 * 1024) { | ||
| return `${(bytes / (1024 * 1024)).toFixed(1)} MB`; | ||
| } | ||
| return `${(bytes / (1024 * 1024 * 1024)).toFixed(1)} GB`; | ||
| } | ||
|
|
||
| /** | ||
| * Format framework cache stats as a human-readable detail string. | ||
| * E.g. "1,189 versions of 155 libraries" or "1 version of 1 library". | ||
| * | ||
| * @param {number} libraryCount | ||
| * @param {number} versionCount | ||
| * @returns {string} | ||
| */ | ||
| function formatFrameworkStats(libraryCount, versionCount) { | ||
| const v = `${versionCount.toLocaleString("en-US")} ${versionCount === 1 ? "version" : "versions"}`; | ||
| const l = `${libraryCount.toLocaleString("en-US")} ${libraryCount === 1 ? "library" : "libraries"}`; | ||
| return `${v} of ${l}`; | ||
| } | ||
|
|
||
| /** | ||
| * Pad a label to the shared column width. | ||
| * | ||
| * @param {string} label | ||
| * @returns {string} | ||
| */ | ||
| function padLabel(label) { | ||
| return label.padEnd(LABEL_WIDTH); | ||
| } | ||
|
|
||
| /** | ||
| * Display information about the cached data that will be removed, | ||
| * including the absolute paths and details about the framework and build caches. | ||
| * | ||
| * @param {*} data | ||
| * @param {object} data.frameworkInfo | ||
| * @param {object} data.buildInfo | ||
| * @param {string} data.frameworkAbsPath | ||
| * @param {string} data.buildAbsPath | ||
| * @param {number} data.buildPreSize | ||
| */ | ||
| async function displayCacheInfo({ | ||
| frameworkInfo, | ||
| buildInfo, | ||
| frameworkAbsPath, | ||
| buildAbsPath, | ||
| buildPreSize, | ||
| }) { | ||
| // Display items that will be removed | ||
| process.stderr.write(chalk.bold("\nThe following cached data will be removed:\n\n")); | ||
| if (frameworkInfo) { | ||
| const detail = formatFrameworkStats(frameworkInfo.libraryCount, frameworkInfo.versionCount); | ||
| process.stderr.write( | ||
| ` ${chalk.yellow("•")} ${padLabel(LABEL_FRAMEWORK)} ${frameworkAbsPath} (${detail})\n` | ||
| ); | ||
| } | ||
| if (buildInfo) { | ||
| const detail = buildPreSize > 0 ? formatSize(buildPreSize) : ""; | ||
| process.stderr.write( | ||
| ` ${chalk.yellow("•")} ${padLabel(LABEL_BUILD)} ${buildAbsPath} (${detail})\n` | ||
| ); | ||
| } | ||
| process.stderr.write("\n"); | ||
| } | ||
|
|
||
| /** | ||
| * Display the result of the cache cleanup operation, | ||
| * including which caches were removed and their details. | ||
| * | ||
| * @param {object} data | ||
| * @param {object} data.frameworkResult | ||
| * @param {object} data.buildResult | ||
| * @param {string} data.frameworkAbsPath | ||
| * @param {string} data.buildAbsPath | ||
| * @param {number} data.buildPreSize | ||
| */ | ||
| async function displayCleanupResult({ | ||
| frameworkResult, | ||
| buildResult, | ||
| frameworkAbsPath, | ||
| buildAbsPath, | ||
| buildPreSize, | ||
| }) { | ||
| process.stderr.write("\n"); | ||
| if (frameworkResult) { | ||
| const detail = formatFrameworkStats( | ||
| frameworkResult.libraryCount, | ||
| frameworkResult.versionCount, | ||
| ); | ||
| process.stderr.write( | ||
| `${chalk.green("✓")} Removed ${chalk.bold(LABEL_FRAMEWORK)}` + | ||
| ` (${frameworkAbsPath} · ${detail})\n`, | ||
| ); | ||
| } | ||
| if (buildResult) { | ||
| // Use pre-clean size so the number matches what was shown before confirmation | ||
| const detail = buildPreSize > 0 ? formatSize(buildPreSize) : ""; | ||
| process.stderr.write( | ||
| `${chalk.green("✓")} Removed ${chalk.bold(LABEL_BUILD)}` + | ||
| ` (${buildAbsPath}${detail ? ` · ${detail}` : ""})\n`, | ||
| ); | ||
| } | ||
|
|
||
| // Success summary | ||
| const cleaned = []; | ||
| if (frameworkResult) { | ||
| cleaned.push(LABEL_FRAMEWORK); | ||
| } | ||
| if (buildResult) { | ||
| cleaned.push(LABEL_BUILD); | ||
| } | ||
| process.stderr.write( | ||
| `\n${chalk.green("Success:")} Cleaned ${cleaned.join(" and ")}\n`, | ||
| ); | ||
| } | ||
|
|
||
| /** | ||
| * Prompt the user for confirmation before proceeding with cache cleanup. | ||
| * | ||
| * @param {Yargs.Arguments} argv | ||
| * @returns {Promise<boolean>} Confirmation result | ||
| */ | ||
| async function getConfirmation(argv) { | ||
| if (argv.yes) { | ||
| return true; | ||
| } | ||
| const {default: yesno} = await import("yesno"); | ||
| return yesno({ | ||
| question: "Do you want to continue? (y/N)", | ||
| defaultValue: false | ||
| }); | ||
| } | ||
|
|
||
| async function handleCache(argv) { | ||
| // Resolve UI5 data directory — uses the same resolution chain as ui5 build/serve: | ||
| // UI5_DATA_DIR env var → ui5DataDir config (~/.ui5rc) → default ~/.ui5 | ||
| // Relative paths are resolved against process.cwd() (project root when invoked from the project). | ||
| const ui5DataDir = await resolveUi5DataDir(); | ||
|
|
||
| // Abort early if a lock is active — before prompting the user | ||
| if (await hasActiveLocks(getLockDir(ui5DataDir))) { | ||
| process.stderr.write( | ||
| `${chalk.red("Error:")} A UI5 server or build process is currently running. ` + | ||
| "Cannot clean the cache while it is in use. " + | ||
| "Please stop all running 'ui5 serve' or wait for 'ui5 build' processes to finish.\n" | ||
| ); | ||
| process.exitCode = 1; | ||
| return; | ||
| } | ||
|
|
||
| // Inform the user immediately — getPackageStats may take a moment on a large cache | ||
| process.stderr.write(`Checking cache at ${chalk.bold(ui5DataDir)} …\n`); | ||
|
|
||
| const frameworkInfo = await frameworkCache.getCacheInfo(ui5DataDir); | ||
| const buildInfo = await CacheManager.getCacheInfo(ui5DataDir); | ||
|
|
||
| // Compute absolute paths once — producers return relative sub-path segments | ||
| const frameworkAbsPath = frameworkInfo ? path.join(ui5DataDir, frameworkInfo.path) : null; | ||
| const buildAbsPath = buildInfo ? path.join(ui5DataDir, buildInfo.path) : null; | ||
| const buildPreSize = buildInfo?.size ?? 0; | ||
|
|
||
| if (!frameworkInfo && !buildInfo) { | ||
| process.stderr.write("Nothing to clean\n"); | ||
| return; | ||
| } | ||
|
|
||
| await displayCacheInfo({ | ||
| frameworkInfo, | ||
| buildInfo, | ||
| frameworkAbsPath, | ||
| buildAbsPath, | ||
| buildPreSize, | ||
| }); | ||
|
|
||
| const confirmed = await getConfirmation(argv); | ||
| if (!confirmed) { | ||
| process.stderr.write("Cancelled\n"); | ||
| return; | ||
| } | ||
|
|
||
| // Perform the actual cleanup (orchestrate both domains) | ||
| const frameworkResult = await frameworkCache.cleanCache(ui5DataDir); | ||
| const buildResult = await CacheManager.cleanCache(ui5DataDir); | ||
|
|
||
| await displayCleanupResult({ | ||
| frameworkResult, | ||
| buildResult, | ||
| frameworkAbsPath, | ||
| buildAbsPath, | ||
| buildPreSize, | ||
| }); | ||
| } | ||
|
|
||
| export default cacheCommand; | ||
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.