{"meta":{"title":"execute upgrades","intro":"[Plumbing] Run upgrade scripts on an existing raw QL dataset.","product":"Security and code quality","breadcrumbs":[{"href":"/en/code-security","title":"Security and code quality"},{"href":"/en/code-security/reference","title":"Reference"},{"href":"/en/code-security/reference/code-scanning","title":"Code scanning"},{"href":"/en/code-security/reference/code-scanning/codeql","title":"CodeQL"},{"href":"/en/code-security/reference/code-scanning/codeql/codeql-cli-manual","title":"CodeQL CLI manual"},{"href":"/en/code-security/reference/code-scanning/codeql/codeql-cli-manual/execute-upgrades","title":"execute upgrades"}],"documentType":"article"},"body":"# execute upgrades\n\n\\[Plumbing] Run upgrade scripts on an existing raw QL dataset.\n\n> \\[!NOTE]\n> This content describes the most recent release of the CodeQL CLI. For more information about this release, see <https://github.com/github/codeql-cli-binaries/releases>.\n>\n> To see details of the options available for this command in an earlier release, run the command with the <span style=\"white-space: nowrap;\">`--help`</span> option in your terminal.\n\n## Synopsis\n\n```shell copy\ncodeql execute upgrades [--threads=<num>] <options>... -- <dataset> <script>...\n```\n\n## Description\n\n\\[Plumbing] Run upgrade scripts on an existing raw QL dataset.\n\nThis command runs a particular sequence of upgrade scripts against the\ndataset. It is the caller's responsibility that the \"old\" dbscheme of\neach upgrade script matches the \"new\" dbscheme of the previous script\n(or, for the first script, the current dbscheme of the dataset). If not,\nan error will be reported.\n\n## Options\n\n### Primary Options\n\n#### `<dataset>`\n\n\\[Mandatory] Path to the raw QL dataset to upgrade.\n\n#### `<script>...`\n\n\\[Mandatory] Paths to upgrade scripts to execute. (Each upgrade script\nis a directory containing several files that define an upgrade\noperation). They must be given in the order they are to be applied.\n\n#### `--search-path=<dir>[:<dir>...]`\n\nA list of directories under which QL packs may be found. Each directory\ncan either be a QL pack (or bundle of packs containing a\n`.codeqlmanifest.json` file at the root) or the immediate parent of one\nor more such directories.\n\nIf the path contains more than one directory, their order defines\nprecedence between them: when a pack name that must be resolved is\nmatched in more than one of the directory trees, the one given first\nwins.\n\nPointing this at a checkout of the open-source CodeQL repository ought\nto work when querying one of the languages that live there.\n\nIf you have checked out the CodeQL repository as a sibling of the\nunpacked CodeQL toolchain, you don't need to give this option; such\nsibling directories will always be searched for QL packs that cannot be\nfound otherwise. (If this default does not work, it is strongly\nrecommended to set up `--search-path` once and for all in a per-user\nconfiguration file).\n\n(Note: On Windows the path separator is `;`).\n\n#### `--additional-packs=<dir>[:<dir>...]`\n\nIf this list of directories is given, they will be searched for packs\nbefore the ones in `--search-path`. The order between these doesn't\nmatter; it is an error if a pack name is found in two different places\nthrough this list.\n\nThis is useful if you're temporarily developing a new version of a pack\nthat also appears in the default path. On the other hand, it is *not\nrecommended* to override this option in a config file; some internal\nactions will add this option on the fly, overriding any configured\nvalue.\n\n(Note: On Windows the path separator is `;`).\n\n### Options to control evaluation of upgrade queries\n\n#### `--[no-]tuple-counting`\n\n\\[Advanced] Display tuple counts for each evaluation step in the query\nevaluator logs. If the `--evaluator-log` option is provided, tuple\ncounts will be included in both the text-based and structured JSON logs\nproduced by the command. (This can be useful for performance\noptimization of complex QL code).\n\n#### `--timeout=<seconds>`\n\n\\[Advanced] Set the timeout length for query evaluation, in seconds.\n\nThe timeout feature is intended to catch cases where a complex query\nwould take \"forever\" to evaluate. It is not an effective way to limit\nthe total amount of time the query evaluation can take. The evaluation\nwill be allowed to continue as long as each separately timed part of the\ncomputation completes within the timeout. Currently these separately\ntimed parts are \"RA layers\" of the optimized query, but that might\nchange in the future.\n\nIf no timeout is specified, or is given as 0, no timeout will be set\n(except for [codeql test run](/en/code-security/reference/code-scanning/codeql/codeql-cli-manual/test-run), where the default timeout is 5 minutes).\n\n#### `-j, --threads=<num>`\n\nUse this many threads to evaluate queries.\n\nDefaults to 1. You can pass 0 to use one thread per core on the machine,\nor -*N* to leave *N* cores unused (except still use at least one\nthread).\n\n#### `--[no-]save-cache`\n\n\\[Deprecated] \\[Advanced] This flag does nothing.\n\n#### `--[no-]expect-discarded-cache`\n\n\\[Advanced] Make decisions about which predicates to evaluate, and what\nto write to the disk cache, based on the assumption that the cache will\nbe discarded after the queries have been executed.\n\n#### `--[no-]keep-full-cache`\n\n\\[Advanced] Don't clean up the disk cache after evaluation completes.\nThis may save time if you're going to do [codeql dataset cleanup](/en/code-security/reference/code-scanning/codeql/codeql-cli-manual/dataset-cleanup) or [codeql database cleanup](/en/code-security/reference/code-scanning/codeql/codeql-cli-manual/database-cleanup) afterwards anyway.\n\n#### `--max-disk-cache=<MB>`\n\nSet the maximum amount of space that the disk cache for intermediate\nquery results can use.\n\nIf this size is not configured explicitly, the evaluator will try to use\na \"reasonable\" amount of cache space, based on the size of the dataset\nand the complexity of the queries. Explicitly setting a higher limit\nthan this default usage will enable additional caching which can speed\nup later queries.\n\n#### `--min-disk-free=<MB>`\n\n\\[Advanced] Set target amount of free space on file system.\n\nIf `--max-disk-cache` is not given, the evaluator will try hard to\ncurtail disk cache usage if the free space on the file system drops\nbelow this value.\n\n#### `--min-disk-free-pct=<pct>`\n\n\\[Advanced] Set target fraction of free space on file system.\n\nIf `--max-disk-cache` is not given, the evaluator will try hard to\ncurtail disk cache usage if the free space on the file system drops\nbelow this percentage.\n\n#### `--external=<pred>=<file.csv>`\n\nA CSV file that contains rows for external predicate *\\<pred>*.\nMultiple `--external` options can be supplied.\n\n#### `--xterm-progress=<mode>`\n\n\\[Advanced] Controls whether to show progress tracking during QL\nevaluation using xterm control sequences. Possible values are:\n\n`no`: Never produce fancy progress; assume a dumb terminal.\n\n`auto` *(default)*: Autodetect whether the command is running in an\nappropriate terminal.\n\n`yes`: Assume the terminal can understand xterm control sequences. The\nfeature still depends on being able to autodetect the *size* of the\nterminal (which is not implemented on Windows, sorry), and will also be\ndisabled if `-q` is given.\n\n`25x80` (or similar): Like `yes`, and also explicitly give the size of\nthe terminal. (Unlike `yes`, this should work on Windows.)\n\n`25x80:/dev/pts/17` (or similar): show fancy progress on a *different*\nterminal than stderr. Mostly useful for internal testing.\n\n### Options for controlling outputting of structured evaluator logs\n\n#### `--evaluator-log=<file>`\n\n\\[Advanced] Output structured logs about evaluator performance to the\ngiven file. The format of this log file is subject to change with no\nnotice, but will be a stream of JSON objects separated by either two\nnewline characters (by default) or one if the `--evaluator-log-minify`\noption is passed. Please use `codeql generate log-summary <file>` to\nproduce a more stable summary of this file, and avoid parsing the file\ndirectly. The file will be overwritten if it already exists.\n\n#### `--evaluator-log-minify`\n\n\\[Advanced] If the `--evaluator-log` option is passed, also passing\nthis option will minimize the size of the JSON log produced, at the\nexpense of making it much less human readable.\n\n### Options for configuring the CodeQL package manager\n\n#### `--registries-auth-stdin`\n\nAuthenticate to GitHub Enterprise Server Container registries by passing\na comma-separated list of \\<registry\\_url>=\\<token> pairs.\n\nFor example, you can pass\n`https://containers.GHEHOSTNAME1/v2/=TOKEN1,https://containers.GHEHOSTNAME2/v2/=TOKEN2`\nto authenticate to two GitHub Enterprise Server instances.\n\nThis overrides the CODEQL\\_REGISTRIES\\_AUTH and GITHUB\\_TOKEN environment\nvariables. If you only need to authenticate to the github.com Container\nregistry, you can instead authenticate using the simpler\n`--github-auth-stdin` option.\n\n#### `--github-auth-stdin`\n\nAuthenticate to the github.com Container registry by passing a\ngithub.com GitHub Apps token or personal access token via standard\ninput.\n\nTo authenticate to GitHub Enterprise Server Container registries, pass\n`--registries-auth-stdin` or use the CODEQL\\_REGISTRIES\\_AUTH environment\nvariable.\n\nThis overrides the GITHUB\\_TOKEN environment variable.\n\n### Common options\n\n#### `-h, --help`\n\nShow this help text.\n\n#### `-J=<opt>`\n\n\\[Advanced] Give option to the JVM running the command.\n\n(Beware that options containing spaces will not be handled correctly.)\n\n#### `-v, --verbose`\n\nIncrementally increase the number of progress messages printed.\n\n#### `-q, --quiet`\n\nIncrementally decrease the number of progress messages printed.\n\n#### `--verbosity=<level>`\n\n\\[Advanced] Explicitly set the verbosity level to one of errors,\nwarnings, progress, progress+, progress++, progress+++. Overrides `-v`\nand `-q`.\n\n#### `--logdir=<dir>`\n\n\\[Advanced] Write detailed logs to one or more files in the given\ndirectory, with generated names that include timestamps and the name of\nthe running subcommand.\n\n(To write a log file with a name you have full control over, instead\ngive `--log-to-stderr` and redirect stderr as desired.)\n\n#### `--common-caches=<dir>`\n\n\\[Advanced] Controls the location of cached data on disk that will\npersist between several runs of the CLI, such as downloaded QL packs and\ncompiled query plans. If not set explicitly, this defaults to a\ndirectory named `.codeql` in the user's home directory; it will be\ncreated if it doesn't already exist.\n\nAvailable since `v2.15.2`."}