diff --git a/doc/update/versions/gitlab_16_changes.md b/doc/update/versions/gitlab_16_changes.md
index aa82b715171..0643af70c47 100644
--- a/doc/update/versions/gitlab_16_changes.md
+++ b/doc/update/versions/gitlab_16_changes.md
@@ -25,10 +25,7 @@ For more information about upgrading GitLab Helm Chart, see [the release notes f
Perform the [workaround](#undefined-column-error-upgrading-to-162-or-later) before upgrading to 16.x.
- Starting with 16.0, GitLab self-managed installations now have two database connections by default, instead of one. This change doubles the number of PostgreSQL connections. It makes self-managed versions of GitLab behave similarly to GitLab.com, and is a step toward enabling a separate database for CI features for self-managed versions of GitLab. Before upgrading to 16.0, determine if you need to [increase max connections for PostgreSQL](https://docs.gitlab.com/omnibus/settings/database.html#configuring-multiple-database-connections).
- This change applies to installation methods with Linux packages (Omnibus), GitLab Helm chart, GitLab Operator, GitLab Docker images, and self-compiled installations.
- - The second database connection can be disabled:
- - [Linux package and Docker installations](https://docs.gitlab.com/omnibus/settings/database.html#configuring-multiple-database-connections).
- - [Helm chart and GitLab Operator installations](https://docs.gitlab.com/charts/charts/globals.html#configure-multiple-database-connections).
- - [Self-compiled installations](../../install/installation.md#configure-gitlab-db-settings).
+ - [The second database connection can be disabled](#disable-the-second-database-connection).
- Most installations can skip 16.0, 16.1, and 16.2, as the first required stop on the upgrade path is 16.3.
In all cases, you should review the notes for those intermediate versions.
@@ -1575,6 +1572,53 @@ praefect['configuration'] = {
}
```
+### Disable the second database connection
+
+In GitLab 16.0, GitLab defaults to using two database connections that point to the same PostgreSQL database.
+
+PostgreSQL might need to be configured with a larger value for `max_connections`.
+[There is a Rake task for checking if this is necessary](https://docs.gitlab.com/omnibus/settings/database.html#configuring-multiple-database-connections).
+
+If you have PgBouncer deployed:
+
+- The frontend pools (including file handle limits and `max_client_conn`) on your PgBouncer servers [might need to be larger](../../administration/postgresql/pgbouncer.md#fine-tuning).
+- PgBouncer is single threaded. The extra connections might fully saturate a single PgBouncer daemon.
+ [We recommend running three load-balanced PgBouncer servers](../../administration/reference_architectures/5k_users.md#configure-pgbouncer) for all
+ scaled GitLab deployments, in part to address this issue.
+
+Follow the instructions for your installation type to switch back to a single database connection:
+
+::Tabs
+
+:::TabTitle Linux package and Docker
+
+1. Add this setting to `/etc/gitlab/gitlab.rb`:
+
+ ```ruby
+ gitlab_rails['databases']['ci']['enable'] = false
+ ```
+
+1. Run `gitlab-ctl reconfigure`.
+
+In a multi-node environment, this setting should be updated on all Rails and Sidekiq nodes.
+
+:::TabTitle Helm chart (Kubernetes)
+
+Set the `ci.enabled` key to `false`:
+
+```yaml
+global:
+ psql:
+ ci:
+ enabled: false
+```
+
+:::TabTitle Self-compiled (source)
+
+Remove the `ci:` section from `config/database.yml`.
+
+::EndTabs
+
## Long-running user type data change
GitLab 16.0 is a required stop for large GitLab instances with a lot of records in the `users` table.
diff --git a/doc/user/project/wiki/index.md b/doc/user/project/wiki/index.md
index e9f09ecba90..8100d4c9cec 100644
--- a/doc/user/project/wiki/index.md
+++ b/doc/user/project/wiki/index.md
@@ -142,7 +142,7 @@ You need at least the Developer role to edit a wiki page:
1. Select **Plan > Wiki**.
1. Go to the page you want to edit, and either:
- Use the
e wiki [keyboard shortcut](../../shortcuts.md#wiki-pages).
- - Select the edit icon (**{pencil}**).
+ - Select **Edit**.
1. Edit the content.
1. Select **Save changes**.
@@ -162,11 +162,13 @@ Prerequisites:
1. On the left sidebar, select **Search or go to** and find your project or group.
1. Select **Plan > Wiki**.
1. Go to the page you want to delete.
-1. Select the edit icon (**{pencil}**).
+1. Select **Edit**.
1. Select **Delete page**.
1. Confirm the deletion.
-## Move a wiki page
+## Move or rename a wiki page
+
+> - [Support for redirecting pages on move or rename](https://gitlab.com/gitlab-org/gitlab/-/issues/257892) added in GitLab 17.1 [with a flag](../../../administration/feature_flags.md) named `wiki_redirection`. Enabled by default.
Prerequisites:
@@ -174,13 +176,18 @@ Prerequisites:
1. On the left sidebar, select **Search or go to** and find your project or group.
1. Select **Plan > Wiki**.
-1. Go to the page you want to move.
-1. Select the edit icon (**{pencil}**).
-1. Add the new path to the **Title** field. For example, if you have a wiki page
- called `about` under `company` and you want to move it to the wiki's root,
- change the **Title** from `about` to `/about`.
+1. Go to the page you want to move or rename.
+1. Select **Edit**.
+1. To move the page, add the new path to the **Title** field. For example,
+ if you have a wiki page called `About` under `Company` and you want to
+ move it to the wiki's root, change the **Title** from `About` to `/About`.
+1. To rename the page, change the **Title**.
1. Select **Save changes**.
+In GitLab 17.1 and later, when you move or rename a page, a redirect is
+automatically set up from the old page to the new page. A list of redirects
+is stored in the `.gitlab/redirects.yml` file in the Wiki repository.
+
## Export a wiki page
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/414691) in GitLab 16.3 [with a flag](../../../administration/feature_flags.md) named `print_wiki`. Disabled by default.
diff --git a/gems/gem-pg.gitlab-ci.yml b/gems/gem-pg.gitlab-ci.yml
index 5ae5a8b768d..f02f4439058 100644
--- a/gems/gem-pg.gitlab-ci.yml
+++ b/gems/gem-pg.gitlab-ci.yml
@@ -27,11 +27,12 @@ variables:
GIT_STRATEGY: "clone"
GIT_SUBMODULE_STRATEGY: "none"
GET_SOURCES_ATTEMPTS: "3"
- RUBY_VERSION_DEFAULT: "3.1.5"
- RUBY_VERSION_NEXT: "3.2.4"
# Default Ruby version for jobs that don't use .ruby_matrix
RUBY_VERSION: "${RUBY_VERSION_DEFAULT}"
+include:
+ - local: .gitlab/ci/version.yml
+
default:
tags:
- gitlab-org
diff --git a/gems/gem.gitlab-ci.yml b/gems/gem.gitlab-ci.yml
index 284d87e4246..d2791c62c2c 100644
--- a/gems/gem.gitlab-ci.yml
+++ b/gems/gem.gitlab-ci.yml
@@ -27,11 +27,12 @@ variables:
GIT_STRATEGY: "clone"
GIT_SUBMODULE_STRATEGY: "none"
GET_SOURCES_ATTEMPTS: "3"
- RUBY_VERSION_DEFAULT: "3.1.5"
- RUBY_VERSION_NEXT: "3.2.4"
# Default Ruby version for jobs that don't use .ruby_matrix
RUBY_VERSION: "${RUBY_VERSION_DEFAULT}"
+include:
+ - local: .gitlab/ci/version.yml
+
default:
tags:
- gitlab-org
diff --git a/lib/gitlab/background_migration/backfill_dast_pre_scan_verifications_project_id.rb b/lib/gitlab/background_migration/backfill_dast_pre_scan_verifications_project_id.rb
new file mode 100644
index 00000000000..2d0148a9d3c
--- /dev/null
+++ b/lib/gitlab/background_migration/backfill_dast_pre_scan_verifications_project_id.rb
@@ -0,0 +1,10 @@
+# frozen_string_literal: true
+
+module Gitlab
+ module BackgroundMigration
+ class BackfillDastPreScanVerificationsProjectId < BackfillDesiredShardingKeyJob
+ operation_name :backfill_dast_pre_scan_verifications_project_id
+ feature_category :dynamic_application_security_testing
+ end
+ end
+end
diff --git a/lib/gitlab/github_import/user_finder.rb b/lib/gitlab/github_import/user_finder.rb
index e0bda5fca75..204851fdeb0 100644
--- a/lib/gitlab/github_import/user_finder.rb
+++ b/lib/gitlab/github_import/user_finder.rb
@@ -130,7 +130,7 @@ module Gitlab
email = read_email_from_cache(username)
if email.blank? && !email_fetched_for_project?(username)
- feature_flag_in_lock(lease_key(username), sleep_sec: 0.2.seconds, retries: 30) do |retried|
+ in_lock(lease_key(username), sleep_sec: 0.2.seconds, retries: 30) do |retried|
# when retried, check the cache again as the other process that had the lease may have fetched the email
if retried
email = read_email_from_cache(username)
@@ -306,14 +306,6 @@ module Gitlab
message: message
)
end
-
- def feature_flag_in_lock(lease_key, sleep_sec:, retries:)
- return yield(false) if Feature.disabled?(:github_import_lock_user_finder, project.creator)
-
- in_lock(lease_key, sleep_sec: sleep_sec, retries: retries) do |retried|
- yield(retried)
- end
- end
end
end
end
diff --git a/scripts/frontend/compare_css_util_classes.mjs b/scripts/frontend/compare_css_util_classes.mjs
index 9dfb568f0d7..d409572c796 100755
--- a/scripts/frontend/compare_css_util_classes.mjs
+++ b/scripts/frontend/compare_css_util_classes.mjs
@@ -11,7 +11,7 @@ import {
darkModeTokenToHex,
mismatchAllowList,
} from './lib/tailwind_migration.mjs';
-import { convertUtilsToCSSInJS } from './tailwind_all_the_way.mjs';
+import { convertUtilsToCSSInJS, toMinimalUtilities } from './tailwind_all_the_way.mjs';
function darkModeResolver(str) {
return str.replace(
@@ -26,10 +26,10 @@ function darkModeResolver(str) {
);
}
-function compareApplicationUtilsToTailwind(appUtils, tailWind, colorResolver) {
+function compareApplicationUtilsToTailwind(appUtils, tailwindCSS, colorResolver) {
let fail = 0;
- const tailwind = extractRules(tailWind);
+ const tailwind = extractRules(tailwindCSS);
Object.keys(appUtils).forEach((selector) => {
if (mismatchAllowList.includes(selector)) {
@@ -50,9 +50,33 @@ function compareApplicationUtilsToTailwind(appUtils, tailWind, colorResolver) {
});
if (fail) {
- console.log(`${fail} selectors failed`);
- process.exitCode = 1;
+ console.log(`\t${fail} selectors failed`);
+ } else {
+ console.log('\tAll good');
}
+
+ return fail;
+}
+
+function ensureNoLegacyUtilIsUsedWithATailwindModifier(minimalUtils) {
+ let fail = 0;
+ for (const [key, value] of Object.entries(minimalUtils)) {
+ if (key.startsWith('.\\!')) {
+ console.warn('Using legacy util with important modifier. This is not supported.');
+ console.warn(`Please migrate ${key} to a proper tailwind util.`);
+ fail += 1;
+ }
+ if (key.endsWith('\\')) {
+ console.warn(`Using legacy util with ${key} modifier. This is not supported.`);
+ console.warn(`Please migrate the following classes to a proper tailwind util:`);
+ console.warn(JSON.stringify(value, null, 2).replace(/^/gm, ' '.repeat(4)));
+ fail += 1;
+ }
+ }
+ if (fail) {
+ console.log(`\t${fail} legacy utils with modifiers found`);
+ }
+ return fail;
}
console.log('# Converting legacy styles to CSS-in-JS definitions');
@@ -64,6 +88,8 @@ if (stats.hardcodedColors || stats.potentialMismatches) {
process.exitCode = 1;
}
+let failures = 0;
+
console.log('# Comparing tailwind to legacy utils');
const applicationUtilsLight = extractRules(
@@ -77,6 +103,20 @@ const applicationUtilsDark = extractRules(
const tailwind = loadCSSFromFile('app/assets/builds/tailwind.css');
console.log('## Comparing tailwind light mode');
-compareApplicationUtilsToTailwind(applicationUtilsLight, tailwind);
+failures += compareApplicationUtilsToTailwind(applicationUtilsLight, tailwind);
console.log('## Comparing tailwind dark mode');
-compareApplicationUtilsToTailwind(applicationUtilsDark, tailwind, darkModeResolver);
+failures += compareApplicationUtilsToTailwind(applicationUtilsDark, tailwind, darkModeResolver);
+
+console.log('# Checking whether legacy GitLab utility classes are used with tailwind modifiers');
+
+console.log('## Reducing utility definitions to minimally used');
+const { rules } = await toMinimalUtilities();
+
+console.log('## Running checks');
+failures += ensureNoLegacyUtilIsUsedWithATailwindModifier(rules);
+
+if (failures) {
+ process.exitCode = 1;
+} else {
+ console.log('# All good – Happiness. May the tailwind boost your journey');
+}
diff --git a/scripts/frontend/tailwind_all_the_way.mjs b/scripts/frontend/tailwind_all_the_way.mjs
index 3b692bb1759..2d6df177b5f 100755
--- a/scripts/frontend/tailwind_all_the_way.mjs
+++ b/scripts/frontend/tailwind_all_the_way.mjs
@@ -136,7 +136,7 @@ const hardCodedColors = ${JSON.stringify(hardcodedColors, null, 2)};
/**
* Writes only the style definitions we actually need.
*/
-async function toMinimalUtilities() {
+export async function toMinimalUtilities() {
// We re-import the config with a `?minimal` query in order to cache-bust
// the previously loaded config, which doesn't have the latest css_in_js
const { default: tailwindConfig } = await import('../../config/tailwind.config.js?minimal');
@@ -166,7 +166,7 @@ async function toMinimalUtilities() {
*/
module.exports = ${JSON.stringify(rules)}`);
- return { minimalUtils };
+ return { minimalUtils, rules };
}
/**
diff --git a/spec/lib/gitlab/background_migration/backfill_dast_pre_scan_verifications_project_id_spec.rb b/spec/lib/gitlab/background_migration/backfill_dast_pre_scan_verifications_project_id_spec.rb
new file mode 100644
index 00000000000..ab50ca0f164
--- /dev/null
+++ b/spec/lib/gitlab/background_migration/backfill_dast_pre_scan_verifications_project_id_spec.rb
@@ -0,0 +1,15 @@
+# frozen_string_literal: true
+
+require 'spec_helper'
+
+RSpec.describe Gitlab::BackgroundMigration::BackfillDastPreScanVerificationsProjectId,
+ feature_category: :dynamic_application_security_testing,
+ schema: 20240603121638 do
+ include_examples 'desired sharding key backfill job' do
+ let(:batch_table) { :dast_pre_scan_verifications }
+ let(:backfill_column) { :project_id }
+ let(:backfill_via_table) { :dast_profiles }
+ let(:backfill_via_column) { :project_id }
+ let(:backfill_via_foreign_key) { :dast_profile_id }
+ end
+end
diff --git a/spec/lib/gitlab/github_import/user_finder_spec.rb b/spec/lib/gitlab/github_import/user_finder_spec.rb
index a5e8d428128..1cd1e76c5c2 100644
--- a/spec/lib/gitlab/github_import/user_finder_spec.rb
+++ b/spec/lib/gitlab/github_import/user_finder_spec.rb
@@ -313,19 +313,6 @@ RSpec.describe Gitlab::GithubImport::UserFinder, :clean_gitlab_redis_shared_stat
email_for_github_username
end
- context 'when github_import_lock_user_finder feature flag is disabled' do
- before do
- stub_feature_flags(github_import_lock_user_finder: false)
- end
-
- it 'does not lock the finder' do
- expect(finder).not_to receive(:in_lock)
- expect(client).to receive(:user)
-
- email_for_github_username
- end
- end
-
context 'if the response contains an email' do
before do
allow(client).to receive(:user).and_return({ email: email })
diff --git a/spec/migrations/20240603121642_queue_backfill_dast_pre_scan_verifications_project_id_spec.rb b/spec/migrations/20240603121642_queue_backfill_dast_pre_scan_verifications_project_id_spec.rb
new file mode 100644
index 00000000000..6b33cf6ed1d
--- /dev/null
+++ b/spec/migrations/20240603121642_queue_backfill_dast_pre_scan_verifications_project_id_spec.rb
@@ -0,0 +1,33 @@
+# frozen_string_literal: true
+
+require 'spec_helper'
+require_migration!
+
+RSpec.describe QueueBackfillDastPreScanVerificationsProjectId, feature_category: :dynamic_application_security_testing do
+ let!(:batched_migration) { described_class::MIGRATION }
+
+ it 'schedules a new batched migration' do
+ reversible_migration do |migration|
+ migration.before -> {
+ expect(batched_migration).not_to have_scheduled_batched_migration
+ }
+
+ migration.after -> {
+ expect(batched_migration).to have_scheduled_batched_migration(
+ table_name: :dast_pre_scan_verifications,
+ column_name: :id,
+ interval: described_class::DELAY_INTERVAL,
+ batch_size: described_class::BATCH_SIZE,
+ sub_batch_size: described_class::SUB_BATCH_SIZE,
+ gitlab_schema: :gitlab_main_cell,
+ job_arguments: [
+ :project_id,
+ :dast_profiles,
+ :project_id,
+ :dast_profile_id
+ ]
+ )
+ }
+ end
+ end
+end
diff --git a/vendor/gems/attr_encrypted/.gitlab-ci.yml b/vendor/gems/attr_encrypted/.gitlab-ci.yml
index 2583f281fb8..b30e9ad1c62 100644
--- a/vendor/gems/attr_encrypted/.gitlab-ci.yml
+++ b/vendor/gems/attr_encrypted/.gitlab-ci.yml
@@ -9,4 +9,4 @@ rspec:
- bundle exec rake test
parallel:
matrix:
- - RUBY_VERSION: ["3.1.5", "3.2.4"]
+ - RUBY_VERSION: ["${RUBY_VERSION_DEFAULT}", "${RUBY_VERSION_NEXT}"]
diff --git a/vendor/gems/diff_match_patch/.gitlab-ci.yml b/vendor/gems/diff_match_patch/.gitlab-ci.yml
index f00418ca118..6765a52f954 100644
--- a/vendor/gems/diff_match_patch/.gitlab-ci.yml
+++ b/vendor/gems/diff_match_patch/.gitlab-ci.yml
@@ -9,4 +9,4 @@ rspec:
- rake test
parallel:
matrix:
- - RUBY_VERSION: ["3.1.5", "3.2.4"]
+ - RUBY_VERSION: ["${RUBY_VERSION_DEFAULT}", "${RUBY_VERSION_NEXT}"]
diff --git a/vendor/gems/sidekiq-7.1.6/.gitlab-ci.yml b/vendor/gems/sidekiq-7.1.6/.gitlab-ci.yml
index 7ac772fde82..1bb1431d600 100644
--- a/vendor/gems/sidekiq-7.1.6/.gitlab-ci.yml
+++ b/vendor/gems/sidekiq-7.1.6/.gitlab-ci.yml
@@ -11,7 +11,7 @@ rspec:
- .with_redis
parallel:
matrix:
- - RUBY_VERSION: ["3.1.5", "3.2.4"]
+ - RUBY_VERSION: ["${RUBY_VERSION_DEFAULT}", "${RUBY_VERSION_NEXT}"]
REDIS_VERSION: ["6.0", "6.2", "7.0"]
.with_redis:
diff --git a/vendor/gems/sidekiq-reliable-fetch/.gitlab-ci.yml b/vendor/gems/sidekiq-reliable-fetch/.gitlab-ci.yml
index 30786ca8eef..c0dc78bf550 100644
--- a/vendor/gems/sidekiq-reliable-fetch/.gitlab-ci.yml
+++ b/vendor/gems/sidekiq-reliable-fetch/.gitlab-ci.yml
@@ -9,7 +9,7 @@ rspec:
- .with_redis
parallel:
matrix:
- - RUBY_VERSION: ["3.1.5"] # 3.2 isn't supported yet
+ - RUBY_VERSION: ["${RUBY_VERSION_DEFAULT}"] # 3.2 isn't supported yet
.with_redis:
services:
diff --git a/workhorse/_support/lint_last_known_acceptable_go1.21.txt b/workhorse/_support/lint_last_known_acceptable_go1.21.txt
index 8397dde183c..9961b4e285e 100644
--- a/workhorse/_support/lint_last_known_acceptable_go1.21.txt
+++ b/workhorse/_support/lint_last_known_acceptable_go1.21.txt
@@ -9,8 +9,8 @@ cmd/gitlab-workhorse/authorization_test.go:114:3: go-require: do not use require
cmd/gitlab-workhorse/authorization_test.go:115:3: go-require: do not use require in http handlers (testifylint)
cmd/gitlab-workhorse/authorization_test.go:116:3: go-require: do not use require in http handlers (testifylint)
cmd/gitlab-workhorse/authorization_test.go:120:3: go-require: do not use require in http handlers (testifylint)
-cmd/gitlab-workhorse/config_test.go:189: cmd/gitlab-workhorse/config_test.go:189: Line contains TODO/BUG/FIXME/NOTE/OPTIMIZE/HACK: "TODO this is meant to be 50*time.Second ..." (godox)
-cmd/gitlab-workhorse/config_test.go:331:16: unused-parameter: parameter 't' seems to be unused, consider removing or renaming it as _ (revive)
+cmd/gitlab-workhorse/config_test.go:191: cmd/gitlab-workhorse/config_test.go:191: Line contains TODO/BUG/FIXME/NOTE/OPTIMIZE/HACK: "TODO this is meant to be 50*time.Second ..." (godox)
+cmd/gitlab-workhorse/config_test.go:333:16: unused-parameter: parameter 't' seems to be unused, consider removing or renaming it as _ (revive)
cmd/gitlab-workhorse/jobs_test.go:37:29: response body must be closed (bodyclose)
cmd/gitlab-workhorse/jobs_test.go:42:29: response body must be closed (bodyclose)
cmd/gitlab-workhorse/listener.go:34:16: G402: TLS MinVersion too low. (gosec)
@@ -164,21 +164,21 @@ internal/config/config.go:84:6: exported: exported type GoCloudConfig should hav
internal/config/config.go:88:6: exported: exported type AzureCredentials should have comment or be unexported (revive)
internal/config/config.go:93:6: exported: exported type GoogleCredentials should have comment or be unexported (revive)
internal/config/config.go:99:6: exported: exported type RedisConfig should have comment or be unexported (revive)
-internal/config/config.go:110:6: exported: exported type ImageResizerConfig should have comment or be unexported (revive)
-internal/config/config.go:116:6: exported: exported type MetadataConfig should have comment or be unexported (revive)
-internal/config/config.go:120:6: exported: exported type TLSConfig should have comment or be unexported (revive)
-internal/config/config.go:127:6: exported: exported type ListenerConfig should have comment or be unexported (revive)
-internal/config/config.go:130:2: var-naming: struct field Tls should be TLS (revive)
-internal/config/config.go:133:6: exported: exported type Config should have comment or be unexported (revive)
-internal/config/config.go:161:5: exported: exported var DefaultImageResizerConfig should have comment or be unexported (revive)
-internal/config/config.go:166:5: exported: exported var DefaultMetadataConfig should have comment or be unexported (revive)
-internal/config/config.go:170:1: exported: exported function NewDefaultConfig should have comment or be unexported (revive)
-internal/config/config.go:177:1: exported: exported function LoadConfigFromFile should have comment or be unexported (revive)
-internal/config/config.go:191:1: exported: exported function LoadConfig should have comment or be unexported (revive)
-internal/config/config.go:200:18: G204: Subprocess launched with variable (gosec)
-internal/config/config.go:218:1: exported: exported method Config.RegisterGoCloudURLOpeners should have comment or be unexported (revive)
-internal/config/config.go:241:70: (*AzureCredentials).getURLOpener - result 1 (error) is always nil (unparam)
-internal/config/config.go:284:8: G101: Potential hardcoded credentials (gosec)
+internal/config/config.go:111:6: exported: exported type ImageResizerConfig should have comment or be unexported (revive)
+internal/config/config.go:117:6: exported: exported type MetadataConfig should have comment or be unexported (revive)
+internal/config/config.go:121:6: exported: exported type TLSConfig should have comment or be unexported (revive)
+internal/config/config.go:128:6: exported: exported type ListenerConfig should have comment or be unexported (revive)
+internal/config/config.go:131:2: var-naming: struct field Tls should be TLS (revive)
+internal/config/config.go:134:6: exported: exported type Config should have comment or be unexported (revive)
+internal/config/config.go:162:5: exported: exported var DefaultImageResizerConfig should have comment or be unexported (revive)
+internal/config/config.go:167:5: exported: exported var DefaultMetadataConfig should have comment or be unexported (revive)
+internal/config/config.go:171:1: exported: exported function NewDefaultConfig should have comment or be unexported (revive)
+internal/config/config.go:178:1: exported: exported function LoadConfigFromFile should have comment or be unexported (revive)
+internal/config/config.go:192:1: exported: exported function LoadConfig should have comment or be unexported (revive)
+internal/config/config.go:201:18: G204: Subprocess launched with variable (gosec)
+internal/config/config.go:219:1: exported: exported method Config.RegisterGoCloudURLOpeners should have comment or be unexported (revive)
+internal/config/config.go:242:70: (*AzureCredentials).getURLOpener - result 1 (error) is always nil (unparam)
+internal/config/config.go:285:8: G101: Potential hardcoded credentials (gosec)
internal/dependencyproxy/dependencyproxy.go:77: Function 'Inject' is too long (70 > 60) (funlen)
internal/dependencyproxy/dependencyproxy.go:115:32: `cancelled` is a misspelling of `canceled` (misspell)
internal/dependencyproxy/dependencyproxy_test.go:101:91: unused-parameter: parameter 'r' seems to be unused, consider removing or renaming it as _ (revive)
@@ -322,7 +322,7 @@ internal/redis/redis.go:1:1: ST1000: at least one file in a package should have
internal/redis/redis.go:16:2: blank-imports: a blank import should be only in a main or test package, or have a comment justifying it (revive)
internal/redis/redis.go:21:24: var-declaration: should omit type error from declaration of var errSentinelMasterAddr; it will be inferred from the right-hand side (revive)
internal/redis/redis.go:23:2: exported: exported var TotalConnections should have comment or be unexported (revive)
-internal/redis/redis.go:95:10: elseif: can replace 'else {if cond {}}' with 'else if cond {}' (gocritic)
+internal/redis/redis.go:102:10: elseif: can replace 'else {if cond {}}' with 'else if cond {}' (gocritic)
internal/redis/redis_test.go:18:2: error-nil: use require.NoError (testifylint)
internal/redis/redis_test.go:23:3: error-nil: use require.NoError (testifylint)
internal/redis/redis_test.go:66:15: `initialise` is a misspelling of `initialize` (misspell)
diff --git a/workhorse/_support/lint_last_known_acceptable_go1.22.txt b/workhorse/_support/lint_last_known_acceptable_go1.22.txt
index 24386e6f25c..e92aabd02d5 100644
--- a/workhorse/_support/lint_last_known_acceptable_go1.22.txt
+++ b/workhorse/_support/lint_last_known_acceptable_go1.22.txt
@@ -9,8 +9,8 @@ cmd/gitlab-workhorse/authorization_test.go:114:3: go-require: do not use require
cmd/gitlab-workhorse/authorization_test.go:115:3: go-require: do not use require in http handlers (testifylint)
cmd/gitlab-workhorse/authorization_test.go:116:3: go-require: do not use require in http handlers (testifylint)
cmd/gitlab-workhorse/authorization_test.go:120:3: go-require: do not use require in http handlers (testifylint)
-cmd/gitlab-workhorse/config_test.go:189: cmd/gitlab-workhorse/config_test.go:189: Line contains TODO/BUG/FIXME/NOTE/OPTIMIZE/HACK: "TODO this is meant to be 50*time.Second ..." (godox)
-cmd/gitlab-workhorse/config_test.go:331:16: unused-parameter: parameter 't' seems to be unused, consider removing or renaming it as _ (revive)
+cmd/gitlab-workhorse/config_test.go:191: cmd/gitlab-workhorse/config_test.go:191: Line contains TODO/BUG/FIXME/NOTE/OPTIMIZE/HACK: "TODO this is meant to be 50*time.Second ..." (godox)
+cmd/gitlab-workhorse/config_test.go:333:16: unused-parameter: parameter 't' seems to be unused, consider removing or renaming it as _ (revive)
cmd/gitlab-workhorse/jobs_test.go:37:29: response body must be closed (bodyclose)
cmd/gitlab-workhorse/jobs_test.go:42:29: response body must be closed (bodyclose)
cmd/gitlab-workhorse/listener.go:34:16: G402: TLS MinVersion too low. (gosec)
@@ -164,21 +164,21 @@ internal/config/config.go:84:6: exported: exported type GoCloudConfig should hav
internal/config/config.go:88:6: exported: exported type AzureCredentials should have comment or be unexported (revive)
internal/config/config.go:93:6: exported: exported type GoogleCredentials should have comment or be unexported (revive)
internal/config/config.go:99:6: exported: exported type RedisConfig should have comment or be unexported (revive)
-internal/config/config.go:110:6: exported: exported type ImageResizerConfig should have comment or be unexported (revive)
-internal/config/config.go:116:6: exported: exported type MetadataConfig should have comment or be unexported (revive)
-internal/config/config.go:120:6: exported: exported type TLSConfig should have comment or be unexported (revive)
-internal/config/config.go:127:6: exported: exported type ListenerConfig should have comment or be unexported (revive)
-internal/config/config.go:130:2: var-naming: struct field Tls should be TLS (revive)
-internal/config/config.go:133:6: exported: exported type Config should have comment or be unexported (revive)
-internal/config/config.go:161:5: exported: exported var DefaultImageResizerConfig should have comment or be unexported (revive)
-internal/config/config.go:166:5: exported: exported var DefaultMetadataConfig should have comment or be unexported (revive)
-internal/config/config.go:170:1: exported: exported function NewDefaultConfig should have comment or be unexported (revive)
-internal/config/config.go:177:1: exported: exported function LoadConfigFromFile should have comment or be unexported (revive)
-internal/config/config.go:191:1: exported: exported function LoadConfig should have comment or be unexported (revive)
-internal/config/config.go:200:18: G204: Subprocess launched with variable (gosec)
-internal/config/config.go:218:1: exported: exported method Config.RegisterGoCloudURLOpeners should have comment or be unexported (revive)
-internal/config/config.go:241:70: (*AzureCredentials).getURLOpener - result 1 (error) is always nil (unparam)
-internal/config/config.go:284:8: G101: Potential hardcoded credentials (gosec)
+internal/config/config.go:111:6: exported: exported type ImageResizerConfig should have comment or be unexported (revive)
+internal/config/config.go:117:6: exported: exported type MetadataConfig should have comment or be unexported (revive)
+internal/config/config.go:121:6: exported: exported type TLSConfig should have comment or be unexported (revive)
+internal/config/config.go:128:6: exported: exported type ListenerConfig should have comment or be unexported (revive)
+internal/config/config.go:131:2: var-naming: struct field Tls should be TLS (revive)
+internal/config/config.go:134:6: exported: exported type Config should have comment or be unexported (revive)
+internal/config/config.go:162:5: exported: exported var DefaultImageResizerConfig should have comment or be unexported (revive)
+internal/config/config.go:167:5: exported: exported var DefaultMetadataConfig should have comment or be unexported (revive)
+internal/config/config.go:171:1: exported: exported function NewDefaultConfig should have comment or be unexported (revive)
+internal/config/config.go:178:1: exported: exported function LoadConfigFromFile should have comment or be unexported (revive)
+internal/config/config.go:192:1: exported: exported function LoadConfig should have comment or be unexported (revive)
+internal/config/config.go:201:18: G204: Subprocess launched with variable (gosec)
+internal/config/config.go:219:1: exported: exported method Config.RegisterGoCloudURLOpeners should have comment or be unexported (revive)
+internal/config/config.go:242:70: (*AzureCredentials).getURLOpener - result 1 (error) is always nil (unparam)
+internal/config/config.go:285:8: G101: Potential hardcoded credentials (gosec)
internal/dependencyproxy/dependencyproxy.go:77: Function 'Inject' is too long (70 > 60) (funlen)
internal/dependencyproxy/dependencyproxy.go:115:32: `cancelled` is a misspelling of `canceled` (misspell)
internal/dependencyproxy/dependencyproxy_test.go:101:91: unused-parameter: parameter 'r' seems to be unused, consider removing or renaming it as _ (revive)
@@ -322,7 +322,7 @@ internal/redis/redis.go:1:1: ST1000: at least one file in a package should have
internal/redis/redis.go:16:2: blank-imports: a blank import should be only in a main or test package, or have a comment justifying it (revive)
internal/redis/redis.go:21:24: var-declaration: should omit type error from declaration of var errSentinelMasterAddr; it will be inferred from the right-hand side (revive)
internal/redis/redis.go:23:2: exported: exported var TotalConnections should have comment or be unexported (revive)
-internal/redis/redis.go:95:10: elseif: can replace 'else {if cond {}}' with 'else if cond {}' (gocritic)
+internal/redis/redis.go:102:10: elseif: can replace 'else {if cond {}}' with 'else if cond {}' (gocritic)
internal/redis/redis_test.go:18:2: error-nil: use require.NoError (testifylint)
internal/redis/redis_test.go:23:3: error-nil: use require.NoError (testifylint)
internal/redis/redis_test.go:66:15: `initialise` is a misspelling of `initialize` (misspell)
diff --git a/workhorse/cmd/gitlab-workhorse/config_test.go b/workhorse/cmd/gitlab-workhorse/config_test.go
index ae35c22b019..1a117ecda62 100644
--- a/workhorse/cmd/gitlab-workhorse/config_test.go
+++ b/workhorse/cmd/gitlab-workhorse/config_test.go
@@ -38,7 +38,8 @@ trusted_cidrs_for_x_forwarded_for = ["127.0.0.1/8", "192.168.0.1/8"]
trusted_cidrs_for_propagation = ["10.0.0.1/8"]
[redis]
-password = "redis password"
+Password = "redis password"
+SentinelUsername = "sentinel-user"
SentinelPassword = "sentinel password"
[object_storage]
provider = "test provider"
@@ -74,6 +75,7 @@ key = "/path/to/private/key"
// fields in each section; that should happen in the tests of the
// internal/config package.
require.Equal(t, "redis password", cfg.Redis.Password)
+ require.Equal(t, "sentinel-user", cfg.Redis.SentinelUsername)
require.Equal(t, "sentinel password", cfg.Redis.SentinelPassword)
require.Equal(t, "test provider", cfg.ObjectStorageCredentials.Provider)
require.Equal(t, uint32(123), cfg.ImageResizerConfig.MaxScalerProcs, "image resizer max_scaler_procs")
diff --git a/workhorse/internal/config/config.go b/workhorse/internal/config/config.go
index b1975c8ee75..85b1e22306d 100644
--- a/workhorse/internal/config/config.go
+++ b/workhorse/internal/config/config.go
@@ -100,6 +100,7 @@ type RedisConfig struct {
URL TomlURL
Sentinel []TomlURL
SentinelMaster string
+ SentinelUsername string
SentinelPassword string
Password string
DB *int
diff --git a/workhorse/internal/redis/redis.go b/workhorse/internal/redis/redis.go
index 1fd30b05de5..df87ec9ce68 100644
--- a/workhorse/internal/redis/redis.go
+++ b/workhorse/internal/redis/redis.go
@@ -53,6 +53,13 @@ const (
defaultIdleTimeout = 3 * time.Minute
)
+// SentinelOptions contains grouped, related values
+type SentinelOptions struct {
+ SentinelUsername string
+ SentinelPassword string
+ Sentinels []string
+}
+
// createDialer references https://github.com/redis/go-redis/blob/b1103e3d436b6fe98813ecbbe1f99dc8d59b06c9/options.go#L214
// it intercepts the error and tracks it via a Prometheus counter
func createDialer(sentinels []string, tlsConfig *tls.Config) func(ctx context.Context, network, addr string) (net.Conn, error) {
@@ -172,12 +179,13 @@ func configureRedis(cfg *config.RedisConfig) (*redis.Client, error) {
}
func configureSentinel(cfg *config.RedisConfig) *redis.Client {
- sentinelPassword, sentinels := sentinelOptions(cfg)
+ options := sentinelOptions(cfg)
client := redis.NewFailoverClient(&redis.FailoverOptions{
MasterName: cfg.SentinelMaster,
- SentinelAddrs: sentinels,
+ SentinelAddrs: options.Sentinels,
Password: cfg.Password,
- SentinelPassword: sentinelPassword,
+ SentinelUsername: options.SentinelUsername,
+ SentinelPassword: options.SentinelPassword,
DB: getOrDefault(cfg.DB, 0),
PoolSize: getOrDefault(cfg.MaxActive, defaultMaxActive),
@@ -187,7 +195,7 @@ func configureSentinel(cfg *config.RedisConfig) *redis.Client {
ReadTimeout: defaultReadTimeout,
WriteTimeout: defaultWriteTimeout,
- Dialer: createDialer(sentinels, nil),
+ Dialer: createDialer(options.Sentinels, nil),
})
client.AddHook(sentinelInstrumentationHook{})
@@ -195,10 +203,13 @@ func configureSentinel(cfg *config.RedisConfig) *redis.Client {
return client
}
-// sentinelOptions extracts the sentinel password and addresses in
: format
+// sentinelOptions extracts the sentinel username and password from the URLs
+// and addresses in : format.
// the order of priority for the passwords is: SentinelPassword -> first password-in-url
-func sentinelOptions(cfg *config.RedisConfig) (string, []string) {
+// SentinelUsername will be the username associated with SentinelPassword.
+func sentinelOptions(cfg *config.RedisConfig) SentinelOptions {
sentinels := make([]string, len(cfg.Sentinel))
+ sentinelUsername := cfg.SentinelUsername
sentinelPassword := cfg.SentinelPassword
for i := range cfg.Sentinel {
@@ -208,10 +219,16 @@ func sentinelOptions(cfg *config.RedisConfig) (string, []string) {
if pw, exist := sentinelDetails.User.Password(); exist && len(sentinelPassword) == 0 {
// sets password using the first non-empty password
sentinelPassword = pw
+
+ // If a password is specified, a username is optional. Ensure that we use the
+ // username associated with the password.
+ if username := sentinelDetails.User.Username(); username != "" && sentinelUsername == "" {
+ sentinelUsername = username
+ }
}
}
- return sentinelPassword, sentinels
+ return SentinelOptions{sentinelUsername, sentinelPassword, sentinels}
}
func getOrDefault(ptr *int, val int) int {
diff --git a/workhorse/internal/redis/redis_test.go b/workhorse/internal/redis/redis_test.go
index cbceb7e6183..e0f34ebdbfe 100644
--- a/workhorse/internal/redis/redis_test.go
+++ b/workhorse/internal/redis/redis_test.go
@@ -112,8 +112,10 @@ func TestConnectToSentinel(t *testing.T) {
func TestSentinelOptions(t *testing.T) {
testCases := []struct {
description string
+ inputSentinelUsername string
inputSentinelPassword string
inputSentinel []string
+ username string
password string
sentinels []string
}{
@@ -142,6 +144,38 @@ func TestSentinelOptions(t *testing.T) {
inputSentinelPassword: "password1",
password: "password1",
},
+ {
+ description: "specific sentinel username defined",
+ inputSentinel: []string{"redis://localhost:26480"},
+ inputSentinelUsername: "username1",
+ inputSentinelPassword: "password1",
+ sentinels: []string{"localhost:26480"},
+ username: "username1",
+ password: "password1",
+ },
+ {
+ description: "specific sentinel username defined in url",
+ inputSentinel: []string{"redis://username2:password2@localhost:26480", "redis://username3:password3@localhost:26481"},
+ sentinels: []string{"localhost:26480", "localhost:26481"},
+ username: "username2",
+ password: "password2",
+ },
+ {
+ description: "usernames and passwords defined specifically and in url",
+ inputSentinel: []string{"tcp://someuser2:password2@localhost:26480", "tcp://someuser3:password3@localhost:26481"},
+ sentinels: []string{"localhost:26480", "localhost:26481"},
+ inputSentinelUsername: "someuser1",
+ inputSentinelPassword: "password1",
+ username: "someuser1",
+ password: "password1",
+ },
+ {
+ description: "username set for first sentinel",
+ inputSentinel: []string{"tcp://someuser2@localhost:26480", "tcp://someuser3:password3@localhost:26481"},
+ sentinels: []string{"localhost:26480", "localhost:26481"},
+ username: "someuser3",
+ password: "password3",
+ },
}
for _, tc := range testCases {
@@ -153,13 +187,15 @@ func TestSentinelOptions(t *testing.T) {
sentinelUrls[i] = config.TomlURL{URL: *parsedURL}
}
- outputPw, outputSentinels := sentinelOptions(&config.RedisConfig{
+ options := sentinelOptions(&config.RedisConfig{
Sentinel: sentinelUrls,
+ SentinelUsername: tc.inputSentinelUsername,
SentinelPassword: tc.inputSentinelPassword,
})
- require.Equal(t, tc.password, outputPw)
- require.Equal(t, tc.sentinels, outputSentinels)
+ require.Equal(t, tc.username, options.SentinelUsername)
+ require.Equal(t, tc.password, options.SentinelPassword)
+ require.Equal(t, tc.sentinels, options.Sentinels)
})
}
}