Add latest changes from gitlab-org/gitlab@master

This commit is contained in:
GitLab Bot
2024-10-02 03:13:39 +00:00
parent c3cfc33cc0
commit 305e0c02a8
23 changed files with 568 additions and 122 deletions

View File

@ -1,8 +0,0 @@
---
name: web_ide_oauth
introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/138015
rollout_issue_url: https://gitlab.com/gitlab-org/gitlab/-/issues/433324
milestone: '16.7'
type: development
group: group::ide
default_enabled: true

View File

@ -765,6 +765,8 @@
- 1
- - search_zoekt_task_failed_event
- 1
- - search_zoekt_update_index_used_bytes
- 1
- - secrets_management_provision_project_secrets_manager
- 1
- - security_create_security_policy_project

View File

@ -5,4 +5,4 @@ feature_category: code_review_workflow
introduced_by_url: https://gitlab.com/gitlab-org/gitlab/-/merge_requests/160035
milestone: '17.3'
queued_migration_version: 20240719090631
finalized_by: # version of the migration that finalized this BBM
finalized_by: '20241001232051'

View File

@ -0,0 +1,21 @@
# frozen_string_literal: true
class FinalizeBackfillApprovalsProjectId < Gitlab::Database::Migration[2.2]
milestone '17.5'
disable_ddl_transaction!
restrict_gitlab_migration gitlab_schema: :gitlab_main_cell
def up
ensure_batched_background_migration_is_finished(
job_class_name: 'BackfillApprovalsProjectId',
table_name: :approvals,
column_name: :id,
job_arguments: [:project_id, :merge_requests, :target_project_id, :merge_request_id],
finalize: true
)
end
def down; end
end

View File

@ -0,0 +1 @@
18a0a5880415bbc6d9b4376b4d5d9733d9d3c9f9b0ce44f0236edcc218e0402b

View File

@ -0,0 +1,236 @@
---
stage: Secure
group: Composition Analysis
info: To determine the technical writer assigned to the Stage/Group associated with this page, see https://handbook.gitlab.com/handbook/product/ux/technical-writing/#assignments
---
# Dependency scanning by using SBOM
DETAILS:
**Tier:** Ultimate
**Offering:** GitLab.com, Self-managed, GitLab Dedicated
**Status:** Experiment
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/issues/395692) in GitLab 17.3 behind the feature flag `dependency_scanning_using_sbom_reports`.
FLAG:
The availability of this feature is controlled by a feature flag.
For more information, see the history.
This feature is available for testing, but not ready for production use.
Dependency scanning using CycloneDX SBOM analyzes your application's dependencies for known
vulnerabilities. All dependencies are scanned, including transitive dependencies, also known as
nested dependencies.
Dependency scanning is often considered part of Software Composition Analysis (SCA). SCA can contain
aspects of inspecting the items your code uses. These items typically include application and system
dependencies that are almost always imported from external sources, rather than sourced from items
you wrote yourself.
Dependency scanning can run in the development phase of your application's life cycle. Every time a
pipeline produces an SBOM report, security findings are identified and compared between the source
and target branches. Findings and their severity are listed in the merge request, enabling you to
proactively address the risk to your application, before the code change is committed. Security
findings can also be identified outside a pipeline by
[Continuous Vulnerability Scanning](../../continuous_vulnerability_scanning/index.md).
GitLab offers both dependency scanning and [container scanning](../../container_scanning/index.md) to
ensure coverage for all of these dependency types. To cover as much of your risk area as possible,
we encourage you to use all of our security scanners. For a comparison of these features, see
[Dependency Scanning compared to Container Scanning](../../comparison_dependency_and_container_scanning.md).
## Supported package managers
For a list of supported package managers, see the analyzer's
[supported files](https://gitlab.com/gitlab-org/security-products/analyzers/dependency-scanning/#supported-files).
## Dependency detection workflow
The dependency detection workflow is as follows:
1. The application to be scanned provides a CycloneDX SBOM file or creates one.
1. GitLab checks each of the dependencies listed in the SBOM against the GitLab Advisory Database.
1. If the dependency scanning job is run on the default branch: vulnerabilities are created, and can be seen in the vulnerability report.
If the dependency scanning job is run on a non-default branch: security findings are created, and can be seen in the pipeline security tab and MR security widget.
## Configuration
Enable the dependency scanning analyzer to ensure it scans your applications dependencies for known vulnerabilities.
You can then adjust its behavior by configuring the CI/CD component's inputs.
## Enabling the analyzer
Prerequisites:
- The component's [stage](https://gitlab.com/explore/catalog/components/dependency-scanning) is required in the `.gitlab-ci.yml` file.
- With self-managed runners you need a GitLab Runner with the
[`docker`](https://docs.gitlab.com/runner/executors/docker.html) or
[`kubernetes`](https://docs.gitlab.com/runner/install/kubernetes.html) executor.
- If you're using SaaS runners on GitLab.com, this is enabled by default.
- A supported lock file or dependency graph must be in the repository.
Alternatively, configure the CI/CD job to output either as a job artifact,
ensuring the artifacts are generated in a stage before the `dependency-scanning`
job's stage. See the following example.
To enable the analyzer, use the `main` [dependency scanning CI/CD component](https://gitlab.com/explore/catalog/components/dependency-scanning).
### Enabling the analyzer for a Maven project
The following example `.gitlab-ci.yml` demonstrates how to enable the CI/CD
component on a Maven project. The dependency graph is output as a job artifact
in the `build` stage, before dependency scanning runs.
```yaml
stages:
- build
- test
image: maven:3.9.9-eclipse-temurin-21
include:
- component: $CI_SERVER_FQDN/components/dependency-scanning/main@0.4.0
build:
# Running in the build stage ensures that the dependency-scanning job
# receives the maven.graph.json artifacts.
stage: build
script:
- mvn install
- mvn dependency:tree -DoutputType=json -DoutputFile=maven.graph.json
# Collect all maven.graph.json artifacts and pass them onto jobs
# in sequential stages.
artifacts:
paths:
- "**/*.jar"
- "**/maven.graph.json"
```
### Enabling the analyzer for a Gradle project
To enable the CI/CD component on a Gradle project:
1. Edit the `build.gradle` or `build.gradle.kts` to use the [gradle-dependency-lock-plugin](https://github.com/nebula-plugins/gradle-dependency-lock-plugin/wiki/Usage#example).
1. Configure the `.gitlab-ci.yml` file to generate the `dependencies.lock` artifacts, and pass them to the `dependency-scanning` job.
The following example demonstrates how to configure the component
for a Gradle project.
```yaml
stages:
- build
- test
# Define the image that contains Java and Gradle
image: gradle:8.0-jdk11
include:
- component: $CI_SERVER_FQDN/components/dependency-scanning/main@0.4.0
build:
# Running in the build stage ensures that the dependency-scanning job
# receives the maven.graph.json artifacts.
stage: build
script:
- gradle generateLock saveLock
- gradle assemble
# generateLock saves the lock file in the build/ directory of a project
# and saveLock copies it into the root of a project. To avoid duplicates
# and get an accurate location of the dependency, use find to remove the
# lock files in the build/ directory only.
after_script:
- find . -path '*/build/dependencies.lock' -print -delete
# Collect all dependencies.lock artifacts and pass them onto jobs
# in sequential stages.
artifacts:
paths:
- "**/dependencies.lock"
```
## Customizing analyzer behavior
The analyzer can be customized by configuring the CI/CD component's
[inputs](https://gitlab.com/explore/catalog/components/dependency-scanning).
## Output
The dependency scanning analyzer produces CycloneDX Software Bill of Materials (SBOM) for each supported
lock file or dependency graph export detected.
### CycloneDX Software Bill of Materials
> - Generally available in GitLab 15.7.
The dependency scanning analyzer outputs a [CycloneDX](https://cyclonedx.org/) Software Bill of Materials (SBOM)
for each supported lock or dependency graph export it detects. The CycloneDX SBOMs are created as job artifacts.
The CycloneDX SBOMs are:
- Named `gl-sbom-<package-type>-<package-manager>.cdx.json`.
- Available as job artifacts of the dependency scanning job.
- Uploaded as `cyclonedx` reports.
- Saved in the same directory as the detected lock or dependency graph exports files.
For example, if your project has the following structure:
```plaintext
.
├── ruby-project/
│ └── Gemfile.lock
├── ruby-project-2/
│ └── Gemfile.lock
└── php-project/
└── composer.lock
```
The following CycloneDX SBOMs are created as job artifacts:
```plaintext
.
├── ruby-project/
│ ├── Gemfile.lock
│ └── gl-sbom-gem-bundler.cdx.json
├── ruby-project-2/
│ ├── Gemfile.lock
│ └── gl-sbom-gem-bundler.cdx.json
└── php-project/
├── composer.lock
└── gl-sbom-packagist-composer.cdx.json
```
### Merging multiple CycloneDX SBOMs
You can use a CI/CD job to merge the multiple CycloneDX SBOMs into a single SBOM.
NOTE:
GitLab uses [CycloneDX Properties](https://cyclonedx.org/use-cases/#properties--name-value-store)
to store implementation-specific details in the metadata of each CycloneDX SBOM, such as the
location of dependency graph exports and lock files. If multiple CycloneDX SBOMs are merged together,
this information is removed from the resulting merged file.
For example, the following `.gitlab-ci.yml` extract demonstrates how the Cyclone SBOM files can be
merged, and the resulting file validated.
```yaml
stages:
- test
- merge-cyclonedx-sboms
include:
- component: $CI_SERVER_FQDN/components/dependency-scanning/main@0.4.0
merge cyclonedx sboms:
stage: merge-cyclonedx-sboms
image:
name: cyclonedx/cyclonedx-cli:0.27.1
entrypoint: [""]
script:
- find . -name "gl-sbom-*.cdx.json" -exec cyclonedx merge --output-file gl-sbom-all.cdx.json --input-files "{}" +
# optional: validate the merged sbom
- cyclonedx validate --input-version v1_6 --input-file gl-sbom-all.cdx.json
artifacts:
paths:
- gl-sbom-all.cdx.json
```

View File

@ -194,10 +194,11 @@ To disable 2FA:
### Enable the extension marketplace for the Web IDE and workspaces
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/161819) as a [beta](../../policy/experiment-beta-support.md#beta) in GitLab 17.0 [with flags](../../administration/feature_flags.md) named `web_ide_oauth` and `web_ide_extensions_marketplace`. Disabled by default.
> - [Enabled on GitLab.com](https://gitlab.com/groups/gitlab-org/-/epics/11769) in GitLab 17.4.
> - Feature flag `web_ide_oauth` [enabled on GitLab.com, self-managed, and GitLab Dedicated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/163181) and feature flag `web_ide_extensions_marketplace` [enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/459028) in GitLab 17.4.
> - Feature flag `web_ide_oauth` [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/167464) in GitLab 17.5.
FLAG:
The availability of this feature is controlled by feature flags.
The availability of this feature is controlled by a feature flag.
For more information, see the history.
If you have the Owner role for a top-level group, you can enable the

View File

@ -404,10 +404,11 @@ DETAILS:
**Offering:** GitLab.com
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/151352) as a [beta](../../policy/experiment-beta-support.md#beta) in GitLab 17.0 [with flags](../../administration/feature_flags.md) named `web_ide_oauth` and `web_ide_extensions_marketplace`. Disabled by default.
> - [Enabled on GitLab.com](https://gitlab.com/groups/gitlab-org/-/epics/11769) in GitLab 17.4.
> - Feature flag `web_ide_oauth` [enabled on GitLab.com, self-managed, and GitLab Dedicated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/163181) and feature flag `web_ide_extensions_marketplace` [enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/459028) in GitLab 17.4.
> - Feature flag `web_ide_oauth` [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/167464) in GitLab 17.5.
FLAG:
The availability of this feature is controlled by feature flags.
The availability of this feature is controlled by a feature flag.
For more information, see the history.
You can use the [extension marketplace](../project/web_ide/index.md#extension-marketplace) to search and

View File

@ -246,10 +246,11 @@ DETAILS:
**Offering:** GitLab.com
> - [Introduced](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/151352) as a [beta](../../../policy/experiment-beta-support.md#beta) in GitLab 17.0 [with flags](../../../administration/feature_flags.md) named `web_ide_oauth` and `web_ide_extensions_marketplace`. Disabled by default.
> - [Enabled on GitLab.com](https://gitlab.com/groups/gitlab-org/-/epics/11769) in GitLab 17.4.
> - Feature flag `web_ide_oauth` [enabled on GitLab.com, self-managed, and GitLab Dedicated](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/163181) and feature flag `web_ide_extensions_marketplace` [enabled on GitLab.com](https://gitlab.com/gitlab-org/gitlab/-/issues/459028) in GitLab 17.4.
> - Feature flag `web_ide_oauth` [removed](https://gitlab.com/gitlab-org/gitlab/-/merge_requests/167464) in GitLab 17.5.
FLAG:
The availability of this feature is controlled by feature flags.
The availability of this feature is controlled by a feature flag.
For more information, see the history.
Prerequisites:

View File

@ -4,7 +4,7 @@ module WebIde
module DefaultOauthApplication
class << self
def feature_enabled?(current_user)
Feature.enabled?(:vscode_web_ide, current_user) && Feature.enabled?(:web_ide_oauth, current_user)
Feature.enabled?(:vscode_web_ide, current_user)
end
def oauth_application

View File

@ -7,8 +7,7 @@ module WebIde
# @return [Boolean]
def self.feature_enabled_for_any_user?
feature_flag_enabled_for_any_actor?(:web_ide_extensions_marketplace) &&
feature_flag_enabled_for_any_actor?(:vscode_web_ide) &&
feature_flag_enabled_for_any_actor?(:web_ide_oauth)
feature_flag_enabled_for_any_actor?(:vscode_web_ide)
end
# This returns true if the extensions marketplace feature is available to the given user
@ -17,8 +16,7 @@ module WebIde
# @return [Boolean]
def self.feature_enabled?(user:)
Feature.enabled?(:web_ide_extensions_marketplace, user) &&
Feature.enabled?(:vscode_web_ide, user) &&
Feature.enabled?(:web_ide_oauth, user)
Feature.enabled?(:vscode_web_ide, user)
end
# This value is used when the end-user is accepting the third-party extension marketplace integration.

View File

@ -74,7 +74,7 @@
"@gitlab/fonts": "^1.3.0",
"@gitlab/query-language": "^0.0.5-a-20240903",
"@gitlab/svgs": "3.117.0",
"@gitlab/ui": "94.4.2",
"@gitlab/ui": "94.5.0",
"@gitlab/web-ide": "^0.0.1-dev-20240909013227",
"@mattiasbuelens/web-streams-adapter": "^0.1.0",
"@rails/actioncable": "7.0.8-4",

View File

@ -156,10 +156,6 @@ RSpec.describe Admin::ApplicationsController do
describe "#reset_oauth_application_settings" do
subject(:reset_oauth_application_settings) { post :reset_web_ide_oauth_application_settings }
before do
stub_feature_flags(web_ide_oauth: true)
end
it 'returns 500 if no oauth application exists' do
stub_application_setting(web_ide_oauth_application: nil)
reset_oauth_application_settings

View File

@ -144,30 +144,23 @@ RSpec.describe IdeHelper, feature_category: :web_ide do
describe '#show_web_ide_oauth_callback_mismatch_callout?' do
let_it_be(:oauth_application) { create(:oauth_application, owner: nil) }
it 'returns false if Web IDE OAuth is not enabled' do
stub_feature_flags(vscode_web_ide: true, web_ide_oauth: false)
before do
stub_feature_flags(vscode_web_ide: true)
end
it 'returns false if no Web IDE OAuth application found' do
expect(helper.show_web_ide_oauth_callback_mismatch_callout?).to be false
end
context 'when Web IDE OAuth is enabled' do
before do
stub_feature_flags(vscode_web_ide: true, web_ide_oauth: true)
end
it "returns true if domain does not match OAuth application callback URLs" do
stub_application_setting({ web_ide_oauth_application: oauth_application })
expect(helper.show_web_ide_oauth_callback_mismatch_callout?).to be true
end
it 'returns false if no Web IDE OAuth application found' do
expect(helper.show_web_ide_oauth_callback_mismatch_callout?).to be false
end
it "returns true if domain does not match OAuth application callback URLs" do
stub_application_setting({ web_ide_oauth_application: oauth_application })
expect(helper.show_web_ide_oauth_callback_mismatch_callout?).to be true
end
it "returns false if domain matches OAuth application callback URL" do
oauth_application.redirect_uri = "#{request.base_url}/oauth-redirect"
stub_application_setting({ web_ide_oauth_application: oauth_application })
expect(helper.show_web_ide_oauth_callback_mismatch_callout?).to be false
end
it "returns false if domain matches OAuth application callback URL" do
oauth_application.redirect_uri = "#{request.base_url}/oauth-redirect"
stub_application_setting({ web_ide_oauth_application: oauth_application })
expect(helper.show_web_ide_oauth_callback_mismatch_callout?).to be false
end
end

View File

@ -2,7 +2,7 @@
require 'spec_helper'
RSpec.describe Gitlab::HookData::MergeRequestBuilder do
RSpec.describe Gitlab::HookData::MergeRequestBuilder, feature_category: :code_review_workflow do
let_it_be(:merge_request) { create(:merge_request) }
let(:builder) { described_class.new(merge_request) }

View File

@ -7,17 +7,16 @@ RSpec.describe WebIde::DefaultOauthApplication, feature_category: :web_ide do
let_it_be(:oauth_application) { create(:oauth_application, owner: nil) }
describe '#feature_enabled?' do
where(:vscode_web_ide, :web_ide_oauth, :expectation) do
where(:vscode_web_ide, :expectation) do
[
[ref(:current_user), false, false],
[false, ref(:current_user), false],
[ref(:current_user), ref(:current_user), true]
[ref(:current_user), true],
[false, false]
]
end
with_them do
it 'returns the expected value' do
stub_feature_flags(vscode_web_ide: vscode_web_ide, web_ide_oauth: web_ide_oauth)
stub_feature_flags(vscode_web_ide: vscode_web_ide)
expect(described_class.feature_enabled?(current_user)).to be(expectation)
end

View File

@ -19,19 +19,17 @@ RSpec.describe WebIde::ExtensionsMarketplace, feature_category: :web_ide do
end
describe 'feature enabled methods' do
where(:vscode_web_ide, :web_ide_extensions_marketplace, :web_ide_oauth, :expectation) do
ref(:current_user) | ref(:current_user) | ref(:current_user) | true
ref(:current_user) | false | ref(:current_user) | false
ref(:current_user) | ref(:current_user) | false | false
false | ref(:current_user) | false | false
where(:vscode_web_ide, :web_ide_extensions_marketplace, :expectation) do
ref(:current_user) | ref(:current_user) | true
ref(:current_user) | false | false
false | ref(:current_user) | false
end
with_them do
before do
stub_feature_flags(
vscode_web_ide: vscode_web_ide,
web_ide_extensions_marketplace: web_ide_extensions_marketplace,
web_ide_oauth: web_ide_oauth
web_ide_extensions_marketplace: web_ide_extensions_marketplace
)
end
@ -72,7 +70,6 @@ RSpec.describe WebIde::ExtensionsMarketplace, feature_category: :web_ide do
before do
stub_feature_flags(
web_ide_extensions_marketplace: current_user,
web_ide_oauth: current_user,
vscode_web_ide: current_user
)
end

View File

@ -179,20 +179,6 @@ RSpec.describe IdeController, feature_category: :web_ide do
end
end
describe 'with web_ide_oauth flag off' do
before do
stub_feature_flags(web_ide_oauth: false)
end
it 'does not create oauth application' do
expect(Doorkeeper::Application).not_to receive(:new)
subject
expect(web_ide_oauth_application).to be_nil
end
end
it 'ensures web_ide_oauth_application' do
expect(Doorkeeper::Application).to receive(:new).and_call_original
@ -264,14 +250,6 @@ RSpec.describe IdeController, feature_category: :web_ide do
expect(response).to have_gitlab_http_status(:not_found)
end
it 'with web_ide_oauth flag off, returns not_found' do
stub_feature_flags(web_ide_oauth: false)
oauth_redirect
expect(response).to have_gitlab_http_status(:not_found)
end
end
end

View File

@ -44,26 +44,46 @@ RSpec.shared_examples 'does not render usage overview background aggregation not
end
end
RSpec.shared_examples 'renders metrics comparison table' do
let(:metric_table) { find_by_testid('panel-dora-chart') }
RSpec.shared_examples 'renders metrics comparison tables' do
let(:metric_tables) { page.all("[data-testid='panel-dora-chart']") }
let(:lifecycle_metrics_table) { metric_tables[0] }
let(:dora_metrics_table) { metric_tables[1] }
let(:security_metrics_table) { metric_tables[2] }
it "renders the available metrics" do
def expect_row_content(id, name, values)
row = find_by_testid("dora-chart-metric-#{id}")
expect(row).to be_visible
expect(row).to have_content name
expect(row).to have_content values
end
before do
wait_for_all_requests
end
expect(metric_table).to be_visible
expect(metric_table).to have_content format(_("Metrics comparison for %{title}"), title: panel_title)
it 'renders the Lifecycle metrics table' do
expect(lifecycle_metrics_table).to be_visible
expect(lifecycle_metrics_table).to have_content format(_("Lifecycle metrics: %{title}"), title: panel_title)
[
['lead-time-for-changes', _('Lead time for changes'), '3.0 d 40.0% 1.0 d 66.7% 0.0 d'],
['time-to-restore-service', _('Time to restore service'), '3.0 d 57.1% 5.0 d 66.7% 0.0 d'],
['lead-time', _('Lead time'), '4.0 d 33.3% 2.0 d 50.0% -'],
['cycle-time', _('Cycle time'), '3.0 d 50.0% 1.0 d 66.7% -'],
['issues', _('Issues created'), '1 66.7% 2 100.0% -'],
['issues-completed', _('Issues closed'), '1 66.7% 2 100.0% -'],
['deploys', _('Deploys'), '10 25.0% 5 50.0% -'],
['merge-request-throughput', _('Merge request throughput'), '1 50.0% 3 200.0% -'],
['median-time-to-merge', _('Median time to merge'), '- - -'],
['vulnerability-critical', _('Critical vulnerabilities over time'), '5 3 -'],
['vulnerability-high', _('High vulnerabilities over time'), '4 2 -'],
['median-time-to-merge', _('Median time to merge'), '- - -']
].each do |id, name, values|
expect_row_content(id, name, values)
end
end
it 'renders the DORA metrics table' do
expect(dora_metrics_table).to be_visible
expect(dora_metrics_table).to have_content format(_("DORA metrics: %{title}"), title: panel_title)
[
['lead-time-for-changes', _('Lead time for changes'), '3.0 d 40.0% 1.0 d 66.7% 0.0 d'],
['time-to-restore-service', _('Time to restore service'), '3.0 d 57.1% 5.0 d 66.7% 0.0 d'],
# The values of these metrics are dependent on the length of the month they are in. Due to the high
# flake risk associated with them, we only validate the expected structure of the table row instead
@ -71,11 +91,18 @@ RSpec.shared_examples 'renders metrics comparison table' do
['deployment-frequency', _('Deployment frequency'), %r{ 0\.\d+/d \d+\.\d% 0\.\d+/d \d+\.\d% 0\.0/d}],
['change-failure-rate', _('Change failure rate'), %r{0\.0% \d+\.\d% \d+\.\d% \d+\.\d% \d+\.\d%}]
].each do |id, name, values|
row = find_by_testid("dora-chart-metric-#{id}")
expect_row_content(id, name, values)
end
end
expect(row).to be_visible
expect(row).to have_content name
expect(row).to have_content values
it 'renders the Security metrics table' do
expect(security_metrics_table).to be_visible
expect(security_metrics_table).to have_content format(_("Security metrics: %{title}"), title: panel_title)
[
['vulnerability-critical', _('Critical vulnerabilities over time'), '5 3 -'],
['vulnerability-high', _('High vulnerabilities over time'), '4 2 -']
].each do |id, name, values|
expect_row_content(id, name, values)
end
end
end

View File

@ -8,11 +8,7 @@ cmd/gitlab-workhorse/main.go:168: Function 'run' has too many statements (62 > 4
cmd/gitlab-workhorse/main.go:200:30: G114: Use of net/http serve function that has no support for setting timeouts (gosec)
cmd/gitlab-workhorse/main.go:268:10: G112: Potential Slowloris Attack because ReadHeaderTimeout is not configured in the http.Server (gosec)
cmd/gitlab-workhorse/main_test.go:60:2: exitAfterDefer: os.Exit will exit, and `defer gitaly.CloseConnections()` will not run (gocritic)
cmd/gitlab-zip-cat/main.go:1:1: package-comments: should have a package comment (revive)
cmd/gitlab-zip-cat/main.go:19:5: exported: exported var Version should have comment or be unexported (revive)
cmd/gitlab-zip-cat/main.go:66:20: Error return value of `reader.Close` is not checked (errcheck)
cmd/gitlab-zip-cat/main.go:72:15: G110: Potential DoS vulnerability via decompression bomb (gosec)
cmd/gitlab-zip-cat/main.go:93:9: superfluous-else: if block ends with call to os.Exit function, so drop this else and outdent its block (revive)
cmd/gitlab-zip-cat/main.go:82:15: G110: Potential DoS vulnerability via decompression bomb (gosec)
cmd/gitlab-zip-metadata/limit/reader.go:1:1: package-comments: should have a package comment (revive)
cmd/gitlab-zip-metadata/limit/reader.go:9:5: exported: exported var ErrLimitExceeded should have comment or be unexported (revive)
cmd/gitlab-zip-metadata/limit/reader.go:20:1: exported: exported method LimitedReaderAt.ReadAt should have comment or be unexported (revive)

View File

@ -1,3 +1,4 @@
// Package main provides a utility for extracting and displaying files from a ZIP archive.
package main
import (
@ -16,6 +17,7 @@ import (
const progName = "gitlab-zip-cat"
// Version holds the version of the program, which is set during the build process.
var Version = "unknown"
var printVersion = flag.Bool("version", false, "Print version and exit")
@ -23,27 +25,35 @@ var printVersion = flag.Bool("version", false, "Print version and exit")
func main() {
flag.Parse()
version := fmt.Sprintf("%s %s", progName, Version)
if *printVersion {
fmt.Println(version)
fmt.Printf("%s %s\n", progName, Version)
os.Exit(0)
}
contextErr, statusErr := run()
if contextErr != nil && statusErr == nil {
fmt.Fprintln(os.Stderr, statusErr)
os.Exit(1)
}
if contextErr != nil && statusErr != nil {
fatalError(contextErr, statusErr)
}
}
func run() (error, error) {
archivePath := os.Getenv("ARCHIVE_PATH")
encodedFileName := os.Getenv("ENCODED_FILE_NAME")
if len(os.Args) != 1 || archivePath == "" || encodedFileName == "" {
fmt.Fprintf(os.Stderr, "Usage: %s\n", progName)
fmt.Fprintf(os.Stderr, "Env: ARCHIVE_PATH=https://path.to/archive.zip or /path/to/archive.zip\n")
fmt.Fprintf(os.Stderr, "Env: ENCODED_FILE_NAME=base64-encoded-file-name\n")
os.Exit(1)
return fmt.Errorf("usage: %s\nEnv: ARCHIVE_PATH=https://path.to/archive.zip or /path/to/archive.zip\nEnv: ENCODED_FILE_NAME=base64-encoded-file-name", progName), nil
}
scrubbedArchivePath := mask.URL(archivePath)
fileName, err := zipartifacts.DecodeFileEntry(encodedFileName)
if err != nil {
fatalError(fmt.Errorf("decode entry %q", encodedFileName), err)
return fmt.Errorf("decode entry %q", encodedFileName), err
}
ctx, cancel := context.WithCancel(context.Background())
@ -51,27 +61,28 @@ func main() {
archive, err := zipartifacts.OpenArchive(ctx, archivePath)
if err != nil {
fatalError(errors.New("open archive"), err)
return errors.New("open archive"), err
}
file := findFileInZip(fileName, archive)
if file == nil {
fatalError(fmt.Errorf("find %q in %q: not found", fileName, scrubbedArchivePath), zipartifacts.ErrorCode[zipartifacts.CodeEntryNotFound])
return fmt.Errorf("find %q in %q: not found", fileName, scrubbedArchivePath), zipartifacts.ErrorCode[zipartifacts.CodeEntryNotFound]
}
// Start decompressing the file
reader, err := file.Open()
if err != nil {
fatalError(fmt.Errorf("open %q in %q", fileName, scrubbedArchivePath), err)
return fmt.Errorf("open %q in %q", fileName, scrubbedArchivePath), err
}
defer reader.Close()
defer reader.Close() //nolint:errcheck
if _, err := fmt.Printf("%d\n", file.UncompressedSize64); err != nil {
fatalError(fmt.Errorf("write file size invalid"), err)
return fmt.Errorf("write file size invalid"), err
}
if _, err := io.Copy(os.Stdout, reader); err != nil {
fatalError(fmt.Errorf("write %q from %q to stdout", fileName, scrubbedArchivePath), err)
return fmt.Errorf("write %q from %q to stdout", fileName, scrubbedArchivePath), err
}
return nil, nil
}
func findFileInZip(fileName string, archive *zip.Reader) *zip.File {
@ -90,7 +101,6 @@ func fatalError(contextErr error, statusErr error) {
if code > 0 {
os.Exit(code)
} else {
os.Exit(1)
}
os.Exit(1)
}

View File

@ -0,0 +1,172 @@
package main
import (
"archive/zip"
"bytes"
"encoding/base64"
"io"
"os"
"sync"
"testing"
)
func TestRun(t *testing.T) {
tests := createTestCases()
for name, tt := range tests {
t.Run(name, func(t *testing.T) {
runTest(t, tt)
})
}
}
func createTestCases() map[string]struct {
archiveContent string
fileName string
expectedOutput string
expectedError string
missingArchiveEnv bool
missingEncodingEnv bool
overrideEncodedFileName string
} {
return map[string]struct {
archiveContent string
fileName string
expectedOutput string
expectedError string
missingArchiveEnv bool
missingEncodingEnv bool
overrideEncodedFileName string
}{
"successful case": {
archiveContent: "sample content",
fileName: "testfile.txt",
expectedOutput: "14\nsample content",
expectedError: "",
},
"missing archive path": {
archiveContent: "sample content",
fileName: "testfile.txt",
expectedOutput: "",
expectedError: "usage: gitlab-zip-cat\nEnv: ARCHIVE_PATH=https://path.to/archive.zip or /path/to/archive.zip\nEnv: ENCODED_FILE_NAME=base64-encoded-file-name",
missingArchiveEnv: true,
},
"missing encoding name": {
archiveContent: "sample content",
fileName: "testfile.txt",
expectedOutput: "",
expectedError: "usage: gitlab-zip-cat\nEnv: ARCHIVE_PATH=https://path.to/archive.zip or /path/to/archive.zip\nEnv: ENCODED_FILE_NAME=base64-encoded-file-name",
missingEncodingEnv: true,
},
}
}
func runTest(t *testing.T, tt struct {
archiveContent string
fileName string
expectedOutput string
expectedError string
missingArchiveEnv bool
missingEncodingEnv bool
overrideEncodedFileName string
}) {
archivePath := createTempZipWithFile(tt.fileName, tt.archiveContent)
encodedFileName := base64.StdEncoding.EncodeToString([]byte(tt.fileName))
setupEnvironment(tt.missingArchiveEnv, tt.missingEncodingEnv, archivePath, tt.overrideEncodedFileName, encodedFileName)
stdoutBuf, _, err := captureOutput(run)
verifyResults(t, tt.expectedError, err, tt.expectedOutput, stdoutBuf)
unsetEnvironment()
}
func setupEnvironment(missingArchiveEnv, missingEncodingEnv bool, archivePath, overrideEncodedFileName, encodedFileName string) {
if !missingArchiveEnv {
os.Setenv("ARCHIVE_PATH", archivePath)
}
if !missingEncodingEnv {
if overrideEncodedFileName != "" {
encodedFileName = base64.StdEncoding.EncodeToString([]byte(overrideEncodedFileName))
}
os.Setenv("ENCODED_FILE_NAME", encodedFileName)
}
os.Args = []string{"gitlab-zip-cat"}
}
func captureOutput(fn func() (error, error)) (stdoutBuf, stderrBuf bytes.Buffer, err error) {
stdoutPipe, stdoutWriter, _ := os.Pipe()
stderrPipe, stderrWriter, _ := os.Pipe()
defer stdoutPipe.Close()
defer stderrPipe.Close()
oldStdout := os.Stdout
oldStderr := os.Stderr
os.Stdout = stdoutWriter
os.Stderr = stderrWriter
defer func() {
os.Stdout = oldStdout
os.Stderr = oldStderr
}()
wg := sync.WaitGroup{}
wg.Add(2)
go func() {
defer wg.Done()
io.Copy(&stdoutBuf, stdoutPipe)
}()
go func() {
defer wg.Done()
io.Copy(&stderrBuf, stderrPipe)
}()
err, _ = fn()
stdoutWriter.Close()
stderrWriter.Close()
wg.Wait()
return stdoutBuf, stderrBuf, err
}
func verifyResults(t *testing.T, expectedError string, err error, expectedOutput string, stdoutBuf bytes.Buffer) {
if expectedError == "" && err != nil {
t.Fatalf("Expected no error, got %v", err)
}
if got := stdoutBuf.String(); got != expectedOutput {
t.Errorf("Expected %q, got %q", expectedOutput, got)
}
}
func unsetEnvironment() {
os.Unsetenv("ARCHIVE_PATH")
os.Unsetenv("ENCODED_FILE_NAME")
}
func createTempZipWithFile(fileName, content string) string {
zipFile, err := os.CreateTemp("", "test-archive-*.zip")
if err != nil {
panic(err)
}
defer zipFile.Close()
archive := zip.NewWriter(zipFile)
defer archive.Close()
w, err := archive.Create(fileName)
if err != nil {
panic(err)
}
_, err = io.Copy(w, bytes.NewReader([]byte(content)))
if err != nil {
panic(err)
}
return zipFile.Name()
}

View File

@ -1362,10 +1362,10 @@
resolved "https://registry.yarnpkg.com/@gitlab/svgs/-/svgs-3.117.0.tgz#a9a45949d73e91f278e019b46220feb63105dd48"
integrity sha512-nBFWh2UN+pFl7nBQUgaUtHRChrZSXdRNmv1J49QPLwyJYWuq51YEBXQW5mPAlvB1BGfiNJPSHSGxWALFZBI5WA==
"@gitlab/ui@94.4.2":
version "94.4.2"
resolved "https://registry.yarnpkg.com/@gitlab/ui/-/ui-94.4.2.tgz#83acfdf782c2efcf8d39adee7f77e021cc43076e"
integrity sha512-yybfTtkjXUHdtKuHIMuOcp3A+J8LYIo+7DeBcyaGJNnEZ3o19EypZ2Yy3UFzWk54vN6yIOtlrdE12gyMBWDwYA==
"@gitlab/ui@94.5.0":
version "94.5.0"
resolved "https://registry.yarnpkg.com/@gitlab/ui/-/ui-94.5.0.tgz#162f63b90293108d961b467726aefb9dca4602f2"
integrity sha512-ps1f76XnpTZjzMjbElqHIeV7eA4ZLKlvqwee3QlO+pYG9mehaD93liMxW2b0NV6v6HlWbe2/fx9x/MtDD2xJZA==
dependencies:
"@floating-ui/dom" "1.4.3"
echarts "^5.3.2"
@ -13135,7 +13135,16 @@ string-length@^4.0.1:
char-regex "^1.0.2"
strip-ansi "^6.0.0"
"string-width-cjs@npm:string-width@^4.2.0", "string-width@^1.0.2 || 2 || 3 || 4", string-width@^4.1.0, string-width@^4.2.0, string-width@^4.2.3:
"string-width-cjs@npm:string-width@^4.2.0":
version "4.2.3"
resolved "https://registry.yarnpkg.com/string-width/-/string-width-4.2.3.tgz#269c7117d27b05ad2e536830a8ec895ef9c6d010"
integrity sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==
dependencies:
emoji-regex "^8.0.0"
is-fullwidth-code-point "^3.0.0"
strip-ansi "^6.0.1"
"string-width@^1.0.2 || 2 || 3 || 4", string-width@^4.1.0, string-width@^4.2.0, string-width@^4.2.3:
version "4.2.3"
resolved "https://registry.yarnpkg.com/string-width/-/string-width-4.2.3.tgz#269c7117d27b05ad2e536830a8ec895ef9c6d010"
integrity sha512-wKyQRQpjJ0sIp62ErSZdGsjMJWsap5oRNihHhu6G7JVO/9jIB6UyevL+tXuOqrng8j/cxKTWyWUwvSTriiZz/g==
@ -13188,7 +13197,7 @@ string_decoder@^1.0.0, string_decoder@^1.1.1, string_decoder@~1.1.1:
dependencies:
safe-buffer "~5.1.0"
"strip-ansi-cjs@npm:strip-ansi@^6.0.1", strip-ansi@^6.0.0, strip-ansi@^6.0.1:
"strip-ansi-cjs@npm:strip-ansi@^6.0.1":
version "6.0.1"
resolved "https://registry.yarnpkg.com/strip-ansi/-/strip-ansi-6.0.1.tgz#9e26c63d30f53443e9489495b2105d37b67a85d9"
integrity sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==
@ -13202,6 +13211,13 @@ strip-ansi@^5.2.0:
dependencies:
ansi-regex "^4.1.0"
strip-ansi@^6.0.0, strip-ansi@^6.0.1:
version "6.0.1"
resolved "https://registry.yarnpkg.com/strip-ansi/-/strip-ansi-6.0.1.tgz#9e26c63d30f53443e9489495b2105d37b67a85d9"
integrity sha512-Y38VPSHcqkFrCpFnQ9vuSXmquuv5oXOKpGeT6aGrr3o3Gc9AlVa6JBfUSOCnbxGGZF+/0ooI7KrPuUSztUdU5A==
dependencies:
ansi-regex "^5.0.1"
strip-ansi@^7.0.1, strip-ansi@^7.1.0:
version "7.1.0"
resolved "https://registry.yarnpkg.com/strip-ansi/-/strip-ansi-7.1.0.tgz#d5b6568ca689d8561370b0707685d22434faff45"
@ -14893,7 +14909,7 @@ worker-loader@^3.0.8:
loader-utils "^2.0.0"
schema-utils "^3.0.0"
"wrap-ansi-cjs@npm:wrap-ansi@^7.0.0", wrap-ansi@^7.0.0:
"wrap-ansi-cjs@npm:wrap-ansi@^7.0.0":
version "7.0.0"
resolved "https://registry.yarnpkg.com/wrap-ansi/-/wrap-ansi-7.0.0.tgz#67e145cff510a6a6984bdf1152911d69d2eb9e43"
integrity sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q==
@ -14911,6 +14927,15 @@ wrap-ansi@^6.2.0:
string-width "^4.1.0"
strip-ansi "^6.0.0"
wrap-ansi@^7.0.0:
version "7.0.0"
resolved "https://registry.yarnpkg.com/wrap-ansi/-/wrap-ansi-7.0.0.tgz#67e145cff510a6a6984bdf1152911d69d2eb9e43"
integrity sha512-YVGIj2kamLSTxw6NsZjoBxfSwsn0ycdesmc4p+Q21c5zPuZ1pl+NfxVdxPtdHvmNVOQ6XSYG4AUtyt/Fi7D16Q==
dependencies:
ansi-styles "^4.0.0"
string-width "^4.1.0"
strip-ansi "^6.0.0"
wrap-ansi@^8.1.0:
version "8.1.0"
resolved "https://registry.yarnpkg.com/wrap-ansi/-/wrap-ansi-8.1.0.tgz#56dc22368ee570face1b49819975d9b9a5ead214"