Skip to content

Commit fd19c37

Browse files
authored
[CLI] Add the reproduce command in CLI (#5029)
This PR adds the `reproduce` command into the `casp` CLI, enabling local reproduction of a ClusterFuzz testcase inside a Docker container. This command simplifies the process by managing the Docker environment and automatically mounting necessary configurations and credentials. --- ## Changes Made A new `reproduce` command was added to the `casp` CLI. Given a testcase-id, the command performs the following actions: 1. Pulls the appropriate ClusterFuzz Docker immutable image (if not already pulled). 2. Mounts the user's gcloud credentials and any custom ClusterFuzz configurations into the container. 3. Executes the butler.py reproduce command within the container to fetch the testcase, set up the job and fuzzer environment, and attempt to reproduce the crash. 4. Streams the output from the container to the local terminal. This PR also introduces some utility functions: - `load_and_validate_config` to `casp.utils.config`; - `build_command` to the new `casp.utils.container`; - As this is a common part for most future commands, these functions build the command to run within the container; - The new module also keeps some common constants; - `prepare_docker_volumes` to `docker_utils`; --- ### How to Test Manually First, install `casp`. If you have not installed yet, please navigate to the `cli` directory and run to install it in editable mode: ```shell $ pip install -e casp ``` 1. Authentication & Setup Now, you must have run `casp init` to configure your credentials. If you haven't, please run it now: ```shell $ casp init ``` 2. Usage Once set up, run the reproduce command with the following format: ```shell $ casp reproduce --testcase-id=<testcase_id> -p internal ``` --- ### Testing Unit tests for this feature have been added in `cli/casp/src/casp/tests/commands/test_reproduce.py`. Unit tests for `casp.utils.container` will be added in future PR. ### Demo prints Here are two examples reproducing testcases of both `internal` and `external` projects. * `casp reproduce --testcase-id=5115444050460672 -p internal` (truncated): <img width="1885" height="297" alt="image" src="https://github.com/user-attachments/assets/6fdb5e63-ea5c-45e2-bf38-247d5ff7c2e2" /> <img width="1884" height="190" alt="image" src="https://github.com/user-attachments/assets/f5295879-c26c-45ec-a83a-33e0e330924b" /> * `casp reproduce -p external --testcase-id=6083012753424384` (truncated): <img width="1884" height="218" alt="image" src="https://github.com/user-attachments/assets/60cfefc4-57b5-422a-9e69-47feeb4ad742" /> <img width="1884" height="190" alt="image" src="https://github.com/user-attachments/assets/55e3126a-1eb7-4de2-9672-cf0a89a6203a" />
1 parent 17b2227 commit fd19c37

File tree

5 files changed

+315
-15
lines changed

5 files changed

+315
-15
lines changed

cli/casp/src/casp/commands/reproduce.py

Lines changed: 57 additions & 5 deletions
Original file line numberDiff line numberDiff line change
@@ -11,12 +11,64 @@
1111
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
1212
# See the License for the specific language governing permissions and
1313
# limitations under the License.
14-
"""Reproduce command."""
14+
"""Reproduces a testcase locally using Docker."""
1515

16+
import sys
17+
18+
from casp.utils import config
19+
from casp.utils import container
20+
from casp.utils import docker_utils
1621
import click
1722

1823

19-
@click.command(name='reproduce', help='Reproduces a testcase locally')
20-
def cli():
21-
"""Reproduces a testcase locally"""
22-
click.echo('To be implemented...')
24+
@click.command(
25+
name='reproduce',
26+
help=('Reproduces a testcase locally. '
27+
' WARN: This essentially runs untrusted code '
28+
'in your local environment. '
29+
'Please acknowledge the testcase (mainly input and build) '
30+
'before running this command.'))
31+
@click.option(
32+
'--project',
33+
'-p',
34+
help='The ClusterFuzz project to use.',
35+
required=True,
36+
type=click.Choice(
37+
docker_utils.PROJECT_TO_IMAGE.keys(), case_sensitive=False),
38+
)
39+
@click.option(
40+
'--config-dir',
41+
'-c',
42+
required=False,
43+
default=str(container.CONTAINER_CONFIG_PATH / 'config'),
44+
help=('Path to the config directory. If you set a custom '
45+
'config directory, this argument is not used.'),
46+
)
47+
@click.option(
48+
'--testcase-id', required=True, help='The ID of the testcase to reproduce.')
49+
def cli(project: str, config_dir: str, testcase_id: str) -> None:
50+
"""Reproduces a testcase locally by running a Docker container.
51+
52+
Args:
53+
project: The ClusterFuzz project name.
54+
config_dir: The default configuration directory path within the container.
55+
testcase_id: The ID of the testcase to be reproduced.
56+
"""
57+
cfg = config.load_and_validate_config()
58+
59+
volumes, container_config_dir = docker_utils.prepare_docker_volumes(
60+
cfg, config_dir)
61+
62+
command = container.build_butler_command(
63+
'reproduce',
64+
config_dir=str(container_config_dir),
65+
testcase_id=testcase_id,
66+
)
67+
68+
if not docker_utils.run_command(
69+
command,
70+
volumes,
71+
privileged=True,
72+
image=docker_utils.PROJECT_TO_IMAGE[project],
73+
):
74+
sys.exit(1)
Lines changed: 154 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,154 @@
1+
# Copyright 2025 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
"""Tests for the reproduce command.
15+
16+
For running all the tests, use (from the root of the project):
17+
python -m unittest discover -s cli/casp/src/casp/tests -p test_reproduce.py -v
18+
"""
19+
20+
from pathlib import Path
21+
import unittest
22+
from unittest.mock import patch
23+
24+
from casp.commands import reproduce
25+
from click.testing import CliRunner
26+
27+
28+
class ReproduceCliTest(unittest.TestCase):
29+
"""Tests for the reproduce command."""
30+
31+
def setUp(self):
32+
self.runner = CliRunner()
33+
self.mock_config = self.enterContext(
34+
patch('casp.commands.reproduce.config', autospec=True))
35+
self.mock_docker_utils = self.enterContext(
36+
patch('casp.commands.reproduce.docker_utils', autospec=True))
37+
self.mock_container = self.enterContext(
38+
patch('casp.commands.reproduce.container', autospec=True))
39+
40+
def test_reproduce_success(self):
41+
"""Tests successful reproduction with default options."""
42+
self.mock_config.load_and_validate_config.return_value = {
43+
'gcloud_credentials_path': '/fake/credentials/path'
44+
}
45+
self.mock_docker_utils.prepare_docker_volumes.return_value = ({
46+
'/fake/credentials': {
47+
'bind': '/root/.config/gcloud/',
48+
'mode': 'rw'
49+
}
50+
}, Path('/container/config/dir'))
51+
self.mock_container.build_butler_command.return_value = ['run']
52+
self.mock_docker_utils.run_command.return_value = True
53+
54+
result = self.runner.invoke(
55+
reproduce.cli, ['--testcase-id', '123', '--project', 'internal'])
56+
57+
self.assertEqual(0, result.exit_code, msg=result.output)
58+
self.mock_docker_utils.run_command.assert_called_once_with(
59+
['run'],
60+
{'/fake/credentials': {
61+
'bind': '/root/.config/gcloud/',
62+
'mode': 'rw'
63+
}},
64+
privileged=True,
65+
image=self.mock_docker_utils.PROJECT_TO_IMAGE['internal'],
66+
)
67+
68+
def test_reproduce_success_with_custom_config(self):
69+
"""Tests successful reproduction with a custom config path."""
70+
self.mock_config.load_and_validate_config.return_value = {
71+
'gcloud_credentials_path': '/fake/credentials/path',
72+
'custom_config_path': '/my/custom/config'
73+
}
74+
self.mock_docker_utils.prepare_docker_volumes.return_value = ({
75+
'/fake/credentials': {
76+
'bind': '/root/.config/gcloud/',
77+
'mode': 'rw'
78+
},
79+
'/my/custom/config': {
80+
'bind': '/container/custom/config',
81+
'mode': 'rw'
82+
}
83+
}, Path('/container/custom/config'))
84+
self.mock_container.build_butler_command.return_value = ['run']
85+
self.mock_docker_utils.run_command.return_value = True
86+
87+
result = self.runner.invoke(
88+
reproduce.cli, ['--testcase-id', '123', '--project', 'internal'])
89+
90+
self.assertEqual(0, result.exit_code, msg=result.output)
91+
self.mock_docker_utils.run_command.assert_called_once()
92+
self.mock_container.build_butler_command.assert_called_once_with(
93+
'reproduce',
94+
config_dir='/container/custom/config',
95+
testcase_id='123',
96+
)
97+
98+
def test_reproduce_no_config(self):
99+
"""Tests when no config is found."""
100+
self.mock_config.load_and_validate_config.side_effect = SystemExit(1)
101+
result = self.runner.invoke(
102+
reproduce.cli, ['--testcase-id', '123', '--project', 'internal'])
103+
104+
self.assertEqual(1, result.exit_code)
105+
self.mock_docker_utils.run_command.assert_not_called()
106+
107+
def test_reproduce_no_gcloud_credentials(self):
108+
"""Tests when gcloud credentials are not in the config."""
109+
self.mock_config.load_and_validate_config.side_effect = SystemExit(1)
110+
result = self.runner.invoke(
111+
reproduce.cli, ['--testcase-id', '123', '--project', 'internal'])
112+
113+
self.assertEqual(1, result.exit_code)
114+
self.mock_docker_utils.run_command.assert_not_called()
115+
116+
def test_reproduce_docker_command_fails(self):
117+
"""Tests when the docker command fails."""
118+
self.mock_config.load_and_validate_config.return_value = {
119+
'gcloud_credentials_path': '/fake/credentials/path'
120+
}
121+
self.mock_docker_utils.run_command.return_value = False
122+
self.mock_docker_utils.prepare_docker_volumes.return_value = (
123+
{}, Path('/mock/path'))
124+
self.mock_container.build_butler_command.return_value = ['fail']
125+
126+
result = self.runner.invoke(
127+
reproduce.cli, ['--testcase-id', '123', '--project', 'internal'])
128+
129+
self.assertEqual(1, result.exit_code)
130+
self.mock_docker_utils.run_command.assert_called_once()
131+
132+
def test_reproduce_with_project_option(self):
133+
"""Tests that the --project option is passed to docker_utils."""
134+
self.mock_config.load_and_validate_config.return_value = {
135+
'gcloud_credentials_path': '/fake/credentials/path'
136+
}
137+
self.mock_docker_utils.prepare_docker_volumes.return_value = (
138+
{}, Path('/mock/path'))
139+
self.mock_container.build_butler_command.return_value = ['false']
140+
self.mock_docker_utils.run_command.return_value = True
141+
self.mock_docker_utils.PROJECT_TO_IMAGE = {'dev': 'dev-image'}
142+
143+
result = self.runner.invoke(reproduce.cli,
144+
['--testcase-id', '123', '--project', 'dev'])
145+
146+
self.assertEqual(0, result.exit_code, msg=result.output)
147+
self.mock_docker_utils.run_command.assert_called_once()
148+
_, kwargs = self.mock_docker_utils.run_command.call_args
149+
self.assertEqual(self.mock_docker_utils.PROJECT_TO_IMAGE['dev'],
150+
kwargs.get('image'))
151+
152+
153+
if __name__ == '__main__':
154+
unittest.main()

cli/casp/src/casp/utils/config.py

Lines changed: 17 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -21,21 +21,36 @@
2121

2222
import json
2323
import os
24+
import sys
25+
from typing import Any
26+
27+
import click
2428

2529
CONFIG_DIR = os.path.expanduser('~/.casp')
2630
CONFIG_FILE = os.path.join(CONFIG_DIR, 'config.json')
2731

2832

29-
def save_config(data):
33+
def save_config(data: dict[str, Any]) -> None:
3034
"""Saves configuration data."""
3135
os.makedirs(CONFIG_DIR, exist_ok=True)
3236
with open(CONFIG_FILE, 'w') as f:
3337
json.dump(data, f)
3438

3539

36-
def load_config():
40+
def load_config() -> dict[str, Any]:
3741
"""Loads configuration data."""
3842
if not os.path.exists(CONFIG_FILE):
3943
return {}
4044
with open(CONFIG_FILE) as f:
4145
return json.load(f)
46+
47+
48+
def load_and_validate_config() -> dict[str, Any]:
49+
"""Loads and validates the configuration."""
50+
cfg = load_config()
51+
if not cfg or 'gcloud_credentials_path' not in cfg:
52+
click.secho(
53+
'Error: gcloud credentials not found. Please run "casp init".',
54+
fg='red')
55+
sys.exit(1)
56+
return cfg
Lines changed: 52 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,52 @@
1+
# Copyright 2025 Google LLC
2+
#
3+
# Licensed under the Apache License, Version 2.0 (the "License");
4+
# you may not use this file except in compliance with the License.
5+
# You may obtain a copy of the License at
6+
#
7+
# http://www.apache.org/licenses/LICENSE-2.0
8+
#
9+
# Unless required by applicable law or agreed to in writing, software
10+
# distributed under the License is distributed on an "AS IS" BASIS,
11+
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
12+
# See the License for the specific language governing permissions and
13+
# limitations under the License.
14+
"""Container-specific utilities."""
15+
16+
from pathlib import Path
17+
18+
# The root directory for the ClusterFuzz source code inside the container.
19+
SRC_ROOT = Path('/data/clusterfuzz/src')
20+
21+
# The path to the ClusterFuzz appengine directory inside the container.
22+
# This directory is the default location for ClusterFuzz configurations.
23+
CONTAINER_CONFIG_PATH = SRC_ROOT / 'appengine'
24+
25+
# The path where gcloud credentials will be mounted inside the container.
26+
# This allows the container to authenticate with Google Cloud services.
27+
CONTAINER_CREDENTIALS_PATH = Path('/root/.config/gcloud/')
28+
29+
# The base command prefix for executing ClusterFuzz butler commands.
30+
# This ensures that commands are run with the correct Python environment
31+
# and logging settings within the container.
32+
_COMMAND_PREFIX = 'pipenv run python butler.py --local-logging'
33+
34+
35+
def build_butler_command(subcommand: str, **kwargs: str) -> list[str]:
36+
"""Builds a butler command to be executed inside the container.
37+
38+
Args:
39+
subcommand: The butler subcommand to execute (e.g., 'reproduce').
40+
**kwargs: A dictionary of command-line arguments to pass to the subcommand.
41+
For example, `testcase_id='123'` becomes
42+
'--testcase-id=123'.
43+
44+
Returns:
45+
A list of strings representing the command to be executed.
46+
"""
47+
command = f'{_COMMAND_PREFIX} {subcommand}'
48+
for key, value in kwargs.items():
49+
key = key.replace('_', '-')
50+
command += f' --{key}={value}'
51+
52+
return ['bash', '-c', command]

0 commit comments

Comments
 (0)