Skip to content

Commit 398994b

Browse files
XinranTangXinranTang
authored
Merge pull request #202 from smokestacklightnin/ci/docs/add-docs
[Docs] Add docs with `mkdocs`
2 parents aa48568 + aea4660 commit 398994b

13 files changed

Lines changed: 314 additions & 55 deletions

File tree

‎.github/workflows/cd-docs.yml‎

Lines changed: 48 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,48 @@
1+
name: deploy-docs
2+
on:
3+
workflow_dispatch:
4+
push:
5+
branches:
6+
- 'master'
7+
pull_request:
8+
permissions:
9+
contents: write
10+
jobs:
11+
deploy:
12+
runs-on: ubuntu-latest
13+
steps:
14+
- name: Checkout repo
15+
uses: actions/checkout@v4
16+
17+
- name: Set up Python 3.12
18+
uses: actions/setup-python@v5
19+
with:
20+
python-version: '3.12'
21+
cache: 'pip'
22+
cache-dependency-path: |
23+
setup.py
24+
25+
- name: Save time for cache for mkdocs
26+
run: echo "cache_id=$(date --utc '+%V')" >> $GITHUB_ENV
27+
28+
- name: Caching
29+
uses: actions/cache@v4
30+
with:
31+
key: mkdocs-material-${{ env.cache_id }}
32+
path: .cache
33+
restore-keys: |
34+
mkdocs-material-
35+
36+
- name: Install Dependencies
37+
run: pip install -r requirements-docs.txt
38+
39+
- name: Deploy to GitHub Pages
40+
if: (github.event_name != 'pull_request')
41+
run: |
42+
git config user.name github-actions[bot]
43+
git config user.email 41898282+github-actions[bot]@users.noreply.github.com
44+
mkdocs gh-deploy --force
45+
46+
- name: Build docs to check for errors
47+
run: mkdocs build
48+
if: (github.event_name == 'pull_request')

‎.gitignore‎

Lines changed: 43 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -9,6 +9,7 @@ __pycache__/
99

1010
# Distribution / packaging
1111
.Python
12+
build/
1213
develop-eggs/
1314
dist/
1415
downloads/
@@ -47,8 +48,10 @@ htmlcov/
4748
nosetests.xml
4849
coverage.xml
4950
*.cover
51+
*.py,cover
5052
.hypothesis/
5153
.pytest_cache/
54+
cover/
5255

5356
# Translations
5457
*.mo
@@ -58,6 +61,7 @@ coverage.xml
5861
*.log
5962
local_settings.py
6063
db.sqlite3
64+
db.sqlite3-journal
6165

6266
# Flask stuff:
6367
instance/
@@ -70,6 +74,7 @@ instance/
7074
docs/_build/
7175

7276
# PyBuilder
77+
.pybuilder/
7378
target/
7479

7580
# Jupyter Notebook
@@ -85,12 +90,33 @@ ipython_config.py
8590
# pipenv
8691
# According to pypa/pipenv#598, it is recommended to include Pipfile.lock in version control.
8792
# However, in case of collaboration, if having platform-specific dependencies or dependencies
88-
# having no cross-platform support, pipenv may install dependencies that don’t work, or not
93+
# having no cross-platform support, pipenv may install dependencies that don't work, or not
8994
# install all needed dependencies.
9095
#Pipfile.lock
9196

92-
# celery beat schedule file
97+
# poetry
98+
# Similar to Pipfile.lock, it is generally recommended to include poetry.lock in version control.
99+
# This is especially recommended for binary packages to ensure reproducibility, and is more
100+
# commonly ignored for libraries.
101+
# https://python-poetry.org/docs/basic-usage/#commit-your-poetrylock-file-to-version-control
102+
#poetry.lock
103+
104+
# pdm
105+
# Similar to Pipfile.lock, it is generally recommended to include pdm.lock in version control.
106+
#pdm.lock
107+
# pdm stores project-wide configurations in .pdm.toml, but it is recommended to not include it
108+
# in version control.
109+
# https://pdm.fming.dev/latest/usage/project/#working-with-version-control
110+
.pdm.toml
111+
.pdm-python
112+
.pdm-build/
113+
114+
# PEP 582; used by e.g. github.com/David-OConnor/pyflow and github.com/pdm-project/pdm
115+
__pypackages__/
116+
117+
# Celery stuff
93118
celerybeat-schedule
119+
celerybeat.pid
94120

95121
# SageMath parsed files
96122
*.sage.py
@@ -136,3 +162,18 @@ bazel-*
136162

137163
MODULE.bazel
138164
MODULE.bazel.lock
165+
# pytype static type analyzer
166+
.pytype/
167+
168+
# Cython debug symbols
169+
cython_debug/
170+
171+
# PyCharm
172+
# JetBrains specific template is maintained in a separate JetBrains.gitignore that can
173+
# be found at https://github.com/github/gitignore/blob/main/Global/JetBrains.gitignore
174+
# and can be added to the global gitignore or merged into this file. For a more nuclear
175+
# option (not recommended) you can uncomment the following to ignore the entire idea folder.
176+
#.idea/
177+
178+
# Bazel build files
179+
bazel-*

g3doc/get_started.md renamed to docs/index.md

Lines changed: 14 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -190,7 +190,7 @@ following list provides a non-exhaustive overview of some of the major benefits.
190190
within a range; find previous executions in a context with the same inputs.
191191

192192
See the
193-
[MLMD tutorial](https://www.tensorflow.org/tfx/tutorials/mlmd/mlmd_tutorial) for
193+
[MLMD tutorial](https://tensorflow.github.io/tfx/tutorials/mlmd/mlmd_tutorial/) for
194194
an example that shows you how to use the MLMD API and the metadata store to
195195
retrieve lineage information.
196196

@@ -484,10 +484,11 @@ the steps together within a single transaction, and the transaction is
484484
rolled-back when an error occurs. The migration script is provided together with
485485
any schema-change commit and verified through testing.
486486

487-
Note: The migration DDLs in MySQL are not transactional. When using MySQL, there
488-
should only be a single connection with the upgrade migration enabled to use the
489-
old database. Take a backup of the database before upgrading to prevent
490-
potential data losses.
487+
!!! Note
488+
The migration DDLs in MySQL are not transactional. When using MySQL, there
489+
should only be a single connection with the upgrade migration enabled to use the
490+
old database. Take a backup of the database before upgrading to prevent
491+
potential data losses.
491492

492493
### Downgrade the database schema
493494

@@ -509,11 +510,12 @@ metadata_store.downgrade_schema(connection_config,
509510
downgrade_to_schema_version = 0)
510511
```
511512

512-
Note: When downgrading, MLMD prevents data loss as much as possible. However,
513-
newer schema versions might be inherently more expressive and hence a downgrade
514-
can introduce data loss. When using backends that do not support DDL
515-
transactions (e.g., MySQL), the database should be backed up before downgrading
516-
and the downgrade script should be the only MLMD connection to the database.
513+
!!! Note
514+
When downgrading, MLMD prevents data loss as much as possible. However,
515+
newer schema versions might be inherently more expressive and hence a downgrade
516+
can introduce data loss. When using backends that do not support DDL
517+
transactions (e.g., MySQL), the database should be backed up before downgrading
518+
and the downgrade script should be the only MLMD connection to the database.
517519

518520
The list of `schema_version` used in MLMD releases are:
519521

@@ -555,7 +557,7 @@ ml-metadata (MLMD) | schema_version
555557

556558
The MLMD library has a high-level API that you can readily use with your ML
557559
pipelines. See the
558-
[MLMD API documentation](https://www.tensorflow.org/tfx/ml_metadata/api_docs/python/mlmd)
560+
[MLMD API documentation](api/mlmd/)
559561
for more details.
560562

561563
Check out
@@ -564,5 +566,5 @@ to learn how to use MLMD declarative nodes filtering capabilities on properties
564566
and 1-hop neighborhood nodes.
565567

566568
Also check out the
567-
[MLMD tutorial](https://www.tensorflow.org/tfx/tutorials/mlmd/mlmd_tutorial) to
569+
[MLMD tutorial](https://tensorflow.github.io/tfx/tutorials/mlmd/mlmd_tutorial/) to
568570
learn how to use MLMD to trace the lineage of your pipeline components.

‎g3doc/images/mlmd_flow.png‎

-48.9 KB
Binary file not shown.

‎g3doc/images/mlmd_overview.png‎

-53.8 KB
Binary file not shown.

‎mkdocs.yml‎

Lines changed: 118 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,118 @@
1+
site_name: "ML Metadata"
2+
repo_name: "ML Metadata"
3+
repo_url: https://github.com/google/ml-metadata
4+
5+
theme:
6+
name: material
7+
palette:
8+
# Palette toggle for automatic mode
9+
- media: "(prefers-color-scheme)"
10+
primary: custom
11+
accent: custom
12+
toggle:
13+
icon: material/brightness-auto
14+
name: Switch to light mode
15+
16+
# Palette toggle for light mode
17+
- media: "(prefers-color-scheme: light)"
18+
primary: custom
19+
accent: custom
20+
scheme: default
21+
toggle:
22+
icon: material/brightness-7
23+
name: Switch to dark mode
24+
25+
# Palette toggle for dark mode
26+
- media: "(prefers-color-scheme: dark)"
27+
primary: custom
28+
accent: custom
29+
scheme: slate
30+
toggle:
31+
icon: material/brightness-4
32+
name: Switch to system preference
33+
favicon: images/favicon.png
34+
35+
features:
36+
- content.code.copy
37+
- content.code.select
38+
plugins:
39+
- search
40+
- autorefs
41+
- mkdocstrings:
42+
default_handler: python
43+
handlers:
44+
python:
45+
options:
46+
show_source: true
47+
show_root_heading: true
48+
unwrap_annotated: true
49+
show_symbol_type_toc: true
50+
show_symbol_type_heading: true
51+
merge_init_into_class: true
52+
show_signature_annotations: true
53+
separate_signature: true
54+
signature_crossrefs: true
55+
group_by_category: true
56+
show_category_heading: true
57+
inherited_members: true
58+
show_submodules: true
59+
show_object_full_path: false
60+
show_root_full_path: true
61+
docstring_section_style: "spacy"
62+
summary: true
63+
filters:
64+
- "!^_"
65+
- "^__init__$"
66+
- "^__call__$"
67+
- "!^logger"
68+
- "!_test$"
69+
extensions:
70+
- griffe_inherited_docstrings
71+
import:
72+
- https://docs.python.org/3/objects.inv
73+
- caption:
74+
figure:
75+
ignore_alt: true
76+
77+
markdown_extensions:
78+
- admonition
79+
- attr_list
80+
- toc:
81+
permalink: true
82+
- pymdownx.highlight:
83+
anchor_linenums: true
84+
linenums: false
85+
line_spans: __span
86+
pygments_lang_class: true
87+
- pymdownx.inlinehilite
88+
- pymdownx.snippets
89+
- pymdownx.superfences
90+
- pymdownx.arithmatex:
91+
generic: true
92+
- md_in_html
93+
- pymdownx.emoji:
94+
emoji_index: !!python/name:material.extensions.emoji.twemoji
95+
emoji_generator: !!python/name:material.extensions.emoji.to_svg
96+
97+
extra_css:
98+
- stylesheets/extra.css
99+
100+
extra_javascript:
101+
- javascripts/mathjax.js
102+
- https://unpkg.com/mathjax@3/es5/tex-mml-chtml.js
103+
104+
watch:
105+
- ml_metadata
106+
nav:
107+
- Guide: index.md
108+
- Tutorial: https://tensorflow.github.io/tfx/tutorials/mlmd/mlmd_tutorial/
109+
- API:
110+
- mlmd:
111+
- Overview: api/mlmd/index.md
112+
- mlmd: api/mlmd/mlmd.md
113+
- mlmd.errors:
114+
- Overview: api/mlmd.errors/index.md
115+
- mlmd.errors: api/mlmd.errors/mlmd.errors.md
116+
- mlmd.proto:
117+
- Overview: api/mlmd.proto/index.md
118+
- mlmd.proto: api/mlmd.proto/mlmd.proto.md

‎ml_metadata/__init__.py‎

Lines changed: 21 additions & 14 deletions
Original file line numberDiff line numberDiff line change
@@ -13,23 +13,30 @@
1313
# limitations under the License.
1414
"""Init module for ML Metadata."""
1515

16-
1716
# pylint: disable=g-import-not-at-top
1817
try:
19-
from ml_metadata import proto
18+
from ml_metadata import proto
2019

21-
# Import metadata_store API.
22-
from ml_metadata.metadata_store import (
23-
ListOptions,
24-
MetadataStore,
25-
OrderByField,
26-
downgrade_schema,
27-
)
20+
# Import metadata_store API.
21+
from ml_metadata.metadata_store import (
22+
ListOptions,
23+
MetadataStore,
24+
OrderByField,
25+
downgrade_schema,
26+
)
2827

29-
# Import version string.
30-
from ml_metadata.version import __version__
28+
# Import version string.
29+
from ml_metadata.version import __version__
3130

3231
except ImportError as err:
33-
import sys
34-
sys.stderr.write(f'Error importing: {err}')
35-
# pylint: enable=g-import-not-at-top
32+
import sys # pylint: enable=g-import-not-at-top
33+
34+
sys.stderr.write(f"Error importing: {err}")
35+
36+
37+
__all__ = [
38+
"downgrade_schema",
39+
"ListOptions",
40+
"MetadataStore",
41+
"OrderByField",
42+
]

‎ml_metadata/metadata_store/__init__.py‎

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -18,3 +18,10 @@
1818
OrderByField,
1919
downgrade_schema,
2020
)
21+
22+
__all__ = [
23+
"downgrade_schema",
24+
"ListOptions",
25+
"MetadataStore",
26+
"OrderByField",
27+
]

‎ml_metadata/metadata_store/metadata_store.py‎

Lines changed: 9 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -53,7 +53,13 @@
5353

5454
@enum.unique
5555
class OrderByField(enum.Enum):
56-
"""Defines the available fields to order results in ListOperations."""
56+
"""Defines the available fields to order results in ListOperations.
57+
58+
Attributes:
59+
CREATE_TIME: `<OrderByField.CREATE_TIME: 1>`
60+
ID: `<OrderByField.ID: 3>`
61+
UPDATE_TIME: `<OrderByField.UPDATE_TIME: 2>`
62+
"""
5763

5864
CREATE_TIME = (
5965
metadata_store_pb2.ListOperationOptions.OrderByField.Field.CREATE_TIME)
@@ -1041,8 +1047,8 @@ def get_artifacts_and_types_by_artifact_ids(
10411047
10421048
Returns:
10431049
Artifacts with matching ids and ArtifactTypes which can be matched by
1044-
type_ids from Artifacts. Each ArtifactType contains id, name,
1045-
properties and custom_properties fields.
1050+
type_ids from Artifacts. Each ArtifactType contains id, name,
1051+
properties and custom_properties fields.
10461052
"""
10471053
del extra_options
10481054
request = metadata_store_service_pb2.GetArtifactsByIDRequest(

0 commit comments

Comments
 (0)