Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 3 additions & 4 deletions docs/components/components.md
Original file line number Diff line number Diff line change
@@ -1,5 +1,3 @@
from distributed import Client

# Components

Fondant makes it easy to build dataset collaborative leveraging reusable components. Fondant
Expand Down Expand Up @@ -70,17 +68,18 @@ schema defined by the `produces` section of the component specification.
### Configuring Dask

You can configure the [Dask client](https://docs.dask.org/en/stable/scheduling.html) based on the
needs of your component by overriding the `dask_client` method:
needs of your component by overriding the `setup` method:

```python
import os
import typing as t

from dask.distributed import Client, LocalCluster
from fondant.component import PandasTransformComponent

class Component(PandasTransformComponent):

def dask_client(self) -> Client:
def setup(self) -> t.Any:
"""Initialize the dask client to use for this component."""
cluster = LocalCluster(
processes=True,
Expand Down
2 changes: 1 addition & 1 deletion docs/dataset.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ You can initialise a dataset from a previous run by using the `read` method.
```python
from fondant.dataset import Dataset

dataset = Dataset.read("path/to/manfiest.json")
dataset = Dataset.read("path/to/manifest.json")

```

Expand Down
2 changes: 1 addition & 1 deletion docs/guides/build_a_simple_dataset.md
Original file line number Diff line number Diff line change
Expand Up @@ -99,7 +99,7 @@ We provide three arguments to the `.create()` method:
To materialize your dataset, you can execute the following command within the directory:

```bash
fondant run local dataset.py --working_direcotry ./data
fondant run local dataset.py --working-directory ./data
```

The workflow execution will start, initiating the download of the dataset from HuggingFace.
Expand Down
2 changes: 1 addition & 1 deletion docs/runners/local.md
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ about this in the [installation](../guides/installation.md) guide.

### Running a dataset with the local runner

Fondant will create a default working directory (for intermediate artifacts) for the dataset in the current working directory called '/.artifacts'. You can override this by passing the `--working-dir` argument to the `run` command. Or by setting the `working_dir` argument in the `run` method of the `DockerRunner` class.
Fondant will create a default working directory (for intermediate artifacts) for the dataset in the current working directory called '/.artifacts'. You can override this by passing the `--working-directory` argument to the `run` command. Or by setting the `working_directory` argument in the `run` method of the `DockerRunner` class.

=== "Console"

Expand Down
6 changes: 3 additions & 3 deletions docs/runners/sagemaker.md
Original file line number Diff line number Diff line change
Expand Up @@ -49,7 +49,7 @@ AWS with a role that has all the required permissions to launch a SageMaker pipe

```bash
fondant run sagemaker <dataset_ref> \
--working-dir $S3_BUCKET \
--working-directory $S3_BUCKET \
--role-arn $SAGEMAKER_ROLE_ARN
```

Expand All @@ -61,8 +61,8 @@ AWS with a role that has all the required permissions to launch a SageMaker pipe

runner = SageMakerRunner()
runner.run(
input=<path_to_dataset>,
working_dir=<s3_bucket>,
dataset=<path_to_dataset>,
working_directory=<s3_bucket>,
role_arn=<role_arn>,
pipeline_name=<sagemaker_pipeline_name>
)
Expand Down
2 changes: 1 addition & 1 deletion docs/runners/vertex.md
Original file line number Diff line number Diff line change
Expand Up @@ -56,7 +56,7 @@ info [here](https://codelabs.developers.google.com/vertex-pipelines-intro#2)
region=project_region,
service_account=service_account)
)
runner.run(input_spec=<path_to_compiled_spec>, working_dir=<working_dir>)
runner.run(dataset=<path_to_compiled_spec>, working_directory=<working_directory>)
```


Expand Down
Loading