[Bug]: can't test external provider (customed provider) when running apache beam yaml test. How to use external provider in apache beam yaml test?

[ZwxwZ]

What happened?

When using Apache Beam tests based on the documentation
(https://beam.apache.org/documentation/sdks/yaml-testing/),
I encountered an issue where my externally defined provider could not be recognized or used.
This means I was only able to use the built-in providers provided by Apache Beam.
I believe this contradicts the statement in the documentation: "Though the various built-in (and externally provided) transform in a Beam YAML pipeline can be expected to be well

error log:
ValueError: Unknown type or missing provider for type WriteToBigQueryCustom for
"WriteToBigQueryHierarchyCustom" at line 11

pipeline:
 transforms:
  - type: Create config:
 elements:
   - { "hrchy_core_id": "1001")
   - {  "hrchy _core_id": "1002"}
   - { "hrchy _core_id": "1003"}
   - { "hrchy core id": "1004"｝
   - { "hrchy core_id": "1005"}
- type: WriteToBigQueryCustom
   name: WriteToBigQueryHierarchyCustom
   input: Create 
   config:
      project_id: bsbc-11545401-cmpdatacwrs-dev
      data_set: gpps_staging
      table_name: gpps_hierarchy
- type: LogForTesting 
   name: LogForTesting
   input: WriteToBigQueryHierarchyCustom
providers:
   type: javaJar
   config: "gs: //hshc-12164545-dp-dev-1 ib=repo/java-repo/java provider/datatlow-java-provider-bundled-v1.014.jar"
   transforms:
      WriteToBigQueryCustom: "beam: schematransform: com.hsbc.compliance.rc.java.beam:bigquery_custom:v1"

local starting command

py -m apache_beam.yaml.main --yaml_pipeline_file=./custombq.yaml --tests

Issue Priority

Priority: 2 (default / most bugs should be filed as P2)

Issue Components

• Component: Python SDK
• Component: Java SDK
• Component: Go SDK
• Component: Typescript SDK
• Component: IO connector
• Component: Beam YAML
• Component: Beam examples
• Component: Beam playground
• Component: Beam katas
• Component: Website
• Component: Infrastructure
• Component: Spark Runner
• Component: Flink Runner
• Component: Samza Runner
• Component: Twister2 Runner
• Component: Hazelcast Jet Runner
• Component: Google Cloud Dataflow Runner

[liferoad]

Does this only happen when testing? Have you followed https://github.com/apache/beam-starter-java-provider to make sure your Java provider works?

[ZwxwZ]

yes, only happen in text, java provider working fine on gcp platform