Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

ERROR : java.lang.IllegalArgumentException: Missing required pipeline options: region #347

Open
indrajitp opened this issue May 27, 2021 · 1 comment
Labels
api: bigtable Issues related to the GoogleCloudPlatform/cloud-bigtable-examples API. type: question Request for information or clarification. Not an issue.

Comments

@indrajitp
Copy link

I have Data in BigQuery and trying to migrate that to BigTable using BigQueryBigtableTransfer DF template.
running it like

mvn package exec:exec \
    -DBigQueryBigtableTransfer \
    -Dbigtable.projectID=google-cloud-project-id \
    -Dbigtable.instanceID=Big-table-instance-id-dev \
    -Dgs=gs://google-cloud-project-id-bigtable/bigtable_table/ \
    -Dbq.query='SELECT * FROM `google-cloud-project-id.SAMPLE.bigtable_test`' \
    -Dregion=us-west1	

With or without region still gives error of region parameter missing even if region paramater is no where mentioned in redme file.
getting below error -

[INFO] --- exec-maven-plugin:1.4.0:exec (default-cli) @ dataflow-connector-examples ---
SLF4J: Class path contains multiple SLF4J bindings.
SLF4J: Found binding in [jar:file:/C:/Users/xyz/.m2/repository/org/slf4j/slf4j-log4j12/1.7.25/slf4j-log4j12-1.7.25.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: Found binding in [jar:file:/C:/Users/xyz/.m2/repository/org/slf4j/slf4j-simple/1.7.21/slf4j-simple-1.7.21.jar!/org/slf4j/impl/StaticLoggerBinder.class]
SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings for an explanation.
SLF4J: Actual binding is of type [org.slf4j.impl.Log4jLoggerFactory]
log4j:WARN No appenders could be found for logger (org.apache.beam.sdk.options.PipelineOptionsFactory).
log4j:WARN Please initialize the log4j system properly.
log4j:WARN See http://logging.apache.org/log4j/1.2/faq.html#noconfig for more info.
Exception in thread "main" java.lang.RuntimeException: Failed to construct instance from factory method DataflowRunner#fromOptions(interface org.apache.beam.sdk.options.PipelineOptions)
        at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:224)
        at org.apache.beam.sdk.util.InstanceBuilder.build(InstanceBuilder.java:155)
        at org.apache.beam.sdk.PipelineRunner.fromOptions(PipelineRunner.java:55)
        at org.apache.beam.sdk.Pipeline.create(Pipeline.java:149)
        at com.google.cloud.bigtable.dataflow.example.BigQueryBigtableTransfer.main(BigQueryBigtableTransfer.java:118)
Caused by: java.lang.reflect.InvocationTargetException
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.beam.sdk.util.InstanceBuilder.buildFromMethod(InstanceBuilder.java:214)
        ... 4 more
Caused by: java.lang.IllegalArgumentException: Missing required pipeline options: region
        at org.apache.beam.runners.dataflow.DataflowRunner.fromOptions(DataflowRunner.java:262)
        ... 9 more

Can you please advice whats wrong?

@product-auto-label product-auto-label bot added the api: bigtable Issues related to the GoogleCloudPlatform/cloud-bigtable-examples API. label May 27, 2021
@meredithslota meredithslota added the type: question Request for information or clarification. Not an issue. label Oct 13, 2021
@andyasoriado002
Copy link

andyasoriado002 commented May 3, 2022

Having the same issue, added this line and I was able to run the file

$ tail -14  src/main/java/com/google/cloud/bigtable/dataflow/example/SourceRowCount.java 
    options.setRegion("us-central1");
    Pipeline p = Pipeline.create(options);

    p.apply(Read.from(CloudBigtableIO.read(config)))
        .apply(Count.<Result>globally())
        .apply(ParDo.of(stringifier))
        .apply(TextIO.write().to(options.getResultLocation()));
    // [END bigtable_dataflow_connector_scan_config]

    p.run().waitUntilFinish();

    // Once this is done, you can get the result file via "gsutil cp <resultLocation>-00000-of-00001"
  }

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
api: bigtable Issues related to the GoogleCloudPlatform/cloud-bigtable-examples API. type: question Request for information or clarification. Not an issue.
Projects
None yet
Development

No branches or pull requests

3 participants