[BigQuery: create(jobInfo)] QueryJob execution status is "DONE" but the destination table isn't created

[sherrywan629]

Thanks for stopping by to let us know something could be better!

PLEASE READ: If you have a support contract with Google, please create an issue in the support console instead of filing on GitHub. This will ensure a timely response.

Please run down the following list and make sure you've tried the usual "quick fixes":

• Search the issues already opened: https://github.com/googleapis/java-bigquery/issues
• Check for answers on StackOverflow: http://stackoverflow.com/questions/tagged/google-cloud-platform

If you are still having issues, please include as much information as possible:

Environment details

1. Specify the API at the beginning of the title. For example, "BigQuery: ...").
   BigQuery: create(jobInfo), with
   JobInfo jobInfo = JobInfo.newBuilder(queryConfig).setJobId(jobId).build();, so query config and job Id are generated in our code.
2. OS type and version: MacOs Sequoia 15.2
3. Java version: java 21
4. version(s):

Steps to reproduce

To include more details:
In our query job, what we do is we construct a SQL command which will select a bunch of records from a table, then we'll construct a QueryJobConfiguration in our code like this:

      QueryJobConfiguration queryConfig = QueryJobConfiguration.newBuilder(sqlString)
          .setDefaultDataset(datasetId)
          .setDestinationTable(tempUnpartitionedTable)
          .build(); 
      JobInfo jobInfo = JobInfo.newBuilder(queryConfig).setJobId(jobId).build();

In 99% the case, it works very well. But recently we've hit some corner cases, where the queryJob's status returned as "DONE", however the destination table isn't created.
We made sure that the sql returns valid records so there're valid records to insert into destination table
We also leveraged bq show and confirmed that the query job "recordsRead":"1000","recordsWritten":"1000" are valid and look correct, but the destination table isn't created.

Code example

Provided above but let me know if you need more details.

Stack trace

A sample job Id, which has a DONE status but not bringing the selected records into destination table: copyToTempUnpartitionedTable_reconciliation-development_development_master_cleansed_payment_events_1_2024-04-09_00_0_1

External references such as API reference guides

Any additional information below

Following these steps guarantees the quickest resolution possible.

Thanks!

[whuffman36]

Thanks for filing this issue @sherrywan629 . Just to clarify, are you explicitly creating the tempUnpartitionedTable table before sending the job to BigQuery, or are you expecting that the table be created implicitly during the job?

Also, when you mention that bq shows "recordsRead":"1000","recordsWritten":"1000", can you see where these records are being written to? If the destination table is not created, are the records written to an anonymous (unnamed) table?