Menu
Is that possible to preview my streaming data at bigquery?
I'm using bigquery to store my streaming data, but these data will go to streaming buffer which means I can't preview this data. Is there any way I can preview this data before conducting any query?
604
Streaming data to BigQuery requires you to write multiple custom integration-based code snippets to establish a connection, thereby making it challenging, especially when the data source is not a Google service. This method of streaming data to BigQuery fails to handle the issue of duplicate records.
Stay organized with collections Save and categorize content based on your preferences. Inserts simple rows into a table using the streaming API (insertAll).
Listing datasets in a project. In the navigation menu, click SQL workspace. In the Explorer panel, expand a project name to see the datasets in that project, or use the search box to search by dataset name.
If the issue is cost, you can use table decorators that will let you scan the "last x milliseconds", instead of doing a whole table/column scan.
For example, this query shows you the last 15 minutes of GDELT record, at a cost of 132 MB (instead of 2.91 TB without the @-900000- decorator):
SELECT *
FROM [gdelt-bq:gdeltv2.gkg@-900000-]
LIMIT 1000
You can see streaming buffer stats via Tables: get API - in streamingBuffer property
or in Web UI
2
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
