01 Cloud Dataflow - Pub/Sub to Big Query Streaming
NextGen Learning NextGen Learning
14.9K subscribers
30,076 views
0

 Published On May 16, 2021

This video will explain how to setup a data flow job that moves data from pub/sub topic to Big Query table.

Commands used:
gcloud pubsub topics create MyTopic01
gsutil mb gs://dataengineer-01
bq mk mydataset01
bq mk mydataset01.mytable01 name:STRING,country:STRING

Message format
{

"name" : "John",
"country" : "US"
}

show more

Share/Embed