Data pipeline design, storage, and machine learning service concepts tested on the Google Cloud Professional Data Engineer exam.
40 cards · basic cards · AI-written, checked twice. Edit anything.
- What is Apache Beam's role in Google Cloud Dataflow?
Show answer
Beam is the open-source programming model that Dataflow executes as a managed service.- Which GCP service scales automatically for both batch and streaming data pipelines?
Show answer
Cloud Dataflow- What is BigQuery's primary storage format for analytical queries?
Show answer
Columnar format, optimized for OLAP workloads.- When should you use BigQuery's slot-based pricing instead of on-demand pricing?
Show answer
When you have predictable, sustained query volume or want to cap costs.- How does Cloud Storage lifecycle management reduce costs?
Show answer
By automatically transitioning objects to cheaper storage classes (Nearline, Coldline, Archive) based on age or access patterns.- Which Cloud Storage class is best for data accessed less than once per quarter?
Show answer
Archive- What is the difference between Cloud Pub/Sub and Dataflow?
Show answer
Pub/Sub is a messaging service for publishing and subscribing; Dataflow processes the messages at scale.- When designing a data pipeline, why would you use Pub/Sub as an intermediate layer?
Show answer
To decouple producers from consumers, buffer bursts, and enable multiple subscribers.- What is Cloud Dataproc used for?
Show answer
Running Apache Hadoop and Spark clusters on GCP for big data processing.- When would you choose Dataproc over Dataflow for a pipeline?
Show answer
When you need to run existing Spark or Hadoop code, or need custom cluster configurations.- What is the purpose of Cloud SQL in a data pipeline?
Show answer
To provide managed MySQL, PostgreSQL, or SQL Server databases for transactional data.- How does Cloud SQL replication improve reliability?
Show answer
Read replicas distribute queries and provide failover capability.- What is Firestore designed for?
Show answer
NoSQL document database for real-time, scalable applications.- When should you use Cloud Bigtable instead of BigQuery?
Show answer
When you need low-latency reads/writes, time-series data, or handling high-throughput streaming.- How does Cloud Bigtable handle scalability?
Show answer
By distributing data across many nodes using row-key sharding.
FlashKeepers