Data pipeline design, storage, and machine learning service concepts tested on the Google Cloud Professional Data Engineer exam.
40 cards · basic cards · AI-written, checked twice. Edit anything.
- What is Apache Beam's role in Google Cloud Dataflow?
- Beam is the open-source programming model that Dataflow executes as a managed service.
- Which GCP service scales automatically for both batch and streaming data pipelines?
- Cloud Dataflow
- What is BigQuery's primary storage format for analytical queries?
- Columnar format, optimized for OLAP workloads.
- When should you use BigQuery's slot-based pricing instead of on-demand pricing?
- When you have predictable, sustained query volume or want to cap costs.
- How does Cloud Storage lifecycle management reduce costs?
- By automatically transitioning objects to cheaper storage classes (Nearline, Coldline, Archive) based on age or access patterns.
- Which Cloud Storage class is best for data accessed less than once per quarter?
- Archive
- What is the difference between Cloud Pub/Sub and Dataflow?
- Pub/Sub is a messaging service for publishing and subscribing; Dataflow processes the messages at scale.
- When designing a data pipeline, why would you use Pub/Sub as an intermediate layer?
- To decouple producers from consumers, buffer bursts, and enable multiple subscribers.
- What is Cloud Dataproc used for?
- Running Apache Hadoop and Spark clusters on GCP for big data processing.
- When would you choose Dataproc over Dataflow for a pipeline?
- When you need to run existing Spark or Hadoop code, or need custom cluster configurations.
- What is the purpose of Cloud SQL in a data pipeline?
- To provide managed MySQL, PostgreSQL, or SQL Server databases for transactional data.
- How does Cloud SQL replication improve reliability?
- Read replicas distribute queries and provide failover capability.
- What is Firestore designed for?
- NoSQL document database for real-time, scalable applications.
- When should you use Cloud Bigtable instead of BigQuery?
- When you need low-latency reads/writes, time-series data, or handling high-throughput streaming.
- How does Cloud Bigtable handle scalability?
- By distributing data across many nodes using row-key sharding.