Data Engineer - GCP

Who We Are

We are a dynamic team focused on building innovative and scalable data solutions on Google Cloud Platform (GCP). Our Google Cloud Data Engineer will play a key role in designing, developing, and managing scalable data pipelines and data infrastructure, ensuring data availability, accuracy, and performance for business insights and machine learning models.

What We Are Looking For

We are seeking an experienced and highly skilled Google Cloud Data Engineer who will be responsible for developing and managing data pipelines on GCP. The ideal candidate will bring strong expertise in cloud-based data processing, big data technologies, and data modeling to help us provide high-performance data solutions.

Responsibilities

*Data Pipeline Development and Management:*

• Design, build, and maintain scalable and reliable data pipelines using Cloud Dataflow, Cloud Pub/Sub, and Cloud Composer.
• Develop ETL/ELT processes to process and transform large volumes of structured and unstructured data.
• Optimize data pipeline performance, scalability, and reliability.
• Ensure data processing and ingestion workflows are monitored and meet performance SLAs.

*Data Storage and Management:*

• Design and implement data storage solutions using BigQuery, Cloud Storage, and Firestore.
• Optimize data structures and partitioning for performance and cost efficiency.
• Ensure data security, integrity, and availability in all storage solutions.
• Manage data lifecycle policies and archiving processes.

*Data Transformation and Processing:*

• Develop data transformation processes using BigQuery, Apache Beam, and Cloud Functions.
• Implement data quality checks, validation rules, and monitoring solutions.
• Support real-time and batch data processing needs.

*Data Integration and Automation:*

• Integrate data from multiple sources, including APIs, databases, and third-party applications.
• Automate data ingestion, transformation, and export using t...