Role Title: Big Data Engineer
Reports to: Head of Data
Location: London - Paddington
Purpose of Role
We are looking for an experienced Big Data Engineer that will work on the collecting, storing, processing, and analyzing of huge sets of data. The primary focus will be on developing and implementing optimal solutions for these purposes, then maintaining and monitoring them. You will also be responsible for integrating them with the architecture used across the company.
DUTIES
- Selecting and integrating any Big Data tools and frameworks required to provide requested capabilities
- Implementing ETL process
- Industrialise Code produced by Data Scientists
- Monitoring performance and advising any necessary infrastructure changes
- Defining and implementing data retention policies
SKILLS & QUALIFICATIONS
- Strong experience with BigData tools: Hadoop, Spark, HDFS, Hive
- Experience with the Following languages: Scala, Java, PySpark
- Experience Implementing Consumers and Producers for Data Streaming Tools, in particular Kafka, and Amazon Kinesis
- Experience with NoSQL databases, such as Cassandra, HBase, MongoDB
- Experience with Hortonworks HDP, Amazon EMR
- Experience with integration of data from multiple data sources
- Experience in building or integrating Monitoring Tools ( Kibana, LogStash, Grafana, Prometheus)
- Knowledge and experience with Amazon AWS platform
- Knowledge of various ETL techniques and frameworks.
PERSON SPECIFICATION
- Understand new technologies and stay up to date with latest technologies
- Data oriented person, attentive to details.
- Excellent analysis, testing and troubleshooting skills
- Highly motivated and project orientated
- Ability to facilitate discussions to resolve conflicting processes / content / opinions
- Ability to work under own initiative and prioritise workload efficiently
- Excellent communication both verbal and presentation
Welcome experience
- Knowledge of advertising industry, and possibly Adtech.
by via developer jobs - Stack Overflow
No comments:
Post a Comment