The Big Data Engineer builds data pipelines and models for data analytics applications in accordance with Business, Data Scientists and Database Administrators needs. He/She analyzes structural requirements for new software and applications to deliver on data and information systems functionality. The Data Engineer designs complex information systems and ensures structural requirements of new solutions by building data models and pipelines, at the same time taking care of data security practices and data standards.
Duties include:
Designs and builds conceptual and logical data models and flowcharts which conform to existing standards and conventions
Analyzes structural requirements and builds data pipeline and models for new software and data analytics applications
Migrates data from legacy systems to new solutions
The Data Engineer operates data cleaning as a preliminary stage of analytics
Maps data sources, documents data architecture, data interfaces, and data movement to have an accurate view of the larger data picture
Contributes to end-to-end data lifecycle management activities
Evaluation and recommendation of new data management and storage technologies/standards
Partners with security specialists to ensure compliance with data security and privacy mandates
Builds and revises data dictionary definitions, governance practices, and standards
Professional Skills:
Work experience as Solution Architect, Data Scientist, Data Analyst or similar role
Advanced knowledge in data modelling, process modelling, master data management, metadata management and enterprise data management
Knowledge of data mining and segmentation techniques
Proficiency in database structure principles and Relational DBMS with 3rd normal form relational modelling and dimensional modelling
Proficiency in data warehousing design, tuning and ETL/ELT process development
Experience with Business Intelligence tools like Business Objects, MS Power BI, BOBJ, Tableau, etc.
Advanced knowledge in at least one programming languages like Python, C++, Java, Ruby, etc.
Knowledge of Cloud Computing
Knowledge of Machine Learning, Big Data and Data Science technologies
Good Problem Solving Skills
Communication and presentation skills
Teamwork
by via developer jobs - Stack Overflow
No comments:
Post a Comment