Data Engineer at The Southwind Group (Boston, MA)

Job Description

We’re looking for a Data Engineer, with data ingestion experience using Python.

Responsibilities:
•    Be an independent thinker, leader, and problem solver.
•    Own the data you are getting and make it load into the system smoothly and efficiently.
•    Build your ingestion code in Python and support your quality with unit-tests and integration-tests.
•    Find the golden line for your ingestion code to be generic and able to easily accommodate new data structures in the future, yet execute in an efficient way that run fast.
•    Negotiate with customers about their data in cases where the data is missing or distorted. Nevertheless, be creative and seek for ways to curate the data and use it at best as possible even with distortion.

Your Experience and Know-How:
•    Bachelor’s degree in Computer Science or equivalent
•    5+ years of development experience
•    3+ years writing Python code for data ingestion or data processing at large scale
•    Experience with Pandas is an advantage
•    Experience with multithreading/multiprocessing in Python
•    Experience with unit-testing methods using Python
•    Experience with data formats: CSV, JSON, XML and how to validate and parse them
•    Familiarity with ETL and data mapping concepts
•    Experience with Docker. Using Python in a Docker – is an advantage
•    Experience with SQL databases
•    Experience with AWS, such as S3 is an advantage
•    Familiarity of dev environments in Linux and Linux scripting
•    Knowledge of common practices GIT, JIRA, or similar
•    Understanding of engineering best practices
•    Understanding of agile methodologies
•    Must have some experience writing in Java

by via developer jobs - Stack Overflow