Data Engineer at City of Philadelphia (Philadelphia, PA)

Job DescriptionData Engineering at the City is an adventure: navigating a diverse technology portfolio, understanding what the data represents and its various caveats, removing sensitive information, and presenting it to the public in a way thats easily understood.We have begun building data pipelines using Apache Airflow along with python based, in-house open source software that run on cloud infrastructure, and the Data Engineer will help expand or refactor this work. These pipelines connect various systems of record (business systems and semi-structured data), centralized Oracle and Postgres based RDBMS that we call Databridge, and SaaS-based environments including ArcGIS Online and Carto.Beyond being a key asset to enterprise integration and open data initiatives, the Data Engineer will help develop and support the Citys Address Information System a cloud-based, address search API leveraging Databridge that enables developers to build applications that pull a myriad of the Citys information.Essential FunctionsHelp City departments share data with other departments and with the public by:Working with business partners of varying technical ability to understand how their data is produced, stored and updatedSetting up automated extract, transform, and load (ETL) workflows, including standardization and enriching of datasets that pass through DatabridgeImproving the Citys platform used to share data (reusable components, scheduling, logging, centralized storage, etc.) so that workflows are easier to write and maintainWrite descriptive and accurate technical documentation of systems or applicationsOther duties as assignedAbility ToCompetencies, Knowledge, Skills and AbilitiesWrite clearly and concisely with organized structureReason about, and work, with data from across a variety of potentially unfamiliar domainsWrite descriptive, technical policies and/or documentation of ETL workflowsCommunicate technical nuances in plain language with partners of varying technical backgroundApply creative problem-solving when considering how to best address technical challengesAdvance several workflows at a time for different department datasetsKnowledge OfPython, with demonstrated experience in ETL SQLDatabases, Oracle/Postgres preferredDockerAbility to maintain clean and secure data environmentsWorking in a hybrid environment (on-premises and Amazon Web Services)GitHubMinimal Linux server administration QualificationsAt least 3 years experience in any of the following areas preferred:Developing softwareData integrationArchitecting and maintaining relational databasesMoving a variety of data types between a variety of systems (ie. Flat file, FTP server, AWS S3, Relational Databases, APIs)Transforming and enriching data (ie. De-normalizing, geocoding, re-projecting, anonymizing)Establishing ETL workflows, processes, and documentation for organizationsCompletion of a Bachelors Degree program at an accredited college or university, which has included major course work in computer science, information science, geography, GIS or a closely related field.A Masters Degree in computer science, information science, geography, GIS or a closely related field can substitute for one year of professional experience.Education requirements can be waived by the hiring committee for applicants that are found to be exceptionally strong in the technical areas described above.Additional InformationPlease submit a resume and cover letter with your application.Salary Range: $65,000 - $75,000The successful candidate must reside in the City of Philadelphia or establish primary residence in the City of Philadelphia within 6 months of employmentThe City of Philadelphia is an Equal Opportunity employer and does not permit discrimination based on race, ethnicity, color, sex, sexual orientation, gender identity, religion, national origin, ancestry, age, disability, marital status, source of income, familial status, genetic information or domestic or sexual violence victim status. If you believe you were discriminated against, call the Philadelphia Commission on Human Relations at 215-686-4670For more information, go to: Human Relations Website: http://bit.ly/2Kfpnwr
by via developer jobs - Stack Overflow