Data Engineer at Rakuten (Tokyo, Japan)

Responsibilities

Designing and building data science platform with selecting various big data processing tools and frameworks
Implementing ETL process of huge structured/unstructured data sets in various data platforms
Solving business or development problems by proposing and implement solution with the latest data science technologies
Developing and maintaining production level data science products with other data science engineers and data scientists
Creating insight reports or implementing visualized report system

Minimum Qualifications

Computer science or related background
5+ years’ experience in software development, especially using Python as a programming language
Experience with common Linux commands and Linux scripting languages
Experience with Hadoop, MapReduce, HDFS and Big Data querying tools, such as Tez, Hive, and Impala
Experience with designing and building tools based on specific requirements
Experience with building and maintaining data science platform
Experience with SQL and some of the modern relational databases
Ability to talk with Product Manager, Data Scientist to gather the requirements

Preferred Qualifications

Able to work in a team-oriented environment
Experiences in web service development in multiply industries.
Solid knowledge of large volumes data processing
Experience with Big Data ML toolkits, such as Mahout, SparkML, or H2O
Experience with NoSQL databases, such as HBase, Redis, CouchBase
Familiar with data mining concepts and machine learning algorithms
Experience with Spark and stream-processing systems, using solutions such as Storm or Spark-Streaming
Knowledge of various ETL techniques and frameworks, such as Flume

by via developer jobs - Stack Overflow

Placement papers | Freshers Walkin | Jobs daily