Big Data Engineer (Closed)

SkillStorm is seeking a Big Data Engineer for our client in Charlotte, NC. Candidates must be able to work on SkillStorm's W2; not a C2C position. EOE, including disability/vets.

Job Description:

• Data Platform Services OMNI initiative is looking for a Big Data Senior Engineer who will be responsible for converting Teradata applications to Hadoop. The ideal candidate is someone who has hands-on experience on Teradata and Hadoop environments. The candidate will be responsible for all aspects of SDLC and work in an agile environment hand in hand with the conversion team on day to day tasks.
• Experience in application development on Teradata/Hadoop/HDFS/SPARK/Impala/Unix and ability to write shell scripts, Spark Dataset, Data frames, HiveQL jobs.
• Strong knowledge of Teradata/Hadoop architectural concepts and data modelling.
• Familiarity with data loading tools like Sqoop, Flume and Kafka. Knowledge of workflow/schedulers like Oozie/Autosys.
• Good aptitude in multi-threading ,concurrency concepts and loading data from disparate data source sets.
• Ability to identify and analyze issues with the existing Teradata and Hadoop application designs/platform and suggest architectural design changes/optimizations.
• Ability/knowledge to implement Data Management, privacy and governance on the Hadoop Platform.
• Excellent analytical skills to understand business process flows and get an end to end understanding of the Teradata applications.
• Work with minimal supervision and possess the ability to coordinate competing priorities.

Primary Skill:

• Teradata

Secondary Skill:

• Hadoop

Required Skills:

• Teradata, Hadoop-Spark, Hive on Tez, Hive 3.x, Impala,Unix, Shell Scripting, Autosys

Desired Skills:

• Java, Python, Scala, Py-Spark ,Kafka ,HBASE ,Oozie, Flume,Machine Learning, Azure Synapse

#LI-DNI