Application Programmer V (Closed)
SkillStorm is seeking a Big Data Developer for our client in Toronto, Ontario. Candidates must be able to work on SkillStorm's W2; not a C2C position. EOE, including disability/vets.
Description:
- The position is for a senior big data developer to work in the Client’s Prime Brokerage technology group. The successful candidate will join an established agile team to work on new applications.
Qualifications:
- At least 7+ years of experience in working with big data technologies like Hive, Hadoop, Impala, Hbase
- Experience in real-time messaging solutions like Kafka, Amps etc
- Working knowledge of data serialization and data exchange services like Avro.
- Experience with relational SQL and NoSQL databases: Oracle, Hive, Impala
- Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL)
- Experience building and optimizing ‘big data’ data pipelines, architectures and data sets.
- Experience performing root cause analysis on data and processes to answer specific business questions and identify opportunities for improvement.
- Strong analytic skills related to working with unstructured datasets and datasets in JSON,XML format , parquet etc
- Build processes supporting data transformation, data structures, metadata, dependency and workload management.
- Experience supporting and working with cross-functional teams in a dynamic environment.
Responsibilities:
- Ensuring stability by designing, tuning, monitoring and troubleshooting databases with large amounts of data (Terabytes) and high transaction rates
- Support IT initiatives through sql analysis, schema revisions, and capacity forecasting
- Homogenize the performance of the database and optimize system resources
- Support efforts related to the proper protection from corruption & encryption of sensitive information
- Perform spike/proof of concept as necessary to mitigate risk or implement new ideas.
- Performing root cause analysis on data and processes to answer specific business questions and identify opportunities for improvement.
- Coordinating with offshore developers
- Contribute to story refinement/defining requirements.
- Code solutions and unit tests to deliver a requirement/story per the defined acceptance criteria and compliance requirements.
- Assists team with resolving technical complexities involved in realizing story work.
- Design/develop/modify architecture components, application interfaces, and solution enablers while ensuring principal architecture integrity is maintained.
- Design/develop/maintain automated test suites (integration, regression, performance).
- Set up and develops a continuous integration/continuous delivery pipeline.
- Mentor other Software Engineers and coach team on big data technologies.
Good to Have:
• Knowledge about Cloudera platform
- Big data
- Impala
- HDFS
- SQL
- Hive
- 7-10years
#LI-DNI