Big Data Architect (Closed)
SkillStorm is seeking a Big Data Architect for our client in Addison, Texas. Candidates must be able to work on SkillStorm's W2; not a C2C position. EOE, including disability/vets.
Required Skills:
- Experience in HDFS, Map Reduce, Hive, Impala, Linux/Unix
- Experience creating Data Pipeline with ETL Processes & Frameworks supporting Business Intelligence
- Hands on experience on any RDBMS (Oracle, SQL Server, DB2, etc.) technologies
- Troubleshoot and identify technical problems in applications processes and provide solutions
- Experience in Hive Partitioning, Bucketing to improve query performance along with knowledge of data warehousing principles supporting Business Intelligence
- Understands of security standards in Hadoop
- Familiarity with Enterprise Data Management Standards
- Create, execute and document the tests necessary to ensure that an application meets performance requirements
- Sound understanding and experience with Hadoop ecosystem (Cloudera). Able to understand and explore the constantly evolving tools within Hadoop ecosystem and apply them appropriately to the relevant problems at hand
- Experience in Unix shell and able to analyze the existing shell scripts
- Exposure to Autosys Scheduling and write JIL scripts
- Provide technical resources to assist in the design, testing and implementation of software code and infrastructure to support data infrastructure and governance activities
Desired Skills:
- Experience in working with a Big Data implementation in production environment
- Experience in Kafka/Flume/Spark is an added advantage
- Exposure to ETL tools e.g. data stage, Sqoop
- Able to analyze the existing shell scripts/python/Perl code to debug any issues or enhance the code
- Assist with the technical design/architecture and implementation of the big data cluster in various environments
- Utilities/libraries that can be reused in multiple big data development efforts
- Work with line of business (LOB) personnel and internal Data Services team to develop system specifications in compliance with corporate standards for architecture adherence and performance guidelines
- Previous experience in the financial services industry
- Broad Client technical experience and good understanding of existing testing/operational processes and an open mind on how to enhance those
- Understanding of industry trends and relevant application technologies
- Experience in designing and implementing analytical environments and business intelligence solutions
- Microsoft Visio, UML tools, Agile
- Handles multiple assignments with multiple deadlines simultaneously
#LI-DNI