Hadoop ETL ELT Pyspark Specialist (Jobs in Columbus, OH)   Requirement id 119518 Job title Specialist Job location in Columbus, OH Skills required Unix Shell Scripting, Hadoop, ETL, ELT Pyspark Open Date 30-Sep-2021 Close Date Job type Contract Duration 9 Months Compensation DOE Status requirement --- Job interview type --- Apply    Email Recruiter: coolsoft Job Description Specialist: Unix Shell Scripting, Hadoop, ETL, ELT Pyspark Description: Responsibilities Our client in Columbus Ohio is seeking an Enterprise Data Warehouse / ETL Developer. Local candidates are highly preferred. Candidates are required to report onsite on Day 1 of the assignment. The Technical Specialist will be responsible for the clients Enterprise data warehouse design, development, implementation, migration, maintenance and operation activities. Works closely with Data Governance and Analytics team. The candidate will closely with Data Governance and Analytics team. Will be one of the key technical resource for data warehouse projects for various Enterprise data warehouse projects and building critical data marts, data ingestion to Big Data platform for data analytics and exchange with state and client partners. This position is a member of the clients ITS and works closely with the Business Intelligence & Data Analytics team. Objectives: Participate in Team activities, Design discussions, Stand up meetings and planning Review with team. Perform data analysis, data profiling, data quality and data ingestion in various layers using big data/Hadoop/Hive/Impala queries, PySpark programs and UNIX shell scripts. Follow the organization coding standard document, Create mappings, sessions and workflows as per the mapping specification document. Perform Gap and impact analysis of ETL and IOP jobs for the new requirement and enhancements. Create jobs in Hadoop using SQOOP, PYSPARK and Stream Sets to meet the business user needs.Perform data analysis, data profiling, data quality and data ingestion in various layers using big data/Hadoop/Hive/Impala queries, PySpark programs and UNIX shell scripts. Create mockup data, perform Unit testing and capture the result sets against the jobs developed in lower environment. Updating the production support Run book, Control M schedule document as per the production release. Create and update design documents, provide detail description about workflows after every production release. Continuously monitor the production data loads, fix the issues, update the tracker document with the issues, Identify the performance issues. Performance tuning long running ETL/ELT jobs by creating partitions, enabling full load and other standard approaches. Perform Quality assurance check, Reconciliation post data loads and communicate to vendor for receiving fixed data. Participate in ETL/ELT code review and design re-usable frameworks. Create Remedy/Service Now tickets to fix production issues, create Support Requests to deploy Database, Hadoop, Hive, Impala, UNIX, ETL/ELT and SAS code to UAT environment. Create Remedy/Service Now tickets and/or incidents to trigger Control M jobs for FTP and ETL/ELT jobs on ADHOC, daily, weekly, Monthly and quarterly basis as needed. Model and create STAGE / ODS / Data warehouse Hive and Impala tables as and when needed. Create Change requests, workplan, Test results, BCAB checklist documents for the code deployment to production environment and perform the code validation post deployment. Work with Hadoop Admin, ETL and SAS admin teams for code deployments and health checks. Create re-usable UNIX shell scripts for file archival, file validations and Hadoop workflow looping. Create re-usable framework for Audit Balance Control to capture Reconciliation, mapping parameters and variables, serves as single point of reference for workflows. Create PySpark programs to ingest historical and incremental data. Create SQOOP scripts to ingest historical data from EDW oracle database to Hadoop IOP, created HIVE tables and Impala views creation scripts for Dimension tables. Participate in meetings to continuously upgrade the Functional and technical expertise. Experience required   Call 502-379-4456 Ext 100 for more details. Please provide Requirement id: 119518 while calling.   Other jobs in OH: Cincinnati (1), Cleveland (1), Columbus (233), Dublin (8), Solon (2), Unix Shell Scripting job openings in Columbus, OH Jobs List Enterprise Data Warehouse / Big Data Developer - 58686 Create date: 26-Oct-2021 Description: Responsibilities Our client in Columbus Ohio is seeking an Enterprise Data Warehouse / Big Data Developer. Local candidates are highly preferred. Candidates are required to report onsite on Day 1 of the assignment. The Technical Specialist will be responsible for the clients Enterprise data w.... (This job is for - ETLELT Jobs in OH Columbus Developer - (in Columbus, OH)) Informatica Developer Create date: 14-May-2019 Description: Informatica, Unix Shell/Perl Scripting, SQL anywhere, Databases - Atleast 2 of Oracle, MS SQL Server, Teradata (This job is for - job Jobs in OH Columbus Developer - (in Columbus, OH)) Oracle DB Developers Create date: 17-Oct-2018 Description: I have immediate need for the the following roles at Nationwide in Columbus. I have attached the job descriptions, though some are vague. I have included the PL/SQL jd below and another jd is attached. PL/SQL Developer. We have an immediate contract need in Columbus, Ohio for a PL/SQL Developer with the following.... (This job is for - job Jobs in OH Columbus Developer - (in Columbus, OH)) Java Web / J2EE Developer - 8516 Create date: 22-Jan-2018 An experienced contractor is needed in Columbus, Ohio to fulfill the role of Java Web / J2EE Developer with our direct client. The position requires the ability to develop, code and unit test within stated technologies on an Agile line. The line services other developer lines and client vendors with B2B services so great customer service is requi.... (This job is for - job Jobs in OH Columbus Developer - (in Columbus, OH)) Java Developer - 7963 Create date: 02-Nov-2017 Requirements: 5+ years Java Unix Shell Scripting (hands-on experience required) SQL (This job is for - Java Jobs in OH Columbus Developer - (in Columbus, OH))    Unix Shell Scripting job openings in other states Jobs List Expert IT Specialist - 7000 Create date: 20-Jul-2022 Description: Qualifications IT Specialist - Works in the field of Information Technology, experienced in the usage and support of a collection of development platforms, technical architectures or business applications and products that run on those platforms - beyond that of a Programmer. Expert 84+ months - Candi.... (This job is for - ETL Jobs in NY Albany Specialist - (in Albany, NY)) Expert Software Architect - 6877 Create date: 22-Jun-2022 Description: Qualifications Software Architect - Makes high-level design choices and dictates technical standards, including software coding standards, tools, and platforms. Designs multi-leveled architecture or component interactions of a large scale software system. Expert - 84+ Months: Candidate is able to prov.... (This job is for - job Jobs in NY Albany Architect - (in Albany, NY)) Expert IT Specialist - 6512 Create date: 04-May-2022 Description: Day-to-Day Tasks include, but not limited to: Capture and load source system data via ETL processes into the data mart. Develop, test and implement program logic for ETL development. Efficiently implement team developed ETL technical specifications. Optimize ETL data load jobs to imp.... (This job is for - ETL Jobs in NY Albany Specialist - (in Albany, NY)) Unix Admin - 71357 Create date: 29-Jan-2021 Note: This is a markup account, please list consultants W2 hourly pay rate in the comments section when submitting. Upon offer, it is required to show consultants W2 take home pay in the Exh A. Candidate must be our W2 Employee. We should submit only GC,GC - EAD, L2-EAD,H4-EAD and Citizens. Job Description: Data Eng.... (This job is for - RDBMS ETL Jobs in NC Charlotte Administrator - (in Charlotte, NC)) Teradata Developer - 71126 Create date: 22-Jan-2021 Candidate must be our W2 Employee. Job Description: Senior Database/ETL Developer Hands-on experience and deep understanding (5+ years) of data transformation using SQL, Stored procedures, triggers, UDFs using Teradata Experience in data warehousing architecture and concept (e.g. 3NF, Dimensional model, Snowf.... (This job is for - Automation Jobs in NC Charlotte Developer - (in Charlotte, NC))    Unix Shell Scripting job openings in OH Jobs List 4GL Developer - 47356 Create date: 16-May-2019 Notes : Candidate must be on your direct W2 or a transfer. Sub-vending is not allowed for this role Description Role Description: 4GL Developer Skillset Requirements: SQL, Informix 4-GL, Informix-SPL, UNIX Shell Essential Job Functions: Critical thinking skills to unders.... (This job is for - ODBC Jobs in OH BlueAsh Developer - (in BLUE ASH, OH)) J-10-632-769 - Oracle Developer Create date: 05-Dec-2011 [Note : Start Date : 01-02-2012 End Date : 12-31-2012] Description : Responsibilities : This individual needs to be proficient in multiple areas of application development, business analysis, and system administration. They would be accountable for project implementation or single dimensional business functio.... (This job is for - Java Jobs in OH CLEVELAND Developer - (in Cleveland, OH)) (Specialist: Unix Shell Scripting, Hadoop, ETL, ELT Pyspark in Columbus, OH)

Featured Jobs VITA Manager (Jobs in Richmond, Virginia) AutoCad - Consultant (Jobs in Chicago, Illinois) outlook SharePoint MS Office Troubleshoot Analyst (Jobs in Colonial Heights NIST Cyber Architect (Jobs in Lansing, Michigan) Jira Analyst (Jobs in Richmond, Virginia) Analyst (Jobs in Columbia, South Carolina) - Consultant (Jobs in Columbia, South Carolina) Advanced Analyst (Jobs in Richmond, Virginia) JavaScript C# Developer (Jobs in Madison, Wisconsin) SAML Specialist (Jobs in Raleigh, North Carolina)