Python Pyspark Scala Shell Scripting Spark BASH GitHub ETL Tools Analytical Skills Ex   Requirement id 97903 Job title Engineer Job location in Mason, OH Skills required Python, Pyspark, Scala, Shell Scripting Spark BASH GitHub ETL To Open Date 18-Sep-2019 Close Date Job type Direct Hire Duration 6 Months Compensation DOE Status requirement --- Job interview type --- Apply    Email Recruiter: coolsoft Job Description Engineer: Python, Pyspark, Scala, Shell Scripting Spark BASH GitHub ETL To Job Description Job Summary: Responsible for construction and development of "large-scale hadoop data processing systems" The Big Data Engineer must have considerable expertise in data warehousing and NoSQL technologies. Requires coding expertise with Python, Scala, and hive SQL languages. Must be able to implement enterprise big data architecture designs, and will work closely with the rest of the Analytics team, IT, and internal business partners to identify, evaluate, design, and implement big data solutions, structured and unstructured, public and proprietary data. The Big Data Engineer will work iteratively on the distributed Hadoop platform to deliver data solutions to stakeholders. Responsibilities: • Experience as Hadoop Developer with sound knowledge in Hadoop ecosystem technologies. • Hands on experience in developing and deploying enterprise based applications using major components in Hadoop ecosystem like Hadoop, YARN, Hive, Map Reduce, HBase, Flume, Scoop, Spark, Kafka, Oozie and Zookeeper. • Excellent Programming skills at a higher level of abstraction using Scala and Spark. • Hands on experience in Importing and exporting data from different databases like SQL, Oracle, Teradata into HDFS using Sqoop • Strong experience working with real time streaming applications and batch style large scale distributed computing applications using tools like Spark, Kafka, Flume, MapReduce, Hive. • Design Extract, Transform, Load (ETL) processes that populate HIVE databases. Population often includes data from multiple sources. ETL design frequently requires building HIVE tables, performing data conversion, creating calculated fields, designing data update routines, and building scheduled jobs. • Managing and scheduling batch Jobs on a Hadoop Cluster using Oozie. • Experience in managing and reviewing Hadoop Log files. • Experienced using Sqoop to import data into HDFS from RDBMS and vice-versa. • Hands on experience in Analysis, Design, Coding and Testing phases of Software Development Life Cycle (SDLC). • Ability to work with different file formats like Avro, Parquet, and JSON. • Ability to advise management on approaches to optimize for big data platform success. • Understands database schemas and data flow diagrams. • Demonstrated success documenting technical specifications. • Identifies, analyzes, and troubleshoots problems in big data solutions. • Understands source systems and the common business and technical keys • Reviews and evaluates data output to ensure it conforms to technical requirements • Collaborate in the planning, design, deployment, and testing of new or enhanced solutions. • Ability to create detailed test plans for new development and maintain regression testing for existing functionality. • Focused on building infrastructure and architecture for big data generation • Able to effectively communicate highly technical information to numerous audiences, including management, the user community, and less-experienced staff. • Able to Collaborate with Product Managers, Operations, and Data Architecture to produce the best possible end products • Consistently communicate on status of project deliverables • Consistently provide work effort estimates to management to assist in setting priorities • Deliver timely work in accordance with estimates • Solve problems as they arise and communicate potential roadblocks to manage expectations • Adhere strictly to all security policies Qualifications Education: Master level engineering degree preferred Bachelors Level engineering degree required Work Experience: • Four to seven years of experience preferred. • Minimum number years of relevant experience: 2 Years • Some of the minimum experience requirement may be met with Mas   Call 502-379-4456 Ext 100 for more details. Please provide Requirement id: 97903 while calling.   Other jobs in OH: Cincinnati (1), Cleveland (1), Columbus (233), Dublin (8), Solon (2), Python job openings in Mason, OH Jobs List NLP Developer - 50499 Create date: 31-Jul-2019 Candidate must be our W2 Employee Description: Project • Build an NLP engine to extract textual content from claims records and process content into digital format for storage. Data will be stored in eHub where it will be standardized for consumption by Anthem customers Required Skills .... (This job is for - PythonAPI Jobs in OH Mason Developer - (in Mason, OH)) NLP Developer - 46590 Create date: 29-Apr-2019 Job Description : NLP Developer Project • Build an NLP engine to extract textual content from claims records and process content into digital format for storage. Data will be stored in eHub where it will be standardized for consumption by Anthem customers Required Skills • Experience with NLP (.... (This job is for - Python Jobs in OH Mason Developer - (in Mason, OH))    Python job openings in other states Jobs List Agentic Data Engineer(766844) Create date: 08-Jul-2025 Start Date :7/21/2025 End Date :12/31/2025 Submission Deadline :7/11/2025 Client Info: VDOT Note : * Agency Interview Type : Both Phone and In Person * Hybrid Description : The Virginia Department of Transportations Information T.... (This job is for - PythonSpark Jobs in VA Richmond Engineer - (in Richmond, VA)) Elect - Enterprise Architect 4 (765006) Create date: 03-Jun-2025 Start date : 06/16/2025 End Date : 12/31/2025 Submission deadline : 06/09/2025 Client Info : ELECT Note: * Agency Interview Type : Web Cam Interview Only * Hybrid Description : The position will act as the Technical Lead for an established development team that.... (This job is for - Azure Python JavaScript Jobs in VA Richmond Architect - (in Richmond, VA)) Elect - Cloud Developer 3 (761296) Create date: 07-Apr-2025 Remote Start date : 04/21/2025 End Date : 12/31/2025 Submission deadline : 04/14/2025 Client Info : ELECT Note: * Agency Interview Type: Web Cam Interview Only * The Virginia Department of Elections (ELECT) is seeking an Azure Developer with minimum 3 years experience to w.... (This job is for - Python CI CD Jobs in VA Richmond Developer - (in Richmond, VA)) System Administrator - 9216402 Create date: 21-Mar-2025 Start Date :4/1/2025 End Date :06/30/2025 Submission Deadline :3/26/2025 Client Info: CO OIT Note: * Remote Position * Candidate must be Local Description : A DMV check will also need to be completed with the standard background check by the.... (This job is for - Python DNS Jobs in CO Denver Administrator - (in Denver, CO)) VSP - IT Security Analyst 4 (760300) Create date: 20-Mar-2025 Start date : 03/31/2025 End Date : 09/30/2025 Submission deadline : 03/26/2025 Client Info : VSP Note: * Agency Interview Type : In Person Only * Hybrid Description : ·Analyze the securityimpact of application, configuration, and infrastructure changes to ensur.... (This job is for - PythonCJIS Jobs in VA Richmond Analyst - (in Richmond, VA))    Python job openings in OH Jobs List Business Analyst (IT) III Senior - 20230407090751 Create date: 07-Apr-2023 Start date :4/28/2023 End Date :8+Months from the start date Submission deadline :04-14-2023 Note: * Job Site On-Site Description : Huntington Bank is seeking a passionate, data-savvy Analyst to join the Enterprise Analytics team to fuel our mission of growth through data-driven insights and opp.... (This job is for - Python Jobs in OH Columbus Analyst - (in Columbus, OH)) Business Analyst (IT) - III (Senior) - 20230322044121 Create date: 23-Mar-2023 Start date :4/12/2023 End Date : 3+months from the start date Submission deadline :03-29-2023 Note : Job Site : On-Site Description : ------------------------------------------------------ Huntington Bank is seeking a passionate, data savvy Analyst to join the Analytics team and fuel .... (This job is for - Python Jobs in OH Columbus Analyst - (in Columbus, OH)) Business Analyst (IT) III (Senior) - 20221201013721 Create date: 01-Dec-2022 Start date :12/19/2022 End Date : 3+months From the Start Date Submission deadline :12/08/2022 Job Site : On-Site Description : --------------------------------------------- Huntington Bank is seeking a passionate, data savvy Analyst to join the Analytics team and fuel our mission of growth thro.... (This job is for - Python Jobs in OH Columbus Analyst - (in Columbus, OH)) Python Developer - 73349 Create date: 25-Mar-2021 Candidate must be our W2 Employee. Job Description: We are looking for a Python Web Developer responsible for managing the interchange of data between the server and the users. Your primary focus will be the development of all server-side logic, ensuring high performance and responsiveness to requests from the front-end. You w.... (This job is for - Python JavaScript Jobs in OH Findlay Developer - (in Findlay, OH)) Data Scientist (IT Security) - 72915 Create date: 12-Mar-2021 Note: This person will be helping to lead a newer team at AEP with the goal of analyzing data to predict any security threats or vulnerabilities. Long term, open-ended contract East coast hours. Fully remote. WILL START IN MARCH - lets go. Job Description: Data Scientist Location: Fully remote. Client is in Columbus, OH .... (This job is for - Python Hadoop Jobs in OH Columbus Consultant - (in Columbus, OH)) (Engineer: Python, Pyspark, Scala, Shell Scripting Spark BASH GitHub ETL To in Mason, OH)

Featured Jobs outlook MS Office Active Analyst (Jobs in Conyers, Georgia) JAVA Analyst (Jobs in Atlanta, Georgia) Requirements Analyst (Jobs in Richmond, Virginia) Sharepoint Analyst (Jobs in Pontiac, Michigan) VPN Firewall RBAC Problem Engineer (Jobs in Raleigh, North Carolina) Spring Consultant (Jobs in Lansing, Michigan) JSON API Developer (Jobs in Raleigh, North Carolina) SharePoint Chatbots Developer (Jobs in Raleigh, North Carolina) Automation JavaScript Python Analyst (Jobs in Gllen Allen, Virginia) SCCM - Consultant (Jobs in Charlotte, North Carolina)