Spark Graph DB Python Scripting Engineer (Jobs in Richmond, VA)   Requirement id 153297 Job title Engineer Job location in Richmond, VA Skills required Developing ETL Processes, Azure Databricks, Spark, Graph DB Python Scripting Open Date 06-May-2025 Close Date Job type Contract Duration 1 Months Compensation DOE Status requirement --- Job interview type --- Apply    Email Recruiter: coolsoft Job Description Engineer: Developing ETL Processes, Azure Databricks, Spark, Graph DB Python Scripting Start date : 05/19/2025 End Date : 06/30/2025 Submission deadline : 5/9/2025 Client Info : VDOT Note: * Agency Interview Type : Both Phone and In Person * Hybrid Description : Agentic Data Engineer to design, develop, and deploy data pipelines that leverage agentic AI that solve real-world problems The Virginia Department of Transportations Information Technology Division is seeking a highly skilled Agentic Data Engineer to design, develop, and deploy data pipelines that leverage agentic AI that solve real-world problems. The ideal candidate will have experience in designing data process to support agentic systems, ensure data quality and facilitating interaction between agents and data.    Responsibilities:  Designing and developing data pipelines for agentic systems, develop Robust data flows to handle complex interactions between AI agents and Data sources. Ability to train and fine tune large language models Design and build the data architecture, including databases, data lakes to support various data engineering tasks. Develop and manage Extract, Load, transform (ELT) processes to ensure data is accurately and efficiently moved from source systems to analytical platforms used in data science. Implement data pipelines that facilitate feedback loops, allowing human input to improve system performance in human-in-the-loop systems. Work with vector databases to store and retrieve embeddings efficiently. Collaborate with data scientists and engineers to preprocess data, train models, and integrate AI into applications. Optimize data storage and retrieval with high performance Statistical analysis, trends, patterns to create data formats from multiple sources.    Qualifications:           Strong Data engineering fundamentals Utilize Big data frameworks like Spark/Databricks Training LLMs with structed and unstructured data sets. Understanding of Graph DB Experience with Azure Blob Storage, Azure Data Lakes, Azure Databricks Experience implementing Azure Machine Learning, Azure Computer Vision, Azure Video Indexer, Azure OpenAI models, Azure Media Services, Azure AI Search  Determine effective data partitioning criteria Utilize data storage system spark to implement partition schemes Understanding core machine learning concepts and algorithms Familiarity with Cloud computing skills Strong programming skills in Python and experience with AI/ML frameworks. Proficiency in vector databases and embedding models for retrieval tasks. Expertise in integrating with AI agent frameworks. Experience with cloud AI services (Azure AI). Experience with GIS spatial data to create markers on maps ( lat long nearest topology of road, geo-locate between datasets, correlation etc.). Experience with Department of Transportation Data Domains developing an AI Composite Agentic Solution designed to identify and analyze data models, connect & correlate information to validate hypotheses, forecast, predict and recommend potential strategies and conduct What-if analysis. Bachelors or masters degree in computer science, AI, Data Science, or a related field.   Call 502-379-4456 Ext 100 for more details. Please provide Requirement id: 153297 while calling.   Other jobs in VA: Alexandria (1), Ashburn (3), Cardinal (1), Charllottesville (1), Charlottesville (6), Chesapeake (2), Chester (2), Dulles (3), Fredericksburg (1), Fremont (1), Glen Allen (2), Reston (4), Richmond (45), Roanoke (14), Virginia Beach (14),    Developing ETL Processes job openings in other states Jobs List Programmer 5-398353 Create date: 06-Feb-2015 Description: BWC is looking for a Programmer 5 to support implementing a new Claims, Policy and Billing system. This new implementation is shifting our Data warehouse and Reporting infrastructure from DB2/Hyperion to Oracle/Cognos. Required Skills: * 10 or more years of experience in design, development, .... (This job is for - job Jobs in OH Columbus Developer - (in Columbus, OH))     (Engineer: Developing ETL Processes, Azure Databricks, Spark, Graph DB Python Scripting in Richmond, VA)

Featured Jobs Sql Server 2000 SSIS Power Developer (Jobs in Richmond, Virginia) Negotiation Analyst (Jobs in Richmond, Virginia) Communication TeamLead (Jobs in Lansing, Michigan) Spark Graph Engineer (Jobs in Richmond, Virginia) OLTP Consultant (Jobs in Columbia, South Carolina) Architect Power Analyst (Jobs in Lansing, Michigan) CICS Analyst (Jobs in Madison, Wisconsin) Production Consultant (Jobs in COLUMBIA, South Carolina) Help Desk MS Office Active Analyst (Jobs in Conyers, Georgia) Azure API Consultant (Jobs in COLUMBIA, South Carolina)