Senior Platform Engineer

Notes: Mariner Finance is shifting from Red Hat to Databricks, so they need a Platform Engineer with strong experience in Databricks to help with this migration. This process is starting from scratch, so they need someone to have experience with Databricks implementation. They ideally need someone who has implemented Databricks in AWS. Azure experience is okay, but AWS experience is what they are looking for. This Platform Engineer will join 2 other internal Engineers in this role, but the existing Engineers are not strong in Databricks, so they need this contractor to be the lead Databricks SME. They would love someone with a Financial or Medical data background.

Summary:

• Design, implement, maintain, and support the company's enterprise data platform and the supporting infrastructure, integrations, and services required to deliver reliable, secure, scalable, and cost-effective data capabilities.
• Partner with data engineering, infrastructure, security, application development, architecture, and business technology teams to support data ingestion, transformation, storage, analytics, integration, governance, and operational data requirements across cloud, on-premises, SaaS, and third-party environments.
• Serve as a technical resource for data platform implementation, connectivity, automation, security, performance, reliability, and modernization initiatives.

Responsibilities:

• Design, implement, configure, and maintain enterprise data platforms and supporting cloud infrastructure.
• Support data warehouse, data lake, lakehouse, operational data, integration, and analytical workloads.
• Implement secure connectivity between the enterprise data platform and on-premises systems, cloud environments, vendor platforms, SaaS applications, relational databases, APIs, and file-transfer services.
• Build and maintain platform integrations across databases, object storage, APIs, SFTP, enterprise applications, and third-party data sources.
• Work with infrastructure and networking teams to implement secure cloud and hybrid connectivity patterns.
• Configure and support platform identity, access controls, service accounts, roles, groups, secrets, authentication, encryption, and least-privilege security standards.
• Develop and maintain automated infrastructure and platform deployment processes using infrastructure-as-code and CI/CD technologies.
• Support Git-based source control, release management, environment promotion, and deployment standards for data-platform components.
• Partner with data engineers to implement scalable ingestion and transformation patterns for batch, near-real-time, streaming, API, database, and file-based workloads.
• Support orchestration and scheduling of data pipelines, platform jobs, integrations, and dependent services.
• Configure and manage platform compute, storage, workload isolation, scaling, scheduling, concurrency, resource governance, and cost controls.
• Monitor platform availability, infrastructure health, job execution, query performance, compute utilization, storage consumption, connectivity, and integration health.
• Implement logging, monitoring, alerting, auditing, operational dashboards, and support procedures.
• Troubleshoot data-platform, infrastructure, network, security, performance, and integration issues.
• Support platform upgrades, configuration changes, infrastructure changes, release deployments, and production support.
• Assist with migration of existing warehouse, ETL, database, API, and file-processing workloads into modern data-platform architectures.
• Evaluate workload requirements and help determine appropriate placement across data warehouse, lakehouse, relational database, cloud service, integration, and application platforms.
• Develop reusable engineering patterns, deployment templates, technical standards, and operational documentation.
• Support proof-of-concept environments, platform evaluations, workload testing, performance testing, and migration assessments.
• Collaborate with security, risk, infrastructure, architecture, application, and data teams to incorporate security, availability, resiliency, and compliance requirements into platform implementations.
• Work with internal teams and external vendors to resolve platform, connectivity, integration, and performance issues.
• Maintain technical documentation covering platform architecture, environments, integrations, network connectivity, security controls, operational procedures, deployment processes, and dependencies.
• Identify opportunities for platform simplification, automation, modernization, standardization, reuse, and cost optimization.
• Share technical knowledge and provide guidance to engineers and project teams supporting the enterprise data platform.