Senior ML Infrastructure Engineer

Full Timesenior

Edison, NJPosted 21 days ago

Role Overview

Orion Innovation is hiring a Senior ML Infrastructure Engineer. This is a full-time role in Edison. posted 3 weeks ago. Full responsibilities, required qualifications, and the apply link are listed in the description below.

Resume Keywords to Include

Make sure these keywords appear in your resume to improve ATS scoring

PythonAzureDockerKubernetesTerraformPyTorchAPISDK

Job description

Orion Innovation is a premier, award-winning, global business and technology services firm. Orion delivers game-changing business transformation and product development rooted in digital strategy, experience design, and engineering, with a unique combination of agility, scale, and maturity. We work with a wide range of clients across many industries including financial services, professional services, telecommunications and media, consumer products, automotive, industrial automation, professional sports and entertainment, life sciences, ecommerce, and education.

Project Overview:

We're building a large-scale document intelligence platform that processes text files up to 5 TB in size, extracts insights using BERT-class NLP models, and surfaces answers to analysts via a low-latency query interface. The platform runs on Azure Kubernetes Service (AKS) with dedicated GPU node pools, uses KEDA for event-driven autoscaling, and integrates with Azure Data Lake Storage Gen2 and Azure OpenAI.

This is a hands-on role that sits at the intersection of platform engineering and applied ML, and requires someone who is equally comfortable debugging a CUDA out-of-memory error and designing a Kubernetes autoscaling policy. As the Senior ML Infrastructure Engineer the resource will own the end-to-end infrastructure layer — from GPU cluster configuration and CUDA runtime management to Kubernetes job orchestration and model serving.

Skill / Technology:

Level: Kubernetes / AKS
Expert: Multi-node-pool design, taint/toleration, autoscaler, GPU node pools (NC/ND series)
Senior: Device plugin, driver compat, resource limits, KEDA
Senior: Scaled Job, queue triggers, cooldown tuning, CUDA / cuDNN
Mid–Senior: Runtime config via PyTorch; raw kernel dev not required, PyTorch (GPU inference)
Senior: Batching, FP16, memory management, profiling, Hugging Face Transformers
Senior: BERT/DistilBERT/BGE loading, pipeline API, tokenization, Python (production)
Senior: Async workers, Azure SDK, queue consumers, Azure infrastructure
Senior: VNet, private endpoints, Key Vault, ADLS, AD, Docker / Helm
Senior: Multi-stage builds, Helm chart authoring, IaC (Terraform / Bicep)
Preferred: willingness to learn is acceptable

Orion is an equal opportunity employer, and all qualified applicants will receive consideration for employment without regard to race, color, creed, religion, sex, sexual orientation, gender identity or expression, pregnancy, age, national origin, citizenship status, disability status, genetic information, protected veteran status, or any other characteristic protected by law.

Candidate Privacy Policy

Orion Systems Integrators, LLC and its subsidiaries and its affiliates (collectively, “Orion,” “we” or “us”) are committed to protecting your privacy. This Candidate Privacy Policy (orioninc.com) (“Notice”) explains:

What information we collect during our application and recruitment process and why we collect it;
How we handle that information; and
How to access and update that information.

Your use of Orion services is governed by any applicable terms in this notice and our general Privacy Policy.

About Orion Innovation

Orion Innovation

orioninc.com

On-site

14 other open roles at Orion Innovation on TryApplyNow.

View all 15 roles at Orion Innovation Visit website

Frequently Asked Questions

How do I apply for the Senior ML Infrastructure Engineer position at Orion Innovation?

Use the Apply button above to submit your application directly to Orion Innovation. Most applications take less than 5 minutes if your resume and contact details are ready, and you'll be routed to the employer's official application system to finish.

Where is the Senior ML Infrastructure Engineer position at Orion Innovation located?

This position is based in Edison. Orion Innovation has not indicated remote or hybrid options for this role, so candidates should plan for on-site work.

What does a Senior ML Infrastructure Engineer at Orion Innovation earn?

Orion Innovation has not disclosed a salary range in this posting. Many employers share specifics later in the interview process; you can also ask during a recruiter screen if compensation transparency is important to you.

When was the Senior ML Infrastructure Engineer role at Orion Innovation posted?

This role was posted on June 30, 2026 (21 days ago). It's still listed as actively hiring; we re-confirm openings against the source system multiple times per day and remove closed roles.

How much experience does the Senior ML Infrastructure Engineer role at Orion Innovation require?

This is a senior-level position. Most senior roles call for 5+ years of directly relevant experience. Orion Innovation lists their specific requirements in the description below, so review the must-have qualifications closely before applying.