Backend Engineer - Monitoring

About Sifflet

We are building the world’s best data observability platform to help companies excel at data-driven decision making.

Today, half of a data team’s time is spent troubleshooting data quality issues. Sifflet is putting an end to that. Our solution allows data engineers and data consumers to visualize how data flows between their services, define data quality checks, and quickly find the root cause of any data anomaly.

About the job

Sifflet implements data quality checks ranging from simple (detecting null values) to very complex (time series forecasting models to validate that the distribution of a set of columns hasn’t changed in an unexpected way, taking into account seasonality, one-off events…). This foundation powers many features, such as automatically merging related alerts into incidents.

The monitoring team is responsible for evolving the data quality checks performed by Sifflet, and all associated workflows. As a member of this team, you will:

• Design and implement new types of data quality checks.

• Build features to allow users to efficiently monitor their entire data stack, such as automated monitor suggestions.

• Design advanced solutions to cut the alerting noise, such as automated incident root cause analysis.

• Scale our monitoring engine to support more and more customers. Some customers require monitoring massive data sets.

Some projects you could be working on

• Add support for automated data profiling (understanding the expected distribution of values in all columns), and build an automated anomaly detection on top of that.

• Implement automated monitoring at scale, without requiring the user to manually define its own quality checks.

• Design new, more powerful data quality checks.

Our stack

• The monitoring engine is built with Python 3 and its large data/ML ecosystem (notably PyTorch).

• Jobs are orchestrated with Temporal.

• The web API is written in (modern) Java with Spring Boot 3, the web frontend is a VueJS application written in Typescript. You may occasionally need to make minor changes to this code base.

• Infrastructure: Kubernetes (AWS EKS clusters), MySQL (on AWS RDS).

• Plus a few supporting services: Gitlab CI, Prometheus/Loki/Grafana, Sentry…

While not directly part of our stack, expect to gain a lot of knowledge on many products in the modern data ecosystem. The subtleties of BigQuery or Snowflake will soon be very familiar to you.

Are we the company you’re looking for?

• We have offices in Paris, but we’re very remote friendly - several team members are fully remote.

• We offer competitive salary and company equity.

• We have experts on many topics, so there’s always someone to help. We also have tech talks where everyone can discuss a cool project or technology.

• We’re constantly exposed to the intricacies of the modern data ecosystem - you’ll become very knowledgeable about data engineering and the modern data stack, and about how data is used in enterprises.

• Our culture emphasises teamwork to efficiently deliver projects to production.

• We’re building a genuinely great product, and we think you’ll love the team!