Technical Stack

Connectors can be written in any language. However the most common languages are:

JDBC makes writing reusable database connector frameworks fairly easy, saving us a lot of development time.

Most contributors felt comfortable writing in Python, so we created a Python CDK to accelerate this development. You can write a connector from scratch in any language as long as it follows the Airbyte Specification.

Simply put, the team has more experience writing production Java code.

Temporal solves the two major hurdles that exist in orchestrating hundreds to thousands of jobs simultaneously: scaling state management and proper queue management. Temporal solves this by offering primitives that allow serialising the jobs' current runtime memory into a DB. Since a job's entire state is stored, it's trivial to recover from failures, and it's easy to determine if a job was assigned correctly.
Copy link
On this page
Airbyte Core Backend
Additional Tools
Why do we write most destination/database connectors in Java?
Why are most REST API connectors written in Python?
Why did we choose to build the server with Java?
Why do we use Temporal for orchestration?