|Airbyte||Open-source data integration platform for modern data teams|
|Node.JS (TypeScript)||Main programming language to write sources and destinations|
|Python||Programming language to write test scripts|
|Docker||For running acceptance tests, we install Docker on local environment to test source|
|GitHub||Version Control System, system’s code, all commits, review, merge, task manager|
Introduction to the Company
Faros AI, a Software Development company, provides to their clients an operational data platform that brings all engineering data in one place. The data platform integrates all company’s engineering data sources to give a holistic visibility into the entire software development lifecycle. It takes the guesswork out of planning so that engineering leaders can make decisions, allocate resources, and improve productivity based on actual data.
For building their data platform Faros AI uses Airbyte open-source data pipeline platform which currently has more than 100 data connectors available enabling data extraction from multiple sources. The data connectors make it possible to move and consolidate data from different sources to data warehouses, data lakes, or databases in an ELT process.
It can be easily understood that the more data connectors are built into the Faros AI, the more business value is provided to their customers. Therefore, company realized that they need someone who would be building those data connectors to multiple data sources.
Faros AI reached out to us explaining in detail the challenge that they were facing. The challenge was to develop the Source and Destination Airbyte connectors and integrate them with Faros AI platform so that the company’s customers could benefit from these integrations by being able to reach all their engineering data in one place and making data-driven decisions.
The thing is that Airbyte connectors have different statuses – Certified, Beta, and Alpha. In case of certified connectors, they are robust and fully tested, so adding them to the Faros AI platform should not be a big deal. However, other types of connectors would require making some changes. Moreover, some of the data sources that Faros AI wanted to connect to did not have their corresponding Airbyte connectors. Therefore, the connectors for such data sources had to be developed from scratch.
Besides, since the structure or the form of the data in the data sources differ from that required for Faros AI platform, there should be some sort of converters which would convert the original data to the required format.
The following Airbyte Connectors were built and integrated with the Faros AI platform by DevCube:
- Google Calendar
- Agile Accelerator
- Azure Active Directory
- Azure Pipeline
- Azure Git
While building these connectors, we have either made changes to the sources to meet the requirements of the customer or developed the connectors from scratch. Besides, for all these connectors a data converter was added to transform the original data to the required format of the Faros AI platform.
There is no need to explain that the more engineering data the Faros AI’s customers can collect and analyze the smarter their decision-making becomes.
With the gradual increase in the number of Airbyte connectors we have developed and connected so far to the platform, the customers of Faros AI have become able to access more and more of their engineering data and benefit from it.
Moreover, it should be noted that nowadays companies use multiple different platforms and systems for their software development. So, not only the customers of Faros AI have now access to their engineering data, but also the variety of these engineering data sources increased. And there are many other Airbyte connectors to be created by DevCube for Faros AI.