Connectors
Connectors define how the platform connects to external data sources. Each connector type brings its own authentication method, data access patterns, and capabilities.
Available Connectors
PostgreSQL
The most mature connector with full streaming, incremental ingestion, and real-time preview.
| Feature | Status |
|---|---|
| Authentication | Username/Password |
| Schema Browser | ✅ |
| Custom SQL Queries | ✅ |
| Preview (AI-enriched) | ✅ |
| Incremental Ingestion | ✅ (change-tracking) |
| Scheduled Execution | ✅ |
How it works: The platform connects to your PostgreSQL database, runs the configured query (or full table extract), streams the rows as CSV, and imports them into your data warehouse.
Google Sheets
Connect your Google account via OAuth to ingest data from Google Sheets spreadsheets.
| Feature | Status |
|---|---|
| Authentication | OAuth 2.0 (Google) |
| Preview | ✅ |
| Scheduled Execution | ✅ |
| Incremental Ingestion | ❌ (full extract) |
Configuration fields:
- Spreadsheet ID — Found in the spreadsheet URL:
docs.google.com/spreadsheets/d/SPREADSHEET_ID/edit - Range — e.g.
Sheet1!A:Zor justA:Zfor the default sheet
How it works: The platform uses the Google Sheets API v4 to read the spreadsheet data. The first row is treated as headers. All subsequent rows are converted to CSV and uploaded to the data warehouse.
Microsoft (Excel / SharePoint)
Connect your Microsoft 365 account via OAuth to access Excel Online files, SharePoint folders, and SharePoint lists.
| Feature | Status |
|---|---|
| Authentication | OAuth 2.0 (Microsoft) |
| Preview | ✅ |
| Scheduled Execution | ✅ |
| Incremental Ingestion | ❌ (full extract) |
Three source types are supported:
Excel File
Read a specific Excel workbook from OneDrive or SharePoint.
SharePoint Folder
Ingest all CSV and Excel files from a SharePoint document library folder.
SharePoint List
Read data from a SharePoint list and convert to tabular format.
MySQL (Partial)
Configuration is accepted but execution uses the schema browser UI. Actual SQL streaming is pending implementation.
SQL Server (Partial)
Same status as MySQL — connection configuration works, execution pending.
Inbound API (Push)
The platform acts as an API server — external systems push data into the platform via a dedicated webhook endpoint. Each connection gets a unique, revocable ingest token.
| Feature | Status |
|---|---|
| Authentication | Ingest Token (auto-generated, 256-bit) |
| Direction | Push (external → platform) |
| Preview | ✅ (last received data) |
| Scheduled Execution | N/A — event-driven |
How it works: Create an Inbound API connection, the platform generates a unique endpoint URL with an ingest token. External systems POST JSON data to the endpoint.
AI API Client (Pull)
Connect to any REST API by providing its documentation and a natural language description of what data to fetch. The platform uses AI to automatically generate the integration code.
| Feature | Status |
|---|---|
| Authentication | Bearer Token / API Key / Basic Auth / Custom Header |
| Direction | Pull (platform → external) |
| AI Code Generation | ✅ |
| Preview | ✅ |
| Scheduled Execution | ✅ |
| Code Review | ✅ (user must approve before scheduling) |
MCP Server (Inbound)
The platform exposes an MCP (Model Context Protocol) server endpoint. External AI agents can discover and call tools exported by the platform.
| Feature | Status |
|---|---|
| Authentication | API Key (auto-generated) |
| Direction | Inbound (external agent → platform) |
| Protocol | SSE (Server-Sent Events) |
MCP Client (Outbound)
Connect to external MCP servers to discover and call their tools for data extraction.
| Feature | Status |
|---|---|
| Authentication | Bearer Token (optional) |
| Direction | Outbound (platform → external MCP server) |
| Tool Discovery | ✅ (auto-discovered on connection) |
| Preview | ✅ |
| Scheduled Execution | ✅ |
Shopify
Ingest data from your Shopify store via the Admin API.
| Feature | Status |
|---|---|
| Authentication | Admin API Access Token |
| Supported Objects | Orders, Products, Customers, Inventory Items, Collections |
| Preview | ✅ |
| Pagination | ✅ (cursor-based via Link header) |
| Incremental Ingestion | ✅ (via updated_at change tracking) |
| Scheduled Execution | ✅ |
| Plan | Starter+ |
Configuration fields:
- Shop Domain — e.g.
my-store.myshopify.com - Access Token — From a Shopify custom app (Admin API)
- API Version — Defaults to
2024-01
How it works: The platform calls the Shopify Admin REST API, paginates through all records using cursor-based pagination (Link header), flattens nested objects, and uploads the data as CSV to the data warehouse.
To get an access token:
- Go to Shopify Admin → Settings → Apps and sales channels → Develop apps
- Create a custom app and configure Admin API scopes (read_orders, read_products, etc.)
- Install the app and copy the Admin API access token
Stripe
Ingest payment and billing data from Stripe.
| Feature | Status |
|---|---|
| Authentication | Restricted API Key (read-only) |
| Supported Objects | Charges, Subscriptions, Customers, Invoices, Payouts, Disputes, Products, Prices |
| Preview | ✅ |
| Pagination | ✅ (cursor-based via starting_after) |
| Incremental Ingestion | ✅ (via created timestamp filter) |
| Scheduled Execution | ✅ |
| Plan | Starter+ |
Configuration fields:
- API Key — Restricted key with read-only permissions
How it works: The platform calls the Stripe REST API with cursor-based pagination, flattens nested objects (metadata, address, etc.), and uploads to the data warehouse.
To get an API key:
- Go to Stripe Dashboard → Developers → API keys
- Create a restricted key with read-only permissions for the data you need
HubSpot
Ingest CRM data from HubSpot.
| Feature | Status |
|---|---|
| Authentication | Private App Access Token |
| Supported Objects | Contacts, Companies, Deals, Tickets, Products, Line Items |
| Preview | ✅ |
| Pagination | ✅ (cursor-based search API) |
| Incremental Ingestion | ✅ (via updatedAt) |
| Scheduled Execution | ✅ |
| Plan | Growth+ |
Configuration fields:
- Access Token — From a HubSpot private app
How it works: The platform uses the HubSpot CRM v3 Search API to paginate through records. Properties are automatically flattened from the nested properties object.
To get an access token:
- Go to HubSpot → Settings → Integrations → Private Apps
- Create a private app with CRM object read scopes
- Copy the access token
TOTVS Protheus
Ingest data from the TOTVS Protheus ERP system via its REST API.
| Feature | Status |
|---|---|
| Authentication | Basic Auth / Bearer Token / API Key |
| Supported Entities | Customers (SA1), Products (SB1), Sales Orders (SC5), Invoices (SF2), Financials (SE1/SE2), custom |
| Preview | ✅ |
| Pagination | ✅ (offset-based) |
| Field Mapping | ✅ (map Protheus fields to standard names) |
| Scheduled Execution | ✅ |
| Plan | Growth+ |
Configuration fields:
- Base URL — e.g.
https://protheus.company.com:8888 - Auth Type — Basic, Bearer, or API Key
- Environment / Company / Branch — Protheus-specific context headers
How it works: The platform calls the TOTVS Protheus REST API using offset pagination. Pre-configured entity endpoints (SA1, SB1, SC5, SF2, SE1, SE2) map to standard business objects. Custom endpoints can be specified for non-standard entities.
S3 / GCS (Cloud Bucket)
Ingest files from Amazon S3 or Google Cloud Storage buckets.
| Feature | Status |
|---|---|
| Authentication | Access Key (S3) / Service Account (GCS) |
| Supported Formats | CSV, JSON, JSONL |
| Preview | ✅ |
| Multi-File Ingestion | ✅ |
| S3-Compatible | ✅ (MinIO, DigitalOcean Spaces, etc.) |
| Scheduled Execution | ✅ |
| Plan | Growth+ |
Configuration fields:
- Provider — S3 or GCS
- Bucket Name — Target bucket
- Region — S3 region (e.g.
us-east-1) - Custom Endpoint — For S3-compatible services like MinIO
How it works: The platform lists files in the bucket (filtered by prefix), downloads and parses each file, then imports the combined data into your data warehouse. A _source_file column is added to track the origin of each row.
Salesforce
Ingest CRM and business data from Salesforce using SOQL queries.
| Feature | Status |
|---|---|
| Authentication | OAuth Access Token |
| Custom SOQL Queries | ✅ |
| Supported Objects | All standard and custom Salesforce objects |
| Preview | ✅ |
| Pagination | ✅ (automatic via nextRecordsUrl) |
| Incremental Ingestion | ✅ (via LastModifiedDate) |
| Scheduled Execution | ✅ |
| Plan | Growth+ |
Configuration fields:
- Instance URL — e.g.
https://mycompany.salesforce.com - Access Token — OAuth access token
- API Version — Defaults to
v59.0
How it works: The platform executes SOQL queries against the Salesforce REST API. Results are automatically paginated using nextRecordsUrl. You can specify individual objects with field selection or write custom SOQL queries.
MongoDB
Ingest documents from MongoDB via the Atlas Data API.
| Feature | Status |
|---|---|
| Authentication | Atlas Data API Key |
| Custom Queries | ✅ (filter, projection, sort) |
| Preview | ✅ |
| Scheduled Execution | ✅ |
| Plan | Starter+ |
Configuration fields:
- Data API URL — Atlas Data API endpoint
- API Key — Data API key
- Data Source — Cluster name (e.g.
Cluster0) - Database — Target database
How it works: The platform uses the MongoDB Atlas Data API /action/find endpoint. Extended JSON types ($oid, $date, $numberDecimal) are automatically converted to primitive values. Arrays and nested objects are JSON-serialized into string columns.
FTP / SFTP
Connect to FTP or SFTP servers to ingest files.
| Feature | Status |
|---|---|
| Authentication | Username/Password or Private Key (SFTP) |
| Connection Test | ✅ (TCP reachability) |
| Full Execution | ⚠️ Requires ssh2-sftp-client dependency |
| Plan | Starter+ |
Note: Full FTP/SFTP execution requires the ssh2-sftp-client and basic-ftp npm packages to be installed. The connection test checks TCP reachability only.
BigQuery
Run SQL queries against Google BigQuery and ingest the results.
| Feature | Status |
|---|---|
| Authentication | Service Account JSON (JWT) |
| Custom SQL Queries | ✅ |
| Preview | ✅ |
| Schema Types | ✅ (preserves BigQuery type metadata) |
| Scheduled Execution | ✅ |
| Plan | Business+ |
Configuration fields:
- Project ID — GCP project ID
- Service Account JSON — Service account key with BigQuery Reader role
- Location — BigQuery dataset location (e.g.
US,EU)
How it works: The platform signs a JWT using the service account key, exchanges it for an access token, then executes the SQL query via the BigQuery REST API. Schema metadata is preserved (field types from BigQuery schema).
Snowflake
Run SQL queries against Snowflake and ingest the results.
| Feature | Status |
|---|---|
| Authentication | Username/Password or Key-Pair |
| Custom SQL Queries | ✅ |
| Preview | ✅ |
| Scheduled Execution | ✅ |
| Plan | Business+ |
Configuration fields:
- Account — Snowflake account identifier (e.g.
xy12345.us-east-1) - Username / Password — Snowflake credentials
- Warehouse — Compute warehouse
- Database / Schema / Role — Default context
How it works: The platform submits SQL queries via the Snowflake SQL REST API (/api/v2/statements). Results are returned synchronously for small queries. Column names and data types are extracted from resultSetMetaData.
SAP (OData)
Ingest data from SAP systems via OData services.
| Feature | Status |
|---|---|
| Authentication | Basic Auth / OAuth / API Key |
| OData Queries | ✅ ($select, $filter, $expand) |
| Preview | ✅ |
| Pagination | ✅ ($skip/$top with inline count) |
| Scheduled Execution | ✅ |
| Plan | Business+ |
Configuration fields:
- OData Service URL — SAP Gateway service URL
- Auth Type — Basic, OAuth, or API Key
- SAP Client — (optional) Client number (e.g.
100)
How it works: The platform calls the SAP OData service with $skip/$top pagination, using $inlinecount=allpages to determine total record count. Metadata objects (__metadata, __deferred) are automatically stripped from results.
Kafka
Consume messages from Kafka topics via the Confluent REST Proxy.
| Feature | Status |
|---|---|
| Authentication | SASL (PLAIN / SCRAM) |
| Protocol | Confluent REST Proxy (v2) |
| Preview | ✅ |
| Offset Control | ✅ (earliest / latest) |
| Scheduled Execution | ✅ |
| Plan | Business+ |
Configuration fields:
- REST Proxy URL — Confluent REST Proxy endpoint
- Schema Registry URL — (optional) For Avro/Protobuf deserialization
- SASL Credentials — (optional) Username/password
How it works: The platform creates a consumer instance via the REST Proxy, subscribes to the specified topic, consumes messages, and then cleans up the consumer. Message values are flattened from JSON; metadata (topic, partition, offset, key, timestamp) is preserved.
Note: For native Kafka connections (without REST Proxy), install kafkajs (coming soon).
Notion
Ingest data from Notion databases and pages.
| Feature | Status |
|---|---|
| Authentication | Integration Token |
| Supported Objects | Databases, Pages |
| Preview | ✅ |
| Pagination | ✅ (cursor-based) |
| Incremental Ingestion | ✅ (via last_edited_time) |
| Scheduled Execution | ✅ |
| Plan | Growth+ |
Configuration fields:
- Integration Token — From a Notion integration (
secret_...)
How it works: The platform queries Notion databases using the API's /databases/{id}/query endpoint. All 18+ Notion property types (title, rich_text, number, select, multi_select, date, checkbox, URL, email, phone, formula, relation, rollup, people, files, created_time, last_edited_time, status) are automatically flattened to scalar values.
To get an integration token:
- Go to notion.so/my-integrations
- Create a new integration
- Share the target database with the integration
Slack
Export messages, users, and channels from Slack workspaces.
| Feature | Status |
|---|---|
| Authentication | Bot User OAuth Token (xoxb-...) |
| Supported Data Types | Messages, Users, Channels |
| Preview | ✅ |
| Pagination | ✅ (cursor-based) |
| Incremental Ingestion | ✅ (via message ts timestamp) |
| Scheduled Execution | ✅ |
| Plan | Growth+ |
Configuration fields:
- Bot Token — Slack Bot User OAuth Token (
xoxb-...)
Data types:
- Messages — Channel message history with reactions, threads, attachments
- Users — Workspace members with profile info, email, status
- Channels — Public and private channels with topic, purpose, member count
To get a bot token:
- Go to api.slack.com/apps and create a new app
- Add Bot Token Scopes:
channels:history,channels:read,users:read,users:read.email - Install the app to your workspace and copy the Bot User OAuth Token
Connector Architecture
All connectors follow the same pipeline pattern:
External Source → Connector Engine → Data Processing → Iara Data Warehouse
- Connection stores encrypted credentials (OAuth tokens or passwords)
- Job defines what to extract (query, spreadsheet, folder) and the schedule
- Execution engine routes to the appropriate connector
- Data is always normalized before being imported into your data warehouse
- The platform handles schema inference, table creation, and data loading
Pricing Tiers
| Tier | Connectors Included |
|---|---|
| Starter | PostgreSQL, MySQL, SQL Server, Google Sheets, Microsoft Excel, File Upload, Inbound API, AI API Client, MCP, Shopify, Stripe, MongoDB, FTP/SFTP |
| Growth | + HubSpot, TOTVS Protheus, S3/GCS, Salesforce, Notion, Slack |
| Business | + BigQuery, Snowflake, SAP, Kafka |