---
name: run-dlt-connector
description: Execute a dlt connector locally to extract and load data. Use when the user wants to run a pipeline, test a connector locally, execute a connector, load data with uv run, or verify a connector works end-to-end.
argument-hint: "[connector-name]"
disable-model-invocation: true
---

# Run a dlt Connector

Run a dlt connector to extract data from an API source and load it to a destination database. Connectors run via `uv` with isolated dependencies.

## Context

Connectors live in `connectors/$ARGUMENTS/` within the dlt-connectors monorepo (for development) or in a client Airflow repo (synced via `sync-connectors.sh`). Each connector is a self-contained uv project with its own `pyproject.toml`, `uv.lock`, and a `<connector_name>_pipeline.py` entry point.

## Prerequisites

- Connector exists in `connectors/$ARGUMENTS/`
- Credentials configured in `.dlt/secrets.toml`
- Destination database running (if using Postgres)

## Steps

1. Navigate to the connector directory.

   ```bash
   cd connectors/<connector_name>
   ```

2. Verify configuration files exist.

   ```bash
   # Check secrets.toml and .env exist (never read secrets.toml in full - it contains credentials)
   ls .dlt/secrets.toml .env

   # Check destination config
   grep "^\[destination" .dlt/secrets.toml
   grep "^host = " .dlt/secrets.toml
   ```

3. If the destination is Postgres, ensure the database container is running.

   ```bash
   docker ps | grep dlt-postgres

   # Start if not running
   docker start dlt-postgres
   ```

4. Install dependencies and run the connector.

   ```bash
   uv sync
   uv run --env-file=.env python <connector_name>_pipeline.py
   ```

5. Override the destination or source name via environment variables.

   ```bash
   DLT_DESTINATION=postgres DLT_SOURCE_NAME=<source_name> uv run python <connector_name>_pipeline.py
   ```

6. Verify data was loaded by checking tables in the destination.

   ```bash
   PGPASSWORD=dev12345_ psql -h localhost -p 5434 -U dlt -d postgres -c "
   SELECT table_name FROM information_schema.tables
   WHERE table_schema = '<connector_name>'
   ORDER BY table_name;"
   ```

   Sample a table to confirm data:

   ```bash
   PGPASSWORD=dev12345_ psql -h localhost -p 5434 -U dlt -d postgres -c "
   SELECT * FROM <connector_name>.<table_name> LIMIT 5;"
   ```

## Validation

- [ ] Pipeline ran without errors
- [ ] Data verified in destination (tables exist, rows loaded)

## Examples

**Troubleshooting:**

| Issue | Solution |
|-------|----------|
| Connection refused | Start Postgres: `docker start dlt-postgres` |
| Pending packages warning | Previous run failed; run again to load pending data |
| Rate limit exceeded | Wait and retry; most connectors handle this automatically |
| Invalid credentials | Check `.dlt/secrets.toml` source section |
| `uv sync` fails | Check `pyproject.toml` for valid dependency specs |
| Wrong Python version | `uv` auto-downloads the required Python version; ensure internet access |