---
name: create-source-file
description: Generate the dbt source YAML files of a given database and schemas, using the dbt-codegen package.
disable-model-invocation: true
argument-hint: "<database> <schema> [schema2 ...]"
---

# Create Source File

Generate dbt source YAML files for the database and schemas provided in `$ARGUMENTS`.

Parse `$ARGUMENTS` as: first token = `<database>`, remaining tokens = one or more `<schema>` names.

## What this skill does

Uses the [`dbt-codegen`](https://github.com/dbt-labs/dbt-codegen) package to introspect the warehouse and generate a `sources.yml` file for each schema, then writes them into the dbt project.

## Prerequisites

Check that `dbt-codegen` is listed in `packages.yml`. If it is missing, add it and run `dbt deps` before proceeding:

```yaml
# packages.yml
packages:
  - package: dbt-labs/codegen
    version: [">=0.12.0", "<1.0.0"]
```

```bash
dbt deps
```

## Procedure

### 1. Parse arguments

Split `$ARGUMENTS` on whitespace:
- `database` = first token
- `schemas` = all remaining tokens (one or more)

If fewer than two tokens are provided, stop and ask the user for the missing database or schema name.

### 2. Generate source YAML for each schema

For each schema, run the `generate_source` macro from dbt-codegen:

```bash
dbt -q run-operation generate_source \
  --args '{"schema_name": "<schema>", "database_name": "<database>", "generate_columns": true}' \
  > _<schema>.yml
```

This write a source YAML file in named in the following format:  _<schema>.yml

### 3. Move the source files

For each schema, verify if a folder exists in models/base/ and if not, create it.
Then move the corresponding source file in the correct folder. 

```
models/base/<schema>/_<schema>.yml
```

If the file already exists, show the diff and ask the user whether to overwrite or merge.

### 4. Generate documentation

Read the written `_<schema>.yml` file and fill in `description:` fields for the source, each table, and each column. Infer descriptions from names using standard data engineering conventions (e.g. `created_at` → "Timestamp when the record was created.", `customer_id` → "Unique identifier for the customer.").

For the source block:

```yaml
sources:
  - name: <schema>
    description: "Raw data from the <database>.<schema> schema."
    database: <database>
    schema: <schema>
    tables:
      - name: <table>
        description: "<Inferred one-sentence description of the table.>"
        columns:
          - name: <column>
            description: "<Inferred one-sentence description of the column.>"
```

Rules:
- Keep descriptions concise — one sentence, no trailing period on column names that are self-evident from the field name.
- For columns whose purpose cannot be inferred, write `"TODO: add description."` as a placeholder.
- Do not remove or alter any other fields generated by `dbt-codegen`.

Overwrite the file with the documented version.

### 5. Validate

Parse the project to confirm dbt can resolve the new sources and documented models:

```bash
dbt parse
```

Fix any errors before finishing.

## Output

One file per schema:

```
models/base/<schema>/_<schema>.yml
```

## Notes

- If `$ARGUMENTS` is empty, ask the user for the database and at least one schema before proceeding.
- Do not delete or overwrite existing source files without user confirmation.
- Follow the project's existing file naming conventions if they differ from the defaults above.