Semantic Layer Export & Import

Introduction

The Semantic Layer Export and Import feature allows you to export and import Waii semantic layer configurations. This feature can be used for:

Migrating Waii semantic layer definitions across environments
Backing up and restoring semantic layer configurations
Sharing and collaborating on semantic layer definitions
Version controlling your semantic layer configurations

Key Features

Configuration Export:
- Generates a human-readable, comprehensive YAML representation of your Waii semantic layer that has:
  - Database and schema definitions.
  - Table and column metadata (descriptions, constraints).
  - Semantic context statements.
  - Liked queries.
  - Similarity search index configurations.
  - Database content filters.
- Supports filtering by search context to limit/select export scope
- Adds statistics, i.e, number of tables, schemas, semantic context statements, etc. exported
Configuration Import:
- Applies exported configurations to your Waii semantic layer
- Supports database and schema remapping during import
- Supports dry-run mode for validation without applying changes
- Offers strict mode option to control how existing configurations are handled
- Generates detailed reports on configurations imported

Configuration Structure

The configuration includes:

Database-level configurations and content filters
Schema-level configurations and descriptions
Table-level configurations, including tables, columns description and constraints
Semantic context statements
Liked/saved queries
Similarity search index configurations

{
    "version": "1.0",                 # Format version
    "metadata": {                     # Export metadata
        "exported_at": "2025-03-31T01:17:32.501001",
        "source_dialect": "snowflake",
        "source_database": "sales_analytics"
    },
    "database": {                     # Database-level configurations
        "content_filters": []         # Content filters applied at database level
    },
    "tables": [                       # Table definitions
        {
            "name": "order_transactions",          # Table name
            "schema_name": "sales",               # Schema name
            "description": "This table stores transaction data for all customer orders", # Table description
            "constraints": [                      # Constraints (PK, FK, etc.)
                # table constraints
            ],
            "columns": [                          # Column definitions
                {
                    "name": "order_id",
                    "type": "INT64",
                    "description": "Unique identifier for each customer order",
                    "description_update_source": "generated"
                }
                # More columns...
            ]
        }
        # More tables...
    ],
    "schemas": [                     # Schema definitions
        {
            "name": "sales",
            "description": {
                "summary": "This schema contains all sales-related tables and views for analyzing customer transactions and orders",
                "common_tables": [   # common tables in the schema
                    {
                        "name": "customer_dimension",
                        "description": "This table stores customer profile information including demographics and contact details"
                    }
                    # More tables...
                ]
            }
        }
        # More schemas...
    ],
    "liked_queries": [
        {
            "query_uuid": "f91a83e7-f098-4c79-9927-9c3f362077f9",
            "ask": "Give me 5 movie names sorted by movie name",
            "query": "SELECT\n  title\nFROM public.movies\nORDER BY\n  title\nLIMIT 5",
            "liked": true,
            "detailed_steps": [
                "Select the 'title' column from the 'schema.public.movies' table.",
                "Order the results by 'title' in ascending order with NULLS LAST.",
                "Limit the result to 5 rows."
            ]
            # Other configs...
        }
    ]              # Saved/liked queries
}

Import Modes

Strict Mode: When enabled, all existing configurations in the target connection are removed before importing. When disabled, existing configurations are preserved and only new configurations are added.
Dry Run Mode: When enabled, the system validates and reports what changes would be made without actually applying them. This is useful for testing before performing the actual import.

Note: It's important to note that importing a configuration does not impact the database/schema/table definition in either mode. Only the configurations generated and used by Waii will be affected.

CLI Reference

Check CLI Reference for more options

Common Use Cases and Examples

Migrating Between Environments

Use the export and import feature to move configurations from development to production environments:

Export the configuration from your development environment:
```
waii semantic-layer-dump export --db_conn_key="dev_connection_key" --file="dev_export.yaml"
```
Note: you can get list of connections and key you have using:
```
waii database list
```

Import to your production environment in dry run mode:

waii semantic-layer-dump import --db_conn_key="prod_connection_key" --file="dev_export.yaml" --dry_run_mode=true

Once validated, perform the actual import:

waii semantic-layer-dump import --db_conn_key="prod_connection_key" --file="dev_export.yaml"

Creating Backups

Regularly export your semantic layer configuration to:

Create a dated backup of your semantic layer definitions:

waii semantic-layer-dump export --db_conn_key="your_connection_key" --file="backup_$(date +%Y%m%d).yaml"

Track changes over time by comparing configurations
Restore configurations if needed

Troubleshooting

If an export fails, check that you have proper permissions to access all schemas and tables
If an import fails in strict mode, consider using non-strict mode first
Use dry-run mode to identify configs that will be affected before performing actual imports
For large configurations, exports and imports may take longer to complete
Use the --verbose flag to get more detailed information about export/import operations

Introduction​

Key Features​

Configuration Structure​

Import Modes​

CLI Reference​

Common Use Cases and Examples​

Migrating Between Environments​

Creating Backups​

Troubleshooting​