RFC-006 Vyasa Package Format

RFC 006: Vyasa Package Format

Status: Draft Date: 2026-01-30 Topics: Packaging, Interop, SQLite

1. Summary

This RFC defines the two standard formats generated by the vyasac pack command:

1. Source Exchange Format (.vypkg): A ZIP archive containing compiled ASTs as JSON. Designed for tooling and incremental compilers.
2. Application Database (.db): A monolithic SQLite database. Designed for efficient consumption by reading apps and analysis tools.

2. Common Metadata (Manifest)

Both formats encode the same core metadata.

Data Model

• Name: Human-readable name of the workspace.
• URN Scheme: The active URN template (e.g., urn:vyasa:bg:{chapter}:{id}).
• Root: The root content directory name.
• Stats: Counts of files, markers, and entities.
• Timestamp: Build time.

3. Source Exchange Format (Zip)

A compressed archive mirroring the source directory structure, but with JSON ASTs instead of .vy files.

3.1 File Layout

sample.vypkg (zip)
├── manifest.json
└── content/
    ├── vol1/
    │   └── book.json  (Compiled from book.vy)
    └── context.json

3.2 manifest.json

{
  "name": "Bhagavad Gita",
  "urn_scheme": "urn:vyasa:vedabase:bg:{chapter}:{id}",
  "root": "content",
  "stats": {
    "files": 45,
    "markers": 700,
    "entities": 150
  }
}

3.3 AST Format

See IR Schema (if defined) or the Vyasa AST specification. Use Node::Command, Node::Text, etc.

4. Application Database (SQLite)

A single-file relational database optimized for random access querying.

4.1 Schema Overview

The database contains four normalized tables.

Table: manifest

Key-value storage for workspace metadata.

CREATE TABLE manifest (
    key TEXT PRIMARY KEY,
    value TEXT
);
-- Rows: 'name', 'urn_scheme', 'stats_files', etc.

Table: streams

Registry of all content streams (Primary and Secondary).

CREATE TABLE streams (
    id INTEGER PRIMARY KEY,
    name TEXT NOT NULL, -- e.g., 'primary', 'translation/en'
    type TEXT NOT NULL  -- 'primary' | 'secondary'
);

Table: nodes

The monolithic content table. All AST nodes from all files are flattened here.

CREATE TABLE nodes (
    id INTEGER PRIMARY KEY,
    stream_id INTEGER,
    parent_id INTEGER,
    urn TEXT,           -- Indexed for fast lookups
    type TEXT NOT NULL, -- 'command', 'text', 'break', 'comment'
    cmd TEXT,           -- Command name (if type='command')
    content TEXT,       -- Text content or Command argument
    ordinal INTEGER     -- Sorting order within parent
);
CREATE INDEX idx_nodes_urn ON nodes(urn);

Table: registry (Entities)

Global table of defined semantic entities.

CREATE TABLE registry (
    name TEXT PRIMARY KEY
);

CREATE TABLE registry_attributes (
    entity_name TEXT,
    key TEXT,
    value TEXT,
    PRIMARY KEY (entity_name, key),
    FOREIGN KEY(entity_name) REFERENCES registry(name)
);

4.2 Query Examples

Find a specific Verse:

SELECT * FROM nodes WHERE urn = 'urn:vyasa:bg:2:12';

Get all translation text for Chapter 1:

SELECT n.content
FROM nodes n
JOIN streams s ON n.stream_id = s.id
WHERE s.name = 'translation/en'
  AND n.urn LIKE 'urn:vyasa:bg:1:%'
  AND n.type = 'text';