@pyric/cli/discover

Classes

LocalEnvironmentCrawlerAdapter

Wrap a LocalEnvironment as a Firestore root that satisfies the discover/crawler + find-collection-group contracts.

Implements

• CrawlerFirestore
• CollectionGroupCapableFirestore

Constructors

Constructor

new LocalEnvironmentCrawlerAdapter(env: LocalEnvironment): LocalEnvironmentCrawlerAdapter;

Parameters

Returns

LocalEnvironmentCrawlerAdapter

Methods

collection()

collection(path: string): CrawlerCollectionRef;

Parameters

Returns

CrawlerCollectionRef

Implementation of

CrawlerFirestore.collection

collectionGroup()

collectionGroup(collectionId: string): CollectionGroupQuery;

Parameters

Returns

CollectionGroupQuery

Implementation of

CollectionGroupCapableFirestore.collectionGroup

doc()

doc(path: string): CrawlerDocumentRef;

Parameters

Returns

CrawlerDocumentRef

Implementation of

CrawlerFirestore.doc

listCollections()

listCollections(): Promise<CrawlerCollectionRef[]>;

Returns

Promise<CrawlerCollectionRef[]>

Implementation of

CrawlerFirestore.listCollections

Semaphore

FIFO counting semaphore. acquire() resolves immediately while fewer than max permits are checked out, otherwise it queues until a release() frees a slot. Waiters are served in arrival order.

release() without a prior acquire() is a no-op (does not go negative). This is intentional — it lets defensive try/finally release in error paths without bookkeeping.

Constructors

Constructor

new Semaphore(max: number): Semaphore;

Parameters

Returns

Semaphore

Accessors

inFlight

Get Signature

get inFlight(): number;

Number of permits currently checked out. For tests and instrumentation only; do not branch on this in production logic.

Returns

number

pending

Get Signature

get pending(): number;

Number of waiters queued. For tests and instrumentation only.

Returns

number

Methods

acquire()

acquire(): Promise<void>;

Returns

Promise<void>

release()

release(): void;

Returns

void

SessionStore

In-process LRU session store with TTL sweep and per-session byte cap.

Eviction policy:

1. On every create/get/update, sweep TTL-expired sessions first. Their tokens land in the eviction log as SESSION_EXPIRED.
2. If create would exceed maxSessions, evict the LRU (oldest-by-lastAccessedAt). Its token lands in the eviction log as SESSION_EVICTED so the displaced agent gets a meaningful error on its next call.
3. update rejects with SESSION_PAYLOAD_TOO_LARGE if the new bytes exceeds maxSessionBytes (per-session cap, not aggregate).

The eviction log is a bounded ring buffer; once it overflows, evicted tokens degrade silently to SESSION_EXPIRED (still actionable — the recoveryHint is the same: re-issue without continuation).

Type Parameters

Constructors

Constructor

new SessionStore<TState>(opts?: SessionStoreOptions): SessionStore<TState>;

Parameters

Returns

SessionStore<TState>

Accessors

size

Get Signature

get size(): number;

Live session count.

Returns

number

Methods

create()

create(state: TState, bytes: number): SessionResult<SessionRecord<TState>>;

Create a new session. Always succeeds unless bytes exceeds the per-session byte cap. On cap-hit, evicts the LRU session — the displaced token will report SESSION_EVICTED on its next access.

Parameters

Returns

SessionResult<SessionRecord<TState>>

delete()

delete(token: string): boolean;

Best-effort delete; returns true if a session was removed.

Parameters

Returns

boolean

get()

get(token: string): SessionResult<SessionRecord<TState>>;

Look up a session by token. Touches lastAccessedAt on success so subsequent reads keep the session warm. On expired/malformed/evicted tokens returns the appropriate structured error.

Parameters

Returns

SessionResult<SessionRecord<TState>>

sweepExpired()

sweepExpired(now?: number): number;

Drop sessions whose lastAccessedAt + ttlMs is in the past. Returns the number of sessions evicted. Public for tests + future scheduled-sweep usage; create/get/update all call it lazily so callers don’t normally need to.

Parameters

Returns

number

update()

update(
   token: string,
   state: TState,
bytes: number): SessionResult<SessionRecord<TState>>;

Replace the state of an existing session. Same lookup/error model as get, plus per-session-bytes enforcement on the new payload.

Parameters

Returns

SessionResult<SessionRecord<TState>>

WireProtoUnavailableError

Thrown when DocumentSnapshot._fieldsProto is unavailable or malformed. Per prerequisite 0.A, the wire reader does NOT silently fall back to data() — that would collapse integer/double at the value boundary and corrupt every downstream codegen consumer.

Extends

• Error

Constructors

Constructor

new WireProtoUnavailableError(opts: {
  docPath: string;
  reason: string;
}): WireProtoUnavailableError;

Parameters

Returns

WireProtoUnavailableError

Overrides

Error.constructor

Properties

Interfaces

CollectionGroupCapableFirestore

Optional source capability used by findCollectionGroup.

Methods

collectionGroup()

collectionGroup(collectionId: string): CollectionGroupQuery;

Parameters

Returns

CollectionGroupQuery

CollectionGroupQuery

Minimal collection-group query shape used by host discovery.

Methods

get()

get(): Promise<{
  docs: CollectionGroupSnapshot[];
}>;

Returns

Promise<{ docs: CollectionGroupSnapshot[]; }>

limit()

limit(n: number): CollectionGroupQuery;

Parameters

Returns

CollectionGroupQuery

select()

select(...fields: string[]): CollectionGroupQuery;

Parameters

Returns

CollectionGroupQuery

CollectionGroupSnapshot

Parent-path projection used by collection-group discovery.

Properties

CollectionSchema

Schema for a single collection as surfaced in the tool output’s finalizedSchemas.

Properties

ConvergenceResult

Properties

CrawlerCollectionRef

Minimal collection-reference shape needed for traversal.

Properties

Methods

listDocuments()

listDocuments(): Promise<CrawlerDocumentRef[]>;

Returns

Promise<CrawlerDocumentRef[]>

CrawlerDocumentRef

Minimal document-reference shape needed for traversal and sampling.

Properties

Methods

get()

get(): Promise<WireDocumentSnapshot>;

Returns

Promise<WireDocumentSnapshot>

listCollections()

listCollections(): Promise<CrawlerCollectionRef[]>;

Returns

Promise<CrawlerCollectionRef[]>

CrawlerFirestore

Firestore-shaped source consumed by the crawler.

collection and doc are needed only when resuming a continuation.

Methods

collection()?

optional collection(path: string): CrawlerCollectionRef;

Parameters

Returns

CrawlerCollectionRef

doc()?

optional doc(path: string): CrawlerDocumentRef;

Parameters

Returns

CrawlerDocumentRef

listCollections()

listCollections(): Promise<CrawlerCollectionRef[]>;

Returns

Promise<CrawlerCollectionRef[]>

CrawlOptions

Crawl options. All fields are optional with documented defaults.

• maxConcurrency: in-flight RPC cap. Default 32 per the Risk #1 sweep on 2026-05-05. Sweep over {4, 8, 16, 32, 64} against the corpus showed 4 → 8 → 16 → 32 → 64 was a steady ~10–15% per-doubling descent (no plateau). 32 was picked over 64 because the curve hadn’t flattened — 64 was the cap of the test range, not a true knee — and doubling in-flight RPCs again increases the chance of tripping per-project connection/quota limits in agent environments. Agents that want max speed can override.
• maxDepth: hard cap on BFS layers from the root. Defaults to 10 (well above any real-world Firestore tree). Used as a runaway guard, not an agent-facing knob.
• rootFilter: optional predicate on root collection IDs. Used by tests and the corpus harness to scope discovery to a known prefix without walking the entire database.

Properties

CrawlResult

Extended by

• FullCrawlResult

Properties

DiscoveredCollection

Per-template-path bookkeeping built up during a crawl. Collection refs are kept here for Item 2.3 to drive document sampling.

Multiple concrete collection paths may collapse to the same template path (e.g. users/uid_1/posts and users/uid_2/posts both map to users/{userId}/posts); their refs are accumulated under one entry.

Properties

DiscoverPathsToolResult

JSON-serializable shape returned by firestore_discover_paths.

Properties

DryRunCostEstimate

Heuristic cost projection returned by dryRun: true. The numbers are upper-bound estimates — agents should treat them as “no more than” figures, not exact predictions. Formulas are documented in-line so consumers can sanity-check.

Properties

EnumCandidate

Captured low-cardinality value set for enum-candidate fields.

Properties

FieldDescriptor

Per-field descriptor accumulated across a sampled stream.

• types is a deduped union; vector-dim drift keeps each dimension as a distinct entry by default (Phase 1.2 lock).
• presenceSeen / presenceTotal ratio drives presence-based agent UX; the denominator includes docs where the field was absent.
• nullable is an annotation, not a peer type (Phase 2.1 lock — null never appears as a FieldType in this descriptor’s types[]).
• enumCandidate populated when the field qualifies per Phase 3.2 lock (distinct ≤ 10 AND distinct ≤ samplesSeen / 2); otherwise undefined. Tracked only for scalar:string and scalar:integer/double; other kinds are not enum candidates.
• example is one observed non-null value per Phase 3.2 lock. Drives form placeholders, fixtures, README payloads. JSON-stringifiable (wire-typed primitives + arrays + plain-object maps).
• reservedReason populated by the wire layer when the field name matches a reserved-name pattern per 0.B; agents/codegen use it to skip or sanitize the field.

Properties

FieldObservation

A single field observation passed into mergeDoc. The wire layer (wire.ts) is responsible for producing this shape.

Properties

FieldSchema

Per-collection accumulated schema. samplesSeen is the doc count fed through mergeDoc, used as the presence denominator.

Properties

FindCollectionGroupHost

Properties

FindCollectionGroupOptions

Properties

FindCollectionGroupResult

Properties

FirestoreDiscoverToolDeps

Methods

resolveDb()

resolveDb(): CrawlerFirestore & CollectionGroupCapableFirestore;

Resolver returning the CrawlerFirestore to scan. Called per dispatch (F4). For firestore_find_collection_group the returned Firestore must also satisfy CollectionGroupCapableFirestore.

Returns

CrawlerFirestore & CollectionGroupCapableFirestore

FullCrawlResult

Result of a full crawl (structure + sampling). Augments CrawlResult with the per-templatePath finalized schemas the agent surface consumes.

continuation is present iff the crawl paused at a maxBatchBytes boundary; agents resume by calling crawl(db, { continuation }, sessions). complete is true iff the crawl finished — equivalent to continuation === undefined but more readable at call sites.

Counter fields (listOps, readOps) are cumulative across batches: a paused crawl returns the running total so the agent’s cost-reporting doesn’t have to do the bookkeeping.

Extends

• CrawlResult

Properties

PersistedCrawlState

JSON-serializable snapshot of an in-progress crawl. Stored in the session between batches. Refs (CollectionRef/DocumentRef) carry methods so they can’t be persisted directly — we serialize their paths and reconstruct via db.collection(path) / db.doc(path) on resume.

Phase invariants:

• structure phase: frontierPaths may be non-empty; samplingQueue is empty.
• sampling phase: frontierPaths is empty; samplingQueue lists the templatePaths still pending. Existing entries in finalizedSchemas are immutable across the rest of the crawl.

Properties

PersistedDiscoveredCollection

Persisted shape of a DiscoveredCollection (refs → paths).

Properties

SessionError

Properties

SessionRecord

A live session record. state is opaque to the store — Item 4.2 will instantiate SessionStore with the concrete crawler-state type.

Type Parameters

Properties

SessionStoreOptions

Properties

WireDocumentSnapshot

Minimal document snapshot shape needed for wire-type inference.

Properties

Type Aliases

DiscoverEvent

type DiscoverEvent =
  | {
  depth: number;
  kind: "collection_discovered";
  parentPath?: string;
  templatePath: string;
}
  | {
  changes: SchemaChange[];
  kind: "schema_updated";
  templatePath: string;
}
  | {
  declaredAt: number | null;
  kind: "sampling_complete";
  samplesSeen: number;
  samplingComplete: SamplingComplete;
  templatePath: string;
}
  | {
  code: string;
  kind: "error";
  message: string;
  templatePath: string;
};

Event stream emitted by firestore_discover_paths. Frozen enum per Phase 3.3 lock. Order within a batch is meaningful — agents may rely on collection_discovered arriving before schema_updated for the same path.

ExampleValue

type ExampleValue =
  | string
  | number
  | boolean
  | null
  | ExampleValue[]
  | {
[k: string]: ExampleValue;
};

A representative observed value, JSON-safe.

FieldPath

type FieldPath = ReadonlyArray<string | "[]">;

Path within a doc to a field. '[]' segment denotes array element scope, used for nested-array/map descriptors.

FieldType

type FieldType =
  | {
  kind: "scalar";
  type: FirestoreScalarType;
}
  | {
  kind: "reference";
  targetCollection: string;
}
  | {
  elementTypes: FieldType[];
  kind: "array";
}
  | {
  fields: Record<string, FieldDescriptor>;
  kind: "map";
}
  | {
  dimension: number | "mixed";
  kind: "vector";
};

One observed type for a field. Vector kept distinct from map per Phase 1.2 vector-sentinel lock; reference target is the template-form full path per Phase 3.1 lock (users/{userId}/posts, not posts).

FirestoreScalarType

type FirestoreScalarType =
  | "null"
  | "boolean"
  | "integer"
  | "double"
  | "timestamp"
  | "string"
  | "bytes"
  | "geopoint";

Firestore scalar wire types. Matches the discriminator emitted by _fieldsProto.<field>.valueType (Phase 0.1 lock).

null is represented as a scalar so descriptors can carry a single union shape; the merge layer separately tracks nullable per descriptor.

ReservedFieldReason

type ReservedFieldReason =
  | "firestore_reserved_name"
  | "dotted_field_name"
  | "numeric_field_name"
  | "double_underscore_wrap";

Why a field name is flagged reserved per 0.B.

SamplingComplete

type SamplingComplete =
  | "converged_via_stable"
  | "converged_via_exhausted"
  | "converged_via_max"
  | "sampling_open";

4-state classification for sampling termination per Phase 2.2 lock.

• converged_via_stable: hit stopOnStable consecutive no-change docs. Schema is probably complete — known false-negative is mid-stream drift later than stopOnStable docs into the stable region (out of scope per Phase 2.1 lock; deferred to a future firestore_re_crawl).
• converged_via_exhausted: iterator returned empty before maxSamples. Schema is provably complete — we read every doc.
• converged_via_max: hit maxSamples cap without converging. Schema may be incomplete; agent should treat with caution.
• sampling_open: crawl interrupted (continuation boundary). Resume with the returned continuation handle to keep sampling.

SchemaChange

type SchemaChange =
  | {
  kind: "field_added";
  path: FieldPath;
  type: FieldType;
}
  | {
  addedType: FieldType;
  kind: "type_expanded";
  path: FieldPath;
}
  | {
  kind: "presence_changed";
  path: FieldPath;
  presenceSeen: number;
  presenceTotal: number;
}
  | {
  kind: "enum_added";
  path: FieldPath;
  values: (string | number)[];
}
  | {
  addedValue: string | number;
  kind: "enum_widened";
  path: FieldPath;
}
  | {
  kind: "enum_dropped";
  path: FieldPath;
  reason: "over_threshold" | "type_widened";
}
  | {
  addedDimension: number;
  kind: "vector_dim_drift";
  path: FieldPath;
}
  | {
  kind: "became_nullable";
  path: FieldPath;
};

Frozen SchemaChange enum per Phase 5 implementation plan lock. Emitted by the merge layer; carried in schema_updated events.

Renamed from v1 scope’s ChangeReason to match the agent-facing terminology in the validation plan’s event-model lock.

SessionErrorCode

type SessionErrorCode =
  | "SESSION_EXPIRED"
  | "SESSION_EVICTED"
  | "SESSION_PAYLOAD_TOO_LARGE"
  | "SESSION_MALFORMED_TOKEN";

SessionResult

type SessionResult<T> =
  | {
  ok: true;
  value: T;
}
  | {
  error: SessionError;
  ok: false;
};

Discriminated-union result so callers don’t have to try/catch.

Type Parameters

WireValue

type WireValue = any;

Variables

DEFAULT_ENUM_THRESHOLD

const DEFAULT_ENUM_THRESHOLD: 10 = 10;

Default enum-candidate distinct-value cap (Phase 3.2 lock).

DEFAULT_MAX_SESSION_BYTES

const DEFAULT_MAX_SESSION_BYTES: number;

DEFAULT_MAX_SESSIONS

const DEFAULT_MAX_SESSIONS: 8 = 8;

DEFAULT_TTL_MS

const DEFAULT_TTL_MS: number;

Functions

classifyFieldName()

function classifyFieldName(name: string): ReservedFieldReason;

Classify a field name. Returns undefined for normal names, or a specific ReservedFieldReason for names that codegen must skip/sanitize.

Rules ordered by specificity (most specific first):

• exact __name__ etc. → firestore_reserved_name
• contains ’.’ → dotted_field_name (breaks dot-path access)
• pure-numeric → numeric_field_name (looks like array index)
• foo → double_underscore_wrap (sentinel collision)

Parameters

Returns

ReservedFieldReason

crawl()

function crawl(
   db: CrawlerFirestore,
   options?: CrawlOptions,
sessions?: SessionStore<PersistedCrawlState>): Promise<FullCrawlResult>;

Full crawl: discover structure, then sample up to maxSamples docs per discovered templatePath and feed them through the merge layer. Emits schema_updated events for every non-empty merge and sampling_complete once per templatePath.

Pause/resume (Item 4.2). When a SessionStore is supplied, the crawler measures the persisted-state size at two pause boundaries:

1. End of every BFS layer in the structure phase
2. End of every templatePath in the sampling phase

If the persisted state exceeds maxBatchBytes (default 1 MB), the crawler persists state and returns a continuation token. The agent resumes by passing { continuation } on the next call. Counters (listOps, readOps) are cumulative across batches; events are per-batch only (agents accumulate them themselves).

Without a SessionStore, the crawler runs to completion regardless of size — single-call mode is unchanged.

Continuation lifecycle. Continuation handles are minted/validated by the supplied SessionStore (see discover/session.ts). Malformed, expired, or evicted tokens surface as a single error event and an otherwise-empty result — agents can re-issue without continuation per the recovery hint.

Parameters

Returns

Promise<FullCrawlResult>

crawlStructure()

function crawlStructure(db: CrawlerFirestore, options?: CrawlOptions): Promise<CrawlResult>;

Walk the Firestore tree breadth-first. Each layer issues its listDocuments + per-doc listCollections calls in parallel under a shared concurrency cap.

Returns once every reachable collection (within maxDepth) is recorded. The returned discovered map is keyed by templatePath; events is the ordered event log emitted during the walk (currently only collection_discovered).

Parameters

Returns

Promise<CrawlResult>

createFirestoreDiscoverTools()

function createFirestoreDiscoverTools(deps: FirestoreDiscoverToolDeps): ToolHandler<unknown, unknown>[];

Parameters

Returns

ToolHandler<unknown, unknown>[]

decodeToken()

function decodeToken(token: string): {
  id: string;
};

Decode a disc_<base64url> token. Returns null on any malformation — caller maps null to SESSION_MALFORMED_TOKEN.

Parameters

Returns

{
  id: string;
}

id

id: string;

emptySchema()

function emptySchema(): FieldSchema;

Returns

FieldSchema

encodeToken()

function encodeToken(ulidBytes: Uint8Array): string;

Encode a 16-byte ULID into a disc_<base64url> token. Internal — used by SessionStore.create.

Parameters

Returns

string

fieldTypeKey()

function fieldTypeKey(t: FieldType): string;

Stable key for FieldType used in dedup. NaN/±Infinity all collapse to s:double so no special handling needed (Phase 1.2 lock).

Parameters

Returns

string

findCollectionGroup()

function findCollectionGroup(
   db: CollectionGroupCapableFirestore,
   collectionId: string,
options?: FindCollectionGroupOptions): Promise<FindCollectionGroupResult>;

Find every collection-group host of a given collection ID.

One read per returned doc — cost is bounded by limit (default 100). Returns the hosts in template-path form (e.g. users/{userId}/posts) with the per-host sample doc count.

Throws on Admin SDK errors (network / permission). The tool is standalone — no session, no continuation, no events — so error propagation is straightforward.

Parameters

Returns

Promise<FindCollectionGroupResult>

inferTemplateVariable()

function inferTemplateVariable(collectionId: string): string;

Convert a collection ID to the conventional template-variable name a Firestore rules author would write for its docs. Strips a trailing snake/dot-cased prefix word so ttt_lobbies → lobbyId (not ttt_lobbieId).

Parameters

Returns

string

mergeDescriptorWithObservation()

function mergeDescriptorWithObservation(
   prev: FieldDescriptor,
   observation: FieldObservation | "absent",
   newTotal: number,
   path: FieldPath): MergeFieldResult;

Merge a single field observation into an existing descriptor. observation === 'absent' means the field was missing from the doc. newTotal is the doc count after this observation.

Parameters

Returns

MergeFieldResult

mergeDoc()

function mergeDoc(prev: FieldSchema, doc: Record<string, FieldObservation>): {
  changes: SchemaChange[];
  next: FieldSchema;
};

Merge a single document’s typed field observations into a collection-level schema. Returns the next schema and the changes emitted.

The wire layer (wire.ts) is responsible for converting Firestore wire values into the Record<string, FieldObservation> shape expected here.

Parameters

Returns

{
  changes: SchemaChange[];
  next: FieldSchema;
}

changes

changes: SchemaChange[];

next

next: FieldSchema;

runConvergence()

function runConvergence(docs: Iterable<Record<string, FieldObservation>>, stopOnStable: number): ConvergenceResult;

Stream-driven convergence runner. Used by the production crawler’s sampling loop and by Phase 2.x tests that replay corpus snapshots.

stopOnStable is the optimistic early-exit signal (Phase 2.1 lock — must be paired with a maxSamples hard cap in the crawler, not here).

Parameters

Returns

ConvergenceResult

runWithLimit()

function runWithLimit<T, R>(
   items: readonly T[],
   limit: number,
producer: (item: T, index: number) => Promise<R>): Promise<R[]>;

Run items through producer concurrently, capping in-flight calls at limit. Returns results in input order (same shape as Promise.all(items.map(producer)) — but bounded).

producer may throw; the rejection propagates after in-flight work settles. Other items continue running so the rejection isn’t masked by a Promise.all-style fast-fail leaving zombie pending work.

Implementation detail: uses an internal Semaphore(limit); callers who need to share a permit pool across multiple runWithLimit calls (e.g. crawler global RPC cap) should use the Semaphore class directly via acquire/release.

Type Parameters

Parameters

Returns

Promise<R[]>

snapshotToObservations()

function snapshotToObservations(snap: WireDocumentSnapshot): {
  observations: Record<string, FieldObservation>;
  reservedNames: Record<string, ReservedFieldReason>;
};

Convert a Firestore document snapshot into a FieldObservation map. Detects reserved field names per 0.B — they appear in the output but the descriptor returned by the merge layer carries reservedReason so codegen can skip them.

Throws WireProtoUnavailableError if _fieldsProto is absent (0.A fail-loud contract). Empty docs (proto present but zero keys) return an empty record without throwing.

Parameters

Returns

{
  observations: Record<string, FieldObservation>;
  reservedNames: Record<string, ReservedFieldReason>;
}

observations

observations: Record<string, FieldObservation>;

reservedNames

reservedNames: Record<string, ReservedFieldReason>;

toTemplatePath()

function toTemplatePath(concretePath: string): string;

Map a concrete collection path to its template-path form per Phase 3.1 lock. Doc-id segments become {singular(parentColl)Id} so the result matches Firestore rules’ path.raw segments under typical naming conventions (TTT corpus verified: ttt_lobbies → {lobbyId}).

Inputs alternate coll/doc/coll/doc/.../coll. Length is always odd (a collection path ends on a collection segment).

Heuristic — agents needing strict alignment with rules should normalize both sides before joining. See Risk 6 in the implementation plan.

Examples: users → users users/uid_1/posts → users/{userId}/posts ttt_lobbies/abc/games/g1/moves → ttt_lobbies/{lobbyId}/games/{gameId}/moves

Parameters

Returns

string

wireValueToFieldType()

function wireValueToFieldType(v: any): FieldType;

Pure type extraction. Does not extract examples or enum samples — use wireValueToObservation for the full observation.

Parameters

Returns

FieldType

wireValueToObservation()

function wireValueToObservation(v: any): FieldObservation;

Full observation including JSON-safe example projection and enum sample extraction. The merge layer consumes this shape.

Parameters

Returns

FieldObservation