Skip to content

static-embed.js should export parquetRead so raw parquet files can be fetched without base64 #857

Description

@paddymul

Problem

static-embed.js is built as a self-executing ESM bundle (packages/js/static-embed.tsx) that auto-runs main() on load. It bundles parquetRead / parquetMetadata from hyparquet internally but does not re-export them.

This means there is no way to load raw parquet bytes from a file and decode them client-side when using the sidecar bundle. The only path today is:

Python → snappy parquet → base64 → inline JSON → JS b64decode → parquetRead

When serving files over HTTP (Mode 3b), you'd want:

Python → snappy parquet → raw file → fetch() ArrayBuffer → parquetRead

The raw-file path skips the ~33% base64 size overhead and lets you serve the data file separately from the HTML (better caching, smaller initial parse). But parquetRead is not accessible from outside the bundle.

Expected behaviour

static-embed.js exports parquetRead and parquetMetadata (already re-exported in buckaroo-js-core/src/index.ts) so a page can:

  1. fetch('data.parquet')ArrayBuffer
  2. const { parquetRead } = await import('./static-embed.js')
  3. Decode rows, inject as a plain array into the artifact, let static-embed's auto-init render it

Workaround

Import hyparquet separately from esm.sh CDN — but that adds a network dependency to what is supposed to be a fully-offline sidecar bundle.

Fix

In packages/js/static-embed.tsx, add:

export { parquetRead, parquetMetadata } from 'hyparquet';
// or re-export from buckaroo-js-core which already re-exports them
export { parquetRead, parquetMetadata, resolveDFDataAsync } from 'buckaroo-js-core';

The esbuild step (build:static) already produces ESM format (--format=esm), so named exports work fine.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions