Skip to content

feat(app,cli): camera + file/PDF attachments with HEIC/size normalization#1387

Open
jlixfeld wants to merge 1 commit into
slopus:mainfrom
jlixfeld:feat/attachments
Open

feat(app,cli): camera + file/PDF attachments with HEIC/size normalization#1387
jlixfeld wants to merge 1 commit into
slopus:mainfrom
jlixfeld:feat/attachments

Conversation

@jlixfeld

@jlixfeld jlixfeld commented Jun 12, 2026

Copy link
Copy Markdown

Summary

Extends the existing expImageUpload attachment pipeline (gallery images only today) to the full scope of #1319:

App (packages/happy-app)

  • Attachment source chooser — the picker button now offers Photo Library / Take Photo / Choose File on native (web keeps direct picking; paste/drag already worked)
  • Paste from clipboard (iOS/Android) — when the clipboard holds an image, the native chooser shows a Paste from Clipboard row, gated by Clipboard.hasImageAsync so it only appears when relevant (and stays silent until tapped — no premature iOS paste banner). Pasted bytes are staged to a cache file (the upload path needs a file:// URI) and run through the same normalize/thumbhash/upload pipeline as picked images. Web already had its own paste listener.
  • Camera capture via expo-image-picker launchCameraAsync + camera permission flow (NSCameraUsageDescription added)
  • File attachments via expo-document-picker (already a dependency, previously unused) — flows through the existing encrypted-blob upload and t:'file' session-event path unchanged
  • Image normalization before upload (attachmentNormalize.ts):
    • HEIC/unsupported formats → JPEG q0.9 — fixes silent HEIC drop: the CLI's magic-byte check (detectClaudeImageMime) skips HEIC with only a debug log today, so iPhone photos often never reach the model
    • Downscale only when long edge > 1568 px (the Claude vision API ceiling — larger is discarded server-side anyway, and >5 MB images are rejected outright)
    • PNGs stay PNG (lossless, keeps screenshot transparency); in-range supported images pass through untouched
  • i18n for all new strings across all 10 locales; settings label updated to “Attachments”

CLI (packages/happy-cli)

  • Attachment→content-block conversion extracted to attachmentContentBlocks.ts and extended beyond images:
    • %PDF- magic → document content block
    • text/* mime / known text extension / clean UTF-8 → fenced text block with filename
    • anything else → visible notice in the message ([Note: attachment(s) … were omitted]) instead of today's silent drop
  • Routing decisions are made on decrypted bytes, not the wire mimeType (iOS pickers lie)

Tests

  • attachmentNormalize.spec.ts — 7 cases for the normalization decision logic
  • attachmentContentBlocks.test.ts — 7 cases (image/PDF/text/extension-fallback/unsupported-notice/empty/HEIC)
  • pnpm typecheck clean in happy-app; happy-app suite 594 passed; pre-existing codex/integration failures in happy-cli are untouched by this PR (no src/codex files in the diff)

Test plan

  • Unit tests above
  • Manual on-device: gallery image, camera photo, PDF, text file, unsupported binary → notice
  • Manual on-device: copy an image elsewhere → open chooser → Paste from Clipboard row appears → attaches and uploads

Closes #1319. Related: #1270, #919, #70.

🤖 Generated with Claude Code

@jlixfeld

Copy link
Copy Markdown
Author

Heads-up for maintainers on the feature flag: the expImageUpload toggle in settings/features.tsx (around line 108 on main) is currently commented out with {/* Image upload hidden — broken, shipping next release */}.

This PR is aimed squarely at the "broken" part — it fixes the silent HEIC drop (the most common failure on iOS, since the gallery/camera hand back HEIC that the CLI's magic-byte check skips) and adds the camera + file/PDF sources. Once this lands, re-enabling that <Item> should be safe, so the feature becomes reachable without a code edit.

I deliberately did not un-comment it in this PR to keep the diff focused on functionality and let you decide the release timing. Happy to either flip it on here or leave it for a follow-up — your call.

Rebased onto upstream v1.1.10, which shipped its own image-attachment +
normalization pipeline. This now layers our unique additions on top of that
foundation instead of carrying a parallel one:

App (useImagePicker, built on upstream's normalizePickedAssetForUpload):
- takePhoto  — camera capture
- pickFiles  — document/file picker (any type)
- pasteImage — paste image from the system clipboard
- source chooser UI (Photo Library / Take Photo / Choose File / Paste)
- camera/photo/files permission strings (app.config.js) + translations

CLI (Claude path):
- attachmentContentBlocks: convert attachments to Claude content blocks —
  PDF -> document block, text -> fenced text block, images by magic byte;
  unknown types surface a visible notice.
- claudeRemoteLauncher wires attachments through.

Dropped vs the original PR: our standalone image-normalization modules
(attachmentNormalize*) — superseded by upstream's normalizer. Their only extra
behavior was client-side downscale >1568px, which the Claude vision API already
applies server-side.

Generated with [Claude Code](https://claude.ai/code)
via [Happy](https://happy.engineering)

Co-Authored-By: Claude <noreply@anthropic.com>
Co-Authored-By: Happy <yesreply@happy.engineering>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

App: add file/image uploads and Opus 4.7 xhigh effort option

1 participant