Much better UX for identifying correct rotation:
- Shows ORIGINAL image scaled to fit the modal
- Draws GREEN bounding box showing the crop area
- Draws ORANGE rotated rectangle showing rotation effect
- Darkens everything outside the crop area (visual focus)
- Rotation slider updates the overlay in real-time
- Quick preset buttons for common angles
- Detailed debug logs showing all calculations
- Better styling: dark theme for readability
User can now visually see what will be extracted with the current
rotation angle. Report the angle that makes the label readable.
New DebugRotationPanel component allows testing different rotation angles
with live preview and detailed debug logs showing:
- Current rotation angle (slider + quick presets)
- Crop bounds and calculations
- Image dimensions at each step
- Final processed image preview
- Confidence score
Integrated into ItemDetailModal via wrench icon in header (only visible
when item has a photo). Helps identify correct rotation values for images
at any angle.
Debug panel shows:
- Original image info
- Crop bounds and resulting size
- Rotation angle and final canvas size
- Step-by-step processing logs
- Live canvas preview of result
Items often have multiple labels/zones at different orientations:
- Main label with part number, specs (PRIMARY)
- Vendor logos or small text (secondary)
- Barcodes at odd angles
Instruction: Always optimize rotation for the PRIMARY label (most text),
ignore secondary/vendor labels that conflict in orientation.
Examples added showing how to handle items with multiple label zones.
Real-world photos come at ANY angle - upside down, sideways, at weird
tilts. Stop assuming small angles. Instead:
- Measure CURRENT text orientation (horizontal/vertical/upside-down/tilted)
- Calculate rotation needed to make it READABLE in standard English
- Allow full -180° to +180° range
- No artificial limits or assumptions about how operator took the photo
Examples: vertical text → ±90°, upside-down → ±180°, tilted → measure tilt
This gives Gemini freedom to analyze real-world messy photos correctly.
Key changes:
- Explicitly state image has EXIF orientation stripped (raw/native state)
- Return ONLY the tilt angle, not orientation shifts
- Expect small angles (-45° to +45°), not large ones
- Add safety check: if angle > 45°, likely measurement error
- Simplify examples to show tilt-only measurement
This ensures Gemini returns consistent, sane rotation values regardless
of the item or how the photo was taken, without needing local offsets.
Backend receives the original blob which may have EXIF orientation metadata.
Strip it before processing to ensure backend analyzes the same raw image space
that Gemini analyzed (which had EXIF stripped before sending).
This ensures rotation_degrees are applied correctly to the same image state.
Instead of transforming coordinates after Gemini returns crop_bounds, strip EXIF
orientation from image before sending to Gemini. This ensures:
- Gemini analyzes the same raw image as our backend
- crop_bounds are in raw image coordinate space
- No coordinate transformation needed
- Works for all images (with or without EXIF)
Added strip_exif_orientation() utility that removes orientation tag and
re-encodes image. Used in extract_label endpoint before sending to Gemini.
Gemini analyzes images with EXIF orientation applied (e.g., 90° CW rotation),
returning crop_bounds in that coordinate space. We need to transform them back
to raw image coordinates before cropping.
For EXIF orientation 6 (Rotate 90 CW):
- Raw: 4032×3024 (landscape)
- Displayed (after EXIF): 3024×4032 (portrait)
- Transform portrait coords back to landscape before cropping
This fixes the issue where crop was applied to wrong image region.
Add logging to compare file sizes:
- [EXTRACT] sent to Gemini
- [CREATE_ITEM] received when creating item
This will reveal if image is being processed/changed between extraction and local processing.
Previous prompt only measured text baseline angle from horizontal (-45° to +45°),
missing that text can be vertical/sideways, requiring larger rotations (up to ±180°).
New guidance:
- Measure TOTAL rotation needed to make text horizontal and readable
- Account for both chassis tilt AND text orientation (horizontal vs vertical)
- Example: 22° tilt + 90° vertical text = 112° total rotation
- Allows full -180° to +180° range instead of limiting to ±45°
No changes to item identification fields, OCR rules, or crop bounds analysis.
Gemini analyzes raw image and returns crop_bounds for that coordinate space.
Previous code applied EXIF rotation first, changing image dimensions, then
used crop_bounds on the rotated image (coordinate mismatch).
Now: crop on raw image (matches AI) → then apply EXIF + manual rotation.
This ensures cropped region contains the actual item, not background.
- Add -9 (force kill) flag to all pkill commands
- Kill Python backend processes explicitly
- Kill npm and node processes
- Use fuser to kill processes bound to ports 8000, 3001, 3002, 3003
- Add 1 second wait after cleanup
- Ensures absolutely clean state before restart
- Cap normal rotation to -45 to +45 degrees (flags errors beyond)
- Add explicit warning about 60°+ being measurement error
- Clarify 'safe default' is 0 if uncertain
- Prevent Gemini from wild angle guesses (like 110°)
- Help AI self-correct measurement errors
- Add signal traps for SIGINT (Ctrl-C) and SIGTERM
- Cleanup function kills all child processes gracefully
- Script now exits cleanly without hanging
- Users can press Ctrl-C to stop all services at once
- Update photo save checkbox to match app design (border-slate-600, htmlFor label)
- Remove success popup overlay - modal closes immediately after save
- Simplify confirmSingleItem to remove setTimeout logic
- Toast notification used for user feedback instead
- Add savingIndex state to track saving operation
- Display success overlay with saved image for 1.5 seconds
- Show item name in confirmation message
- Prevents modal from closing immediately after save
Fixes user complaint about image disappearing too quickly without confirmation
- Move manual rotation before cropping and text detection
- Detect text orientation on full rotated image (not just cropped region)
- This allows text angle detection to see full context and properly orient labels
- Crop happens after orientation correction for cleaner results
- Delete photo_path and photo_thumbnail_path files on item deletion
- Handle file not found gracefully with logging
- Preserves audit logs while removing actual image files
- Add rotation_degrees parameter to ImageProcessor.process_photo()
- Pass rotation through _auto_save_photo_from_extraction() to processor
- Allow no-crop fallback when crop_bounds is None
- Add buildPhotoUrl() helper to resolve backend URLs correctly
- Update frontend components to use backend URL for image sources
- Replace Use/Skip Photo buttons with checkbox in AI extraction UI
- Add images/ to .gitignore to prevent accidental commits
Addresses: rotation never applied, image 404s (relative to Next.js not backend), preview blank in edit form
Documented the complete image pipeline feature:
- v1.14.1: Type system fix (photo_path fields)
- v1.14.2: Serialization fix (Blob to base64)
- v1.14.3: User control (image confirmation buttons)
Users now have full control over whether extracted photos are auto-saved
through the 'Use Photo' / 'Skip Photo' buttons in the editing form.
Users now see the extracted photo during item editing and can choose to:
- 'Use Photo': Auto-save the image with the item
- 'Skip Photo': Create item without saving the photo
Changes:
1. frontend/components/AIOnboarding.tsx: Added image preview panel in edit form
- Shows extracted image to user
- Buttons to accept/reject photo
- Visual feedback when photo is skipped
2. frontend/hooks/useAIExtraction.ts: Updated confirmSingleItem and confirmAllItems
- Respect user's photo decision (_skipPhoto flag)
- Only pass extractedImageBlob if user approved it
- Prevents unwanted auto-photo-save
This gives users full control over which extracted photos are auto-saved.
Documented the complete image pipeline bugfix:
- Part 1 (v1.14.1): Type system mismatch for photo_path fields
- Part 2 (v1.14.2): Blob serialization issue - image not reaching backend
The extracted image blob is now converted to base64 before sending to the API,
ensuring it's JSON-safe and matches the backend's ItemCreate schema.
The extracted image blob from AI extraction was not being sent to the backend
because Blob objects cannot be JSON serialized. Fixed by:
1. Converting Blob to base64 before sending to API
2. Renaming fields: extractedImageBlob → extracted_image_bytes, imageProcessing → image_processing
3. Removing Blob from local DB (keep original data structure for IndexedDB)
4. Applying same fix to both single item and batch update flows
This ensures the auto-photo-save feature receives the image data it needs.
Documented the image display issue, root cause (type system mismatch),
and the fix applied in v1.14.1. Images now correctly display from
the auto-photo-save feature in both ItemDetailModal and InventoryTable.
toast.warning is not a valid method in react-hot-toast API. Changed to
toast.success since the item was successfully created even if photo upload failed.
The backend returns photo_path, photo_thumbnail_path, and photo_upload_date
from the AI auto-save feature, but the frontend Item interface was missing
these fields, causing images not to display in ItemDetailModal and InventoryTable.
Updated:
- frontend/lib/db.ts: Added missing photo fields to Item interface
- frontend/components/ItemDetailModal.tsx: Use photo_path first, fallback to image_url
- frontend/components/InventoryTable.tsx: Use photo_path first, fallback to image_url
This ensures the UI can now display photos saved by the auto-photo-save feature.
- Updated confirmSingleItem() to include extractedImageBlob and imageProcessing
- Updated confirmAllItems() to pass image data for bulk item creation
- Each extracted item now carries its own image_processing metadata
- All items in bulk creation share the same extracted image blob
- Added 12 comprehensive tests verifying data is passed correctly
- All 465 frontend tests passing, zero regressions
- Extend ItemCreate schema with optional extracted_image_bytes (base64) and image_processing (dict)
- Update create_item endpoint to call _auto_save_photo_from_extraction after item creation
- Decode base64 image bytes and pass crop_bounds, rotation_degrees to helper
- Don't block item creation if photo save fails (log warning instead)
- Item returned with photo_path, photo_thumbnail_path populated if save succeeded
- Full backward compatibility: old clients without image fields work unchanged
- Add 5 integration tests covering all scenarios:
- Create item WITH image_processing → photo auto-saved
- Create item WITHOUT image_processing → no photo (backward compatible)
- Create item WITH invalid image_processing → item created, photo skipped
- Create item WITH crop_bounds=None → item created, photo skipped
- Create item WITH bytes but NO processing metadata → item created, photo skipped
- All 158 backend tests passing, zero regressions
