- AI Prompt: Detailed categorization to distinguish spare parts from consumables
- Search: Web scraping (requests + BeautifulSoup) to extract Google results, no API key
- Trigger: Automatic background search after AI extraction if category matches whitelist
- UX: Block until search completes, user reviews and edits all fields before save
- Mapping: Extract product type/specs/manufacturer/description to Notes field
- Retry on failure, user can skip if needed
Removed the rotation/zoom adjustment modal feature. Approach was too complex and
time-consuming without delivering stable results.
New simplified flow:
1. User extracts item with AI
2. Item saved with original image (resized/compressed by backend)
3. No rotation adjustments in frontend
4. Image editing can be implemented as a future feature
This unblocks the workflow and gets users to a working state.
Backend still applies compression/resizing, just no rotation processing.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Root cause: Image was being rotated TWICE:
1. Frontend rotated it in the modal and sent processed blob
2. Backend rotated it AGAIN based on rotation_degrees
This double rotation made the image appear unrotated or distorted.
Solution: Remove frontend image processing entirely.
- Modal now sends ONLY the rotation value
- No imageBlob from modal (uses original)
- Backend receives original image + rotation value
- Backend applies rotation ONCE
Now image is rotated correctly by backend without duplication.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
When rotating images, the rotated corners extend beyond the original canvas
bounds, causing the image to be clipped and appear skewed.
Fixed by:
1. Calculate canvas size needed to fit rotated image
2. Use formula: newSize = sqrt((w*cos)^2 + (h*sin)^2)
3. Center image and rotate around center of new canvas
4. Update cropBounds to new canvas dimensions
Now when user rotates an image, the full rotated result is saved without
any clipping or skewing. The saved image dimensions will be larger than
the original to accommodate the rotation.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Cropping UI was a placeholder and never fully implemented. Users couldn't
actually select a crop region - it always used full image bounds.
Simplified modal to focus on what works:
- Rotation adjustment ✅ (fully working)
- Zoom/pan for preview ✅ (fully working)
- Removed aspect ratio controls (crop not functional)
- Changed header to "Rotate Image"
Image processing now:
1. Takes user's rotation adjustment
2. Applies rotation to full image
3. No cropping (uses full bounds)
4. Saves rotated image
This ensures saved image matches the rotation user selected in modal.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Root cause found: setState is async, so hookConfirmSingleItem was called
before extractedItems updated, causing it to use OLD image_processing values.
Fixed by:
1. Building the final newItem directly in handleImageAdjustmentConfirm
2. Using the UPDATED image_processing values (with user adjustments)
3. Calling onComplete synchronously with correct values
4. Not relying on async setState ordering
This ensures backend receives the user-adjusted crop_bounds and rotation_degrees,
not the original AI-detected values.
Backend logs will now show user adjustments, not original AI values.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Added detailed console logging across entire image adjustment pipeline:
1. ImageAdjustmentModal.handleConfirm():
- Original image dimensions
- User inputs (rotation, zoom, pan, crop)
- Canvas processing steps
- Final blob size
2. AIOnboarding.handleImageAdjustmentConfirm():
- Adjustments received from modal
- Item being updated
- extractedImageBlob status before/after
3. useAIExtraction.confirmSingleItem():
- newItem being built
- extractedImageBlob attached
- imageProcessing attached
This will help identify where values are lost or incorrect in the flow.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Critical fix: modal processes image and returns blob, but AIOnboarding
was not actually using that blob. Now properly calls setExtractedImageBlob()
so confirmSingleItem sends the processed image to backend.
Before: Backend received original image, applied original AI crop/rotation
After: Backend receives processed image (already cropped/rotated by user)
Now the saved image matches exactly what user sees after adjusting.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Critical fixes:
1. Modal now applies rotation and crop to image, returns processed blob
2. Frontend sends processed image to backend (not original)
3. Zoom slider min/max now calculated based on image size
4. Initial zoom shows entire image in canvas
5. Zoom constraints prevent over-zooming or under-zooming
User experience improved:
- Sees full item at start (auto-fit zoom)
- Can adjust rotation/crop smoothly
- Adjusted image is what gets saved (not original)
- Zoom slider works correctly for any image size
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Fixes issue where image was too small to see full item in modal canvas.
Changes:
- Calculate initial zoom to fit entire image in 800x600 canvas
- Don't zoom in (only zoom out to fit), preserving image clarity
- Reset button also resets to fit zoom instead of always 1.0
This ensures users can see the full item from the start and understand
what they're adjusting.
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Fixed two critical issues:
1. Adjustments now properly override image_processing values
2. State updates explicitly ensure extractedItems are modified before confirm
Changes:
- handleImageAdjustmentConfirm now updates extractedItems state
- Adjustments properly stored in image_processing (rotation_degrees, crop_bounds)
- user_adjusted flag added to mark user-modified values
- Backend will use adjusted values instead of AI detection
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
Added user-controlled image adjustment modal that displays after item
confirmation, allowing users to adjust rotation/crop before final save.
Changes:
- AIOnboarding: wrapper confirmSingleItem to show modal post-selection
- State: showImageAdjustment, adjustingItemIndex, pendingItemData
- Handlers: confirm/cancel for applying or discarding adjustments
- Flow: item save → modal display → adjustments applied → DB commit
Co-Authored-By: Claude Haiku 4.5 <noreply@anthropic.com>
- Document AI non-determinism finding (critical discovery)
- Record ImageAdjustmentModal completion
- Detail integration checklist for next session
- Explain new user-controlled image adjustment workflow
- All commits and features documented for continuity
- Shows original image after item save
- Rotation slider + gesture rotation
- Zoom control (scroll/pinch)
- Pan support (drag on desktop/mobile)
- Preset aspect ratios for crop
- Touch support for mobile (pinch zoom, drag pan)
- Checkbox to confirm/reject image use (default ON)
- Compresses image if adjusted before saving
- Shows which edge is Top, Bottom, Left, Right on crop rectangle
- Removes ambiguity about crop box orientation
- Green labels on all four edges for clear visual reference
- Increase padding from 10-15px to 20-30px for context
- Emphasize ENTIRE item must be visible, not tight crop
- Add examples of correct vs wrong crop detection
- Address findings from debug panel testing: AI was cropping too tightly
- Add drag-to-move functionality for entire crop box
- Add resize handles at corners (drag to resize)
- Show adjusted bounds separately from AI bounds in log
- Add Reset button to revert to original crop bounds
- Allows visual verification of AI crop detection accuracy
- Add TOP/BTM labels to rotated box showing which edge is which after rotation
- Allows user to visually see rotation direction without calculating degrees
- Fix allowedDevOrigins to include common private IP ranges (192.168.*, 10.*, 172.16.*)
- Resolves CORS warning when accessing from local network IPs
- Display UP/DOWN/LEFT/RIGHT labels at canvas edges
- Shows which direction is which in the original image
- Helps verify rotation correctness visually
- Removed black overlay that was hiding the image
- Draw crop box directly on full-brightness image with drop shadow
- Item under crop area now fully visible
- Improved visibility of both crop bounds (green) and rotation (orange)
- Clear logs between rotation updates (no more expanding text)
- Increase canvas size from 600x450 to 900x600 for better visibility
- Reorganize layout: left controls, right canvas + single-line log
- Make crop box much larger and easier to see
- Save original image before crop/rotation with '_debug_original' variant
- Store original_photo_path in labels_data.image_processing for debug access
- Update DebugRotationPanel to display original image instead of processed
- Update ItemDetailModal to pass original image path to debug panel
- Enables accurate crop/rotation visualization with true original image
Layout changes:
- Reduced padding and margins throughout
- Changed from 3-column to 2-column layout
- Canvas + logs on right side (side-by-side)
- Smaller fonts and spacing
- Sticky header that stays in view
- Max-height with scrolling for modal
- Removed footer text to save space
- More efficient use of screen real estate
Now fits on most screen sizes even with large rotation angles.
Much better UX for identifying correct rotation:
- Shows ORIGINAL image scaled to fit the modal
- Draws GREEN bounding box showing the crop area
- Draws ORANGE rotated rectangle showing rotation effect
- Darkens everything outside the crop area (visual focus)
- Rotation slider updates the overlay in real-time
- Quick preset buttons for common angles
- Detailed debug logs showing all calculations
- Better styling: dark theme for readability
User can now visually see what will be extracted with the current
rotation angle. Report the angle that makes the label readable.
New DebugRotationPanel component allows testing different rotation angles
with live preview and detailed debug logs showing:
- Current rotation angle (slider + quick presets)
- Crop bounds and calculations
- Image dimensions at each step
- Final processed image preview
- Confidence score
Integrated into ItemDetailModal via wrench icon in header (only visible
when item has a photo). Helps identify correct rotation values for images
at any angle.
Debug panel shows:
- Original image info
- Crop bounds and resulting size
- Rotation angle and final canvas size
- Step-by-step processing logs
- Live canvas preview of result
Items often have multiple labels/zones at different orientations:
- Main label with part number, specs (PRIMARY)
- Vendor logos or small text (secondary)
- Barcodes at odd angles
Instruction: Always optimize rotation for the PRIMARY label (most text),
ignore secondary/vendor labels that conflict in orientation.
Examples added showing how to handle items with multiple label zones.
Real-world photos come at ANY angle - upside down, sideways, at weird
tilts. Stop assuming small angles. Instead:
- Measure CURRENT text orientation (horizontal/vertical/upside-down/tilted)
- Calculate rotation needed to make it READABLE in standard English
- Allow full -180° to +180° range
- No artificial limits or assumptions about how operator took the photo
Examples: vertical text → ±90°, upside-down → ±180°, tilted → measure tilt
This gives Gemini freedom to analyze real-world messy photos correctly.
Key changes:
- Explicitly state image has EXIF orientation stripped (raw/native state)
- Return ONLY the tilt angle, not orientation shifts
- Expect small angles (-45° to +45°), not large ones
- Add safety check: if angle > 45°, likely measurement error
- Simplify examples to show tilt-only measurement
This ensures Gemini returns consistent, sane rotation values regardless
of the item or how the photo was taken, without needing local offsets.
Backend receives the original blob which may have EXIF orientation metadata.
Strip it before processing to ensure backend analyzes the same raw image space
that Gemini analyzed (which had EXIF stripped before sending).
This ensures rotation_degrees are applied correctly to the same image state.
Instead of transforming coordinates after Gemini returns crop_bounds, strip EXIF
orientation from image before sending to Gemini. This ensures:
- Gemini analyzes the same raw image as our backend
- crop_bounds are in raw image coordinate space
- No coordinate transformation needed
- Works for all images (with or without EXIF)
Added strip_exif_orientation() utility that removes orientation tag and
re-encodes image. Used in extract_label endpoint before sending to Gemini.
Gemini analyzes images with EXIF orientation applied (e.g., 90° CW rotation),
returning crop_bounds in that coordinate space. We need to transform them back
to raw image coordinates before cropping.
For EXIF orientation 6 (Rotate 90 CW):
- Raw: 4032×3024 (landscape)
- Displayed (after EXIF): 3024×4032 (portrait)
- Transform portrait coords back to landscape before cropping
This fixes the issue where crop was applied to wrong image region.
Add logging to compare file sizes:
- [EXTRACT] sent to Gemini
- [CREATE_ITEM] received when creating item
This will reveal if image is being processed/changed between extraction and local processing.
Previous prompt only measured text baseline angle from horizontal (-45° to +45°),
missing that text can be vertical/sideways, requiring larger rotations (up to ±180°).
New guidance:
- Measure TOTAL rotation needed to make text horizontal and readable
- Account for both chassis tilt AND text orientation (horizontal vs vertical)
- Example: 22° tilt + 90° vertical text = 112° total rotation
- Allows full -180° to +180° range instead of limiting to ±45°
No changes to item identification fields, OCR rules, or crop bounds analysis.
