AI price analysis testing and unexpected challenges

What I worked on today

• Focused entirely on the AI price analysis job.
• The flow is working well, but testing revealed disappointing results.
• Price extraction functions correctly, but plan names are hallucinating and unreliable.
• Discovered that OpenAI doesn’t support WebP, forcing me to revert AWS Lambda processing back to PNG format.

Lessons learned

• A working flow doesn’t guarantee good results—fine-tuning is crucial.
• Image format compatibility matters when dealing with AI processing.
• AI models can still struggle with structured data interpretation like plan names.

Challenges faced

• AI hallucination in plan name extraction, leading to inaccurate data.
• Unexpected WebP compatibility issues, requiring additional conversion steps.
• Frustration in figuring out how to solve the AI accuracy problem.

What’s next

• Investigate alternative AI fine-tuning methods or data preprocessing to improve accuracy.
• Explore rule-based corrections to counteract AI hallucinations.
• Take a break to reset and approach the problem with fresh eyes.