I'd heard that GPT-4's image analysis feature wasn't available to the public because it could be used to break Captcha.
Turns out it's true: The new Bing can break captcha, despite saying it won't:
Getting many comments about whether this is a new capability, so to clarify: solving such CAPTCHAs using ML has been possible for at least two decades.
For example, this NeurIPS paper described techniques to solve similar text-based CAPTCHAs… in 2004:
The trick to making CAPTCHA useful is to keep re-inventing tasks once earlier methods have been broken.
The figure shows how long it took to break various CAPTCHA schemes (in red) and which ones still haven't been broken (green):
(Image from: https://dl.acm.org/doi/pdf/10.1145/3477142)
Update: Image analysis on Bing is no longer available. (screenshots show yesterday vs. today)
Either MS disabled the rollout entirely.
Or they specifically removed my access. This would suck, because they are actively disincentivizing people from finding issues!