Rendered at 22:43:27 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
waldrews 50 minutes ago [-]
The VLM's are so good at complex document understanding now. But you just can't trust them not to invisibly censor sensitive clinical/legal docs, even at the maximally permissive settings.
And the deep learning OCR-only models won't censor, but can and do hallucinate. I've yet to see a 'scan with different approaches and reconcile and say you're not sure if they don't agree' system just work for generic complex documents.
ComputerPerson 5 hours ago [-]
I've got a scan from a book that I OCR with new releases. Ligatures, critical sigla, Fraktur letterforms, subscripts, superscripts, etc.
Nothing special about this model for overly-detailed work like mine.
It's been a while since I last tested (and discontinued my subscription), but the "pro" models from OpenAI dominate. Not surprising, given the price difference, but it would be nice if an OCR-specific model could perform better. It's worth mentioning that even the highest-end models do a pretty poor job with intricate text like mine.
razemio 2 minutes ago [-]
I made a benchmark for handwriting recognition for a project while keeping line breaks and errors (grammar, spelling). Sonnet absolutely dominates it since a good half a year. 5.6 did not change that for me. This should also translate to better ocr.
SyneRyder 2 hours ago [-]
While I haven't tried OpenAI for OCR, I've put my small scale OCR work through both Claude and Mistral OCR. Claude is absolutely better - even in OCR work I did last week and compared with Mistral OCR 4.0.
Mistral's one advantage is that Anthropic now flags OCR, because they don't allow anything that could be considered "reproduction", even of work for which you own the copyright. So my new workflow is Mistral OCR for the actual OCR, followed by a proofreading pass by Claude (which is allowed). Claude is obviously more expensive, but it caught entirely hallucinated sentences created by Mistral OCR 4.0, so I was glad for the backup check.
Oras 2 hours ago [-]
Same company that OCRed millions of books, the irony.
I feel Anthropic is destroying itself with all these restriction. They got away because their models were the best for coding, but that is not an advantage anymore as OpenAI and other open source are already better.
jassyr 10 minutes ago [-]
>Anthropic now flags OCR
I haven't seen any difference in my ocr workflows, what do you mean by this?
kergonath 3 hours ago [-]
I have been quite happy with Mistral OCR for the documents I needed to process (typeset, but old, with questionable scan quality, sometimes elaborate typesetting or, much worse, typewriter-and-handwriting approximations of it). I do not test every new model when they are released, but I did a review shortly after Mistral OCR 3 was released and it was a very good compromise: cheap, fast, and good results without further processing. I found generalist models to be way too much faf to get them to avoid unnecessary modifications to the text and report accurate bounding boxes for figures and tables.
That said, models have sometimes surprising weaknesses and a model could be terrible overall but magically work for one type of document.
kmitz 4 hours ago [-]
I got the opposite experience very recently : tried to OCR a bunch of handwritten emails addresses with chatGPT and I had to make so many corrections that I gave up. Whereas Mistral nailed it on first pass.
rtaylorgarlock 5 hours ago [-]
Yet: how is pricing?
Evaluating contents and routing appropriately isn't a new challenge in OCR, one of the oldest fields of applications in ML.
Thus, how do the smaller open models perform in tandem with relatively pricy $/pg models & APIs?
Your use case is remarkably rare relative to the volume and price sensitivity of enterprise data warehouse ops.
giancarlostoro 2 hours ago [-]
The big difference is traditional OCR used basic pattern matching to find text, whereas models like Mistral OCR (and GPT, etc) use computer vision instead and deep learning to parse text, math equations, and apparently in some cases extract images too.
I'd love to see some advancements in traditional OCR based on ideas and concepts we've learned from newer "OCR-like" models since traditional OCR is drastically cheaper.
4 hours ago [-]
petcat 5 hours ago [-]
> the "pro" models from OpenAI dominate. Not surprising considering the price difference, but it would ne nice if an OCR-specific model could do better.
I haven't been impressed with any of Mistral's models. They obviously realized that they couldn't compete at the frontier so they decided to go for smaller focused models but even those have not been that good.
booi 3 hours ago [-]
This is what I found as well.
We moved away from Cursor but I was looking for a model that would help with FIM (fill-in-middle) multiline autocompletion and people were recommending Mistral's Codestral. We gave it a shot and it was lackluster at best.. Even Google's Gemini did a significantly better job than Codestral.
Ultimately Opus-class models got good enough and I don't do much manual coding anymore.
giancarlostoro 2 hours ago [-]
So is Mistral OCR the best one? Have any other OCR models caught some of what you describe? I've been kind of interested in how "OCR" type models work compared to old school OCR.
ComputerPerson 2 hours ago [-]
My use case isn't in the realm of old-school OCR, so it's not a good comparison, but anyway:
As another user pointed out, it's surprisingly random (task-specific). Llama Scout outperformed Gemini Flash 2.5 on a benchmark I built at the time. I didn't include an OCR models.
Mistral might indeed be the best OCR-specific model for my task, now that you ask. Funny. It's so bad at my work that I didn't register it might be the best in its category. This is just based on vibes from my single scan.
bugglebeetle 2 hours ago [-]
The datalab models are the best ones.
josu 2 hours ago [-]
Whats the best open OCR at the moment?
king_crimson 5 hours ago [-]
At this point I lost all hope for Europe playing any significant role in the AI race. If that’s a good or a bad thing I don’t know, but it seems to me like that’s the reality.
hadlock 2 hours ago [-]
Much like spaceflight, aerospace or nuclear engineering, you need to retain local talent for national defense purposes. Being 70% as good is still way better than being 100% dependent and heavily leveraged by your opponents.
kubb 5 hours ago [-]
It's not a race. You don't get anything for winning.
BenzeneDream 4 hours ago [-]
? It absolutely is a race. Whether thats a positive thing or not is debatable but every lab is definitely in a race. What prize do you win? Imagine a world where only one country has AGI/ASI. Or a world where Europe only gets access to frontier models 6 months later. Far from ideal.
ux266478 3 hours ago [-]
> What prize do you win? Imagine a world where only one country has AGI/ASI.
"Winning the race" doesn't give you that in any meaningful capacity. It gives you, at the absolute most, a temporary window where that's the case. See: nuclear weapons.
booi 2 hours ago [-]
Citing nuclear weapons isn't the flex you think it is. There are only 9 countries that have nuclear weapons and they absolutely flex this power over non-nuclear powers (see Ukraine, Germany, SE Asia etc..)
Europe already has an innovation problem that's already causing structural economic instabilities which Germany has been struggling (and lately failing) to prop up.
As much as it pains me to say this, AI is already a tech revolution and it seems like Europe is just ignoring it. There's more innovation in 3 blocks in downtown San Francisco than the entire continent of Europe.
ux266478 1 hours ago [-]
> There are only 9 countries that have nuclear weapons
QED being first didn't grant exclusivity.
The premise wasn't that AGI isn't useful. There are actually layers to the metaphor where first-mover advantage of AGI is even less meaningful than it was for nuclear weapons, but I leave those as an exercise to the reader to discover.
fn-mote 54 minutes ago [-]
> first-mover advantage of AGI is even less meaningful
This is your opinion. Lots of tech money appears to disagree.
You should at least try to explain your contrarian position, or cite your favorite source that makes an argument that you believe.
xeromal 1 hours ago [-]
Russia invaded Ukraine specifically because they have nukes. Everyone is afraid to fight back.
jr3592 3 hours ago [-]
As if time isn't money? Everything is temporary... Getting somewhere first has immense value.
ux266478 3 hours ago [-]
A reductive equation, economics isn't thermodynamics. Money is fictional and value is subjective and unstable. Within this context, being first to AGI means nothing if the second invention of it comes 2 months later and works an order of magnitude faster than what the first iteration had self-improved to at that point in time. First mover advantage isn't decisive, you have to actually be able to capitalize on it in a robust way.
fn-mote 51 minutes ago [-]
> works an order of magnitude faster
This is the weakest part of the argument. Absolutely no reason to believe the second inventor will be 10x faster.
Does this ever happen? Even in traditional manufacturing, the second “inventor” starts behind and has to improve their own process to surpass the first.
799tppp 18 minutes ago [-]
sophist nonsense right here
just pure drivel from the mind of an insect
vrganj 3 hours ago [-]
This presupposes AGI or ASI are a real, reachable thing. My read is they might be, but not as LLMs. Until there's a fundamental rearchitecture, I'm AGI-agnostic and given that view, it doesn't seem rational to bet the house on it.
We'll see.
InsideOutSanta 2 hours ago [-]
Well, you get hundreds of billions in debt, and then you get open weight models distilling your proprietary models.
Palpatineli 4 hours ago [-]
How about "being able to align ASI somewhat to your values"?
ben_w 4 hours ago [-]
True, but that isn't EU/US/China, it's OpenAI/Anthropic/Grok/ …/DeepMind (based in UK)/… DeepSeek
With a lot of Chinese nationals in American companies, and an American corporation owning DeepMind, and a lot of people very upset with all of them at the same time, this is very messy.
procgen 5 hours ago [-]
The only prize is control of the light cone.
1 hours ago [-]
tedggh 3 hours ago [-]
A race to the bottom.
bpodgursky 5 hours ago [-]
It's red queen. You stay alive by winning, you lose everything by losing.
ben_w 4 hours ago [-]
Not sure you get either outcome in either case.
Race dynamics increases p(doom) for everyone.
The non-doom scenarios include "utopia for all", and "power flows to investors, not citizens of whichever nation the winning model's corp. was registered in".
Independently, "oh look all the investors went bankrupt" can happen in both "doom" and "normal technology" timelines.
ChrisClark 4 hours ago [-]
Unless you manage to build a god, and keep it under control... okay, we're all going to lose
pbkompasz 3 hours ago [-]
Not being a rat in the rat race is the real win.
missedthecue 1 hours ago [-]
The largest non-US non-China model is Russian which surprised me.
sgt 1 hours ago [-]
Are you saying Russia is winning over Europe in AI already? Dang, sanctions really don't work.
NicuCalcea 43 minutes ago [-]
Largest by what metric? And which model is that?
deadbabe 57 minutes ago [-]
Even Africa will surpass Europe with its massive data center upstarts breaking ground.
thadt 5 hours ago [-]
Yeah? And here I've been a happy Transkribus customer for some time now. If there are better models or interfaces out there for analyzing historical handwriting, I'll definitely take a look.
t3hTao 2 hours ago [-]
[dead]
onetwig 2 hours ago [-]
Huh? Mistral 7b was pioneering in its day and IMO they have been very on top of releasing niche useful models like moderation, OCR, etc.
I’m glad Mistral is working on useful solutions.
OpenAI/Anthropic is like a retarded little sibling chasing “AGI” and giving up on rich media and other modalities.
OpenAI/Anthropic is the worst of the mainstream AI.
It goes:
1. Gemini
2. Vidu
3. Le Chat (Mistral)
4. DeepAI
5. [insert MiniMax provider]
fn-mote 48 minutes ago [-]
Totally fake post. Gemini in the top?? For what?
ks2048 2 hours ago [-]
Does anyone know a site that lets you browse examples of input / output pairs?, particularly with layout analysis (bounding boxes of figures, tables, etc).
merb 5 hours ago [-]
1000 Pages / 3.5€ this is expensive as hell.
If this is not fastly superior than something like tesseract it is not worth it.
Oras 2 hours ago [-]
Even comparing to AWS Textract or Azure Document Intelligence, this is very expensive (more than double)
beernet 4 hours ago [-]
Agreed. Does the GTM team there really sit together like "oh yeah, that sounds reasonable" while being totally beyond typical market prices?
piterrro 4 hours ago [-]
For anyone interested, I have an ocr pipeline running on rented GPUs, doing around 1000pages for 0.05-01 usd with around 0.8 seconds per page with full bounding boxes support for grounding.
If you’re interested you can find contact to me via this profile.
3.5 usd/1000 pages is just too expensive…
aliljet 4 hours ago [-]
Accuracy is truly what people die for in the OCR game. Price isn't the primary function here.. it's an equation of price, accuracy, speed, and in mayn cases regulation.
merb 4 hours ago [-]
Tbf even with tesseract you already get shit ton of accuracy and you can probably do these 1000 pages for way less than 3.5€. For 3.5€ you can spin up a cloud instance with 8vCPU+32gb on gcloud for 11 hours (or 11 instances for an hour) which can do way more than 1000 pages per hour on tesseract. It takes you around 6 second per page +-4 seconds start/stop depending on what you are doing on that instance size without too much optimization (you can probably even run multiple processes on a single node)
Google documentai costs 1.5$ per 1000 which is probably better in quality and speed.
kergonath 3 hours ago [-]
Tesseract is not a substitute for these models, which understand complex layouts and also extract bounding boxes for things like tables and pictures. They are also much better at making sense of cursive scripts.
I’ve been there, implementing a way to linearise text from a document with pages with 1, 2 or 3 columns, some of them in landscape is a nightmare. And that’s not even considering equations.
In the end it’s way easier to use a specialised model, trained by other people to do exactly what I need.
tjoff 2 hours ago [-]
Tesseract is super picky though often failing pixel-perfect screenshots...
So, maybe it can be tuned for your usecase but with that kind of investment €3.5 for 1000 pages is a bargain...
merb 1 hours ago [-]
It‘s also cheaper than the murican ones from Google, Amazon, …. And tesseract was an example. Heck you can go xberg and use paddleocr. Most often layout is less of a problem for ocr. Most often you need high accuracy, which tools like these are often worse in the 95 percentile.
piterrro 4 hours ago [-]
Most use cases dont need that kind of accuracy, just doesnt justify the 3-4usd range. I build for that exact case (tender documents, we’re processing north of 100k pages per day), it doesnt need to recognize scanned written text from 1930s, its usually pdf/docs/scanned printed pages.
The accuracy is great, bounding boxes are must have for proper grounding for building answers by LLMs. Tesseract was too slow and not enough in some cases (for example tables or images which we also recognize and describe)
pogue 3 hours ago [-]
If you're getting inaccurate results from OCR what's the purpose of even doing it? Inaccuracy of text of any kind seems like a completely obvious failure of the entire purpose of scanning text into a computer.
fluoridation 3 hours ago [-]
It depends on what you need. For example a while ago I scanned and OCR'ed a bunch of receipts to get a timeline of my salary. I only cared about the gross and net figures, and nothing else mattered. Tesseract's output had a bunch of errors and misdetections, but the main figures always came out OK, and a local LLM was able to pick them out from the noise every time.
There's a big gulf between "it's as if a human being had transcribed it and reconstructed the original document" and "so completely broken it can't be used for anything".
piterrro 3 hours ago [-]
Accuracy can have different dimensions, depends on what you can tolerate and whether you can detect it to apply more powerful methods.
Imagine you have a cheap and 99% accurate ocr. The other 1% you can detect and apply more powerful (more accurate but slower and more expensive) ocr method.
What would you use? At scale these things add up.
Telemakhos 3 hours ago [-]
Can it produce accessible PDF files that will pass accessibility tests? Someone who can do that will make a killing laundering PDFs for academia: by April 26, every PDF, syllabus, and academic document needs to comply with WCAG 2.1 Level AA, which means structural tagging, alt text, and lots of other checklist items that AI could probably generate.
to21as 2 hours ago [-]
[flagged]
x3ro 4 hours ago [-]
You should put contact details in your profile :)
vrganj 3 hours ago [-]
Is it European-hosted and fully outside of both CLOUD Act and CCP reach?
Because I'm assuming that's why they get to charge more for the right type of customer.
piterrro 3 hours ago [-]
You can even run it on your desk if you want, a single gtx 4090 is enough. It can be fully air gapped.
ianhawes 5 hours ago [-]
I won't comment on accuracy, but in internal benchmarks, Mistral OCR is significantly faster than comparable APIs.
oliveralbertini 2 hours ago [-]
I'm wondering if this model performs better on french (and other european languages) documents than others
maelito 2 hours ago [-]
Given the latest vibe release's new "follow default" model option, we should see a new coding / general Mistral model, mistral 4, soon.
Johnny_Bonk 5 hours ago [-]
How does this compare to Baidu Unlimited OCR. I've been very impressed with Baidu and it's essentially free to run on a decent computer, other than electricity costs.
spiderfarmer 4 hours ago [-]
Where do your documents go?
rescbr 4 hours ago [-]
They go to the decent computer hosting the model, which can be yours if you pay the electricity costs
ad_fontes 4 hours ago [-]
I've been experimenting with using NuExtract this week on locally OCRing bank statements that don't have a predefined document structure. It's way better than Tesseract or a generic vision-enabled model. It runs great on a single RTX 4090 at the modest throughput I need.
Their hosted, API-based service is something like a third of the cost of this model.
parhamn 3 hours ago [-]
Mistral is bumping the price of this thing every release. I think we're at 2x now?
maz1b 5 hours ago [-]
How does this compare to 4?
tethys 55 minutes ago [-]
Who the hell at Mistral thinks it is a good idea to register CMD + T as a shortcut for switching theme!?
sajithdilshan 48 minutes ago [-]
it's control + T right?
mainecoder 5 hours ago [-]
The chinese did it better, mistral is alive thanks to regulations.
Bombthecat 3 hours ago [-]
Yeah, I'm not sending personal bills etc to china. No thanks
gkbrk 2 hours ago [-]
It's still open-weight models that you can download and run locally. You don't need to send anything to China.
sajithdilshan 47 minutes ago [-]
and you trust the french?
maelito 3 hours ago [-]
No it's the other way round : the Chinese do better thanks to regulations : massive amounts of money from Big tech and public money.
mangecoeur 4 hours ago [-]
i.e. it's one AI company that's basically guaranteed to never fail since it has a market niche guaranteed by European companies and governments.
petcat 4 hours ago [-]
Which is also why their most recent model "Shieldstral" does nothing except monitor and moderate internet content.
After stuff like Chat Control I think they're obviously seeing a big demand for this kind of "internet safety" technology in Europe.
rtaylorgarlock 5 hours ago [-]
I've been a bit more careful about complaining about regulations broadly due to competitive advantage, e.g. ITAR
t3hTao 2 hours ago [-]
[dead]
hmokiguess 4 hours ago [-]
Whoever is paying all that for OCR is being scammed.
And the deep learning OCR-only models won't censor, but can and do hallucinate. I've yet to see a 'scan with different approaches and reconcile and say you're not sure if they don't agree' system just work for generic complex documents.
Nothing special about this model for overly-detailed work like mine.
It's been a while since I last tested (and discontinued my subscription), but the "pro" models from OpenAI dominate. Not surprising, given the price difference, but it would be nice if an OCR-specific model could perform better. It's worth mentioning that even the highest-end models do a pretty poor job with intricate text like mine.
Mistral's one advantage is that Anthropic now flags OCR, because they don't allow anything that could be considered "reproduction", even of work for which you own the copyright. So my new workflow is Mistral OCR for the actual OCR, followed by a proofreading pass by Claude (which is allowed). Claude is obviously more expensive, but it caught entirely hallucinated sentences created by Mistral OCR 4.0, so I was glad for the backup check.
I feel Anthropic is destroying itself with all these restriction. They got away because their models were the best for coding, but that is not an advantage anymore as OpenAI and other open source are already better.
I haven't seen any difference in my ocr workflows, what do you mean by this?
That said, models have sometimes surprising weaknesses and a model could be terrible overall but magically work for one type of document.
I'd love to see some advancements in traditional OCR based on ideas and concepts we've learned from newer "OCR-like" models since traditional OCR is drastically cheaper.
I haven't been impressed with any of Mistral's models. They obviously realized that they couldn't compete at the frontier so they decided to go for smaller focused models but even those have not been that good.
We moved away from Cursor but I was looking for a model that would help with FIM (fill-in-middle) multiline autocompletion and people were recommending Mistral's Codestral. We gave it a shot and it was lackluster at best.. Even Google's Gemini did a significantly better job than Codestral.
Ultimately Opus-class models got good enough and I don't do much manual coding anymore.
As another user pointed out, it's surprisingly random (task-specific). Llama Scout outperformed Gemini Flash 2.5 on a benchmark I built at the time. I didn't include an OCR models.
Mistral might indeed be the best OCR-specific model for my task, now that you ask. Funny. It's so bad at my work that I didn't register it might be the best in its category. This is just based on vibes from my single scan.
"Winning the race" doesn't give you that in any meaningful capacity. It gives you, at the absolute most, a temporary window where that's the case. See: nuclear weapons.
Europe already has an innovation problem that's already causing structural economic instabilities which Germany has been struggling (and lately failing) to prop up.
As much as it pains me to say this, AI is already a tech revolution and it seems like Europe is just ignoring it. There's more innovation in 3 blocks in downtown San Francisco than the entire continent of Europe.
QED being first didn't grant exclusivity.
The premise wasn't that AGI isn't useful. There are actually layers to the metaphor where first-mover advantage of AGI is even less meaningful than it was for nuclear weapons, but I leave those as an exercise to the reader to discover.
This is your opinion. Lots of tech money appears to disagree.
You should at least try to explain your contrarian position, or cite your favorite source that makes an argument that you believe.
This is the weakest part of the argument. Absolutely no reason to believe the second inventor will be 10x faster.
Does this ever happen? Even in traditional manufacturing, the second “inventor” starts behind and has to improve their own process to surpass the first.
We'll see.
With a lot of Chinese nationals in American companies, and an American corporation owning DeepMind, and a lot of people very upset with all of them at the same time, this is very messy.
Race dynamics increases p(doom) for everyone.
The non-doom scenarios include "utopia for all", and "power flows to investors, not citizens of whichever nation the winning model's corp. was registered in".
Independently, "oh look all the investors went bankrupt" can happen in both "doom" and "normal technology" timelines.
I’m glad Mistral is working on useful solutions.
OpenAI/Anthropic is like a retarded little sibling chasing “AGI” and giving up on rich media and other modalities.
OpenAI/Anthropic is the worst of the mainstream AI.
It goes:
1. Gemini
2. Vidu
3. Le Chat (Mistral)
4. DeepAI
5. [insert MiniMax provider]
If you’re interested you can find contact to me via this profile.
3.5 usd/1000 pages is just too expensive…
Google documentai costs 1.5$ per 1000 which is probably better in quality and speed.
I’ve been there, implementing a way to linearise text from a document with pages with 1, 2 or 3 columns, some of them in landscape is a nightmare. And that’s not even considering equations.
In the end it’s way easier to use a specialised model, trained by other people to do exactly what I need.
So, maybe it can be tuned for your usecase but with that kind of investment €3.5 for 1000 pages is a bargain...
There's a big gulf between "it's as if a human being had transcribed it and reconstructed the original document" and "so completely broken it can't be used for anything".
Imagine you have a cheap and 99% accurate ocr. The other 1% you can detect and apply more powerful (more accurate but slower and more expensive) ocr method. What would you use? At scale these things add up.
Because I'm assuming that's why they get to charge more for the right type of customer.
Their hosted, API-based service is something like a third of the cost of this model.
After stuff like Chat Control I think they're obviously seeing a big demand for this kind of "internet safety" technology in Europe.