PyMuPDF 2.1 fails to use the combined OCR approach

PyMuPDF
1

The “select_ocr_function()” in “pymupdf4llm\helpers\document_layout.py” only uses the combined approach (RapidOCR for bounding and Tesseract for text) outlined in the blog when using the legacy rapidocr_onnxruntime, not the latest RapidOCR. Is that intentional and if so, why?

Edit:
The following question was part two of the post and has been resolved.

Additionally, if the exec_ocr() from the rapidtess_api is used directly as an OCR function it fails as text_det is not initialized automatically in RapidOCR. It is only set when RapidOCRs “_load_det_model” is called (this looks to be called only when “detect_and_crop” is called).

Is there a configuration I am missing? Any help would be very appreciated!

RapidOCR previously set text_det during the initialization of RapidOCR in Version 3.9.1. This has changed in the current version 3.10.0.

From what I have found, PyMuPDF does not list the required version in the documentation but it is described in code.

2

Hi @PurpleOrange
Can you confirm the versions of PyMuPDF & PyMuPDF4LLM which you are using?

3

Hi @Jamie_Lemon,

I am using the 2.1 version of pymupdf and pymupdf4llm. I performed a force reinstall as directed on the blog.

The code below is what I am referring to:

def select_ocr_function():
    """Check availability of OCR tools and language data.

    Return the best OCR function available or None.
    """
    from pymupdf4llm.ocr.detect_rapidocr import detect_rapidocr_backend

    tessdata = None
    rapidocr_available = False
    paddleocr_available = False
    try:
        tessdata = pymupdf.get_tessdata()
    except:
        tessdata = None

    rapidocr_backend = detect_rapidocr_backend()
    if rapidocr_backend and rapidocr_backend != "rapidocr_onnxruntime":
        # the new RapidOCR backend is available, so use this as default
        from pymupdf4llm.ocr import rapidocr_api

        print("Using RapidOCR for OCR processing.", file=INFO_MESSAGES)
        return rapidocr_api.exec_ocr

    if rapidocr_backend is not None:
        rapidocr_available = True
        paddleocr_available = True  # for now: we have no own paddleocr yet
    else:  # no RapidOCR backend detected: print a warning
        print("Warning: RapidOCR is missing. Install for much improved OCR results.", file=INFO_MESSAGES)

    if {tessdata, rapidocr_available, paddleocr_available} == {None, False, False}:
        return None

    if tessdata:
        if rapidocr_available:
            from pymupdf4llm.ocr import rapidtess_api

            print(
                "Using RapidOCR & Tesseract for OCR processing.",
                file=INFO_MESSAGES,
            )
            return rapidtess_api.exec_ocr
        elif paddleocr_available:
            from pymupdf4llm.ocr import paddletess_api

            print("Using PaddleOCR & Tesseract for OCR processing.", file=INFO_MESSAGES)
            return paddletess_api.exec_ocr
        else:
            from pymupdf4llm.ocr import tesseract_api

            print("Using Tesseract for OCR processing.", file=INFO_MESSAGES)
            return tesseract_api.exec_ocr
    else:
        if rapidocr_available:
            from pymupdf4llm.ocr import rapidocr_api

            print("Using RapidOCR for OCR processing.", file=INFO_MESSAGES)
            return rapidocr_api.exec_ocr
        elif paddleocr_available:
            from pymupdf4llm.ocr import paddleocr_api

            print("Using PaddleOCR for OCR processing.", file=INFO_MESSAGES)
            return paddleocr_api.exec_ocr
4

Welcome to this forum @PurpleOrange !

I conversed with @Jamie_Lemon about this already. Probably my fault that we did not update the documentation with the most recent details.

I have introduced the RapidOCR+Tesseract cooperation as the default option with the version rapidocr-onnxruntime. At this point, the OCR result became much better and faster by this, because RO’s text recognition phase was both, extremely slow and full of errors WRT to “inabilitytoseparatewords”.

The recent RadpidOCR versions deliver this overall impression:

  1. Much improved text recognition quality (still not as good as Tesseract though)
  2. Much reduced overall OCR time duration
  3. Inability to safely (!) execute bbox detection alone (and omit recognition phase).

In view of all the above, I do no longer call the RapidOCR+Tesseract combination by default. If RO is available with version 3.9.2 or later (in particular not equal to rapidocr-onnxruntime) it will be used as the default choice.
I felt that I shouldn’t impose the full, overall RapidOCR response time on you by default with a now only marginally better recognition result.

But if you absolutely want to use RO+Tesseract you can decide to accept the longer duration and use the respective plugin just as before.

I haven’t seen that other error you mentioned and therefore need your configuration. RapidOCR is known for its unreliable development processes and Chinese-only documentation. They won’t hesitate to change their API in in code-breaking ways … just to name one of several annoyances.

5

Thank you @HaraldLieder for the in depth explanation. That makes sense!

Putting that into practice, I tested a few different configurations (all with pymupdf/pymupdf_layout/pymupdf4llm==2.1 and tessdata set as an environment variable) and ran the code below:

from pymupdf4llm.helpers.document_layout import select_ocr_function
import pymupdf

ocr_function = select_ocr_function()

with pymupdf.open(input_pdf_path) as test_doc:
    for pno in range(test_doc.page_count):
        page = test_doc.load_page(pno)
        ocr_function(page)

    test_doc.save(output_pdf_path)

rapidocr-onnxruntime==1.4.4
onnxruntime==1.31.1

Result: Uses RapidOCR & Tesseract for OCR processing.

rapidocr==3.9.1
onnxruntime==1.31.1

Result: Uses RapidOCR for OCR processing.

rapidocr==3.10.0
onnxruntime==1.31.1

Result: Uses RapidOCR for OCR processing.

rapidocr==3.9.1
rapidocr-onnxruntime==1.4.4
onnxruntime==1.31.1

Result: Uses RapidOCR for OCR processing.

<No Additional Packages>

Result: Uses Tesseract for OCR processing.

This all aligns with what you described, barring:

If RO is available with version 3.9.2 or later (in particular not equal to rapidocr-onnxruntime) it will be used as the default choice.

Even if using version 3.9.1, it is still used as the default over the combo. (Though perhaps my interpretation is incorrect)

Regarding the second issue, I had tessdata set as an environment variable and

pymupdf==2.1
pymupdf_layout==2.1
pymupdf4llm==2.1
rapidocr==3.10.0
onnxruntime==1.31.1

This resulted in the following code failing:

from pymupdf4llm.ocr.rapidtess_api import exec_ocr
import pymupdf

with pymupdf.open(input_pdf_path) as test_doc:
    for pno in range(test_doc.page_count):
        page = test_doc.load_page(pno)
        exec_ocr(page)

    test_doc.save(output_pdf_path)

With the error:

Traceback (most recent call last):
File "DemoOCR.py", line 25, in 
exec_ocr(page)
File "...\\pymupdf4llm\\ocr\\rapidtess_api.py", line 22, in exec_ocr
return exec_ocr_detection(
^^^^^^^^^^^^^^^^^^^
File "...\\pymupdf4llm\\ocr\\exec_ocr_interface.py", line 138, in exec_ocr_detection
result = det_only(img)
^^^^^^^^^^^^^
File "...\\pymupdf4llm\\ocr\\rapidocr_391_backend.py", line 72, in det_only
result = engine.text_det(img)
^^^^^^^^^^^^^^^^^^^^
TypeError: 'NoneType' object is not callable

I see now that RapidOCR 3.9.1 is explicitly stated as being supported in the code and 3.10.0 is not. Testing the same code with 3.9.1 works as expected.

The issue arose because newer versions RapidOCR (in my case 3.10.0) switched to dynamic loading of models. As a result of this; text_det must be manually initialized before it can be used.

With all of that said, what is the current, recommended packages/versions for running OCR?

Thank you again for your help!

6

Thanks for your appreciation!
As per 3.9.1.vs 3.9.2 please forgive me: I didn’t bother to confirm those numbers from the code.
So, 3.9.1 is it then.

We do not explicitly recommend OCR engines or versions. Use whatever you like.
We only state what we support with pre-factured plugins in the hope that they will be popular enough to spare our users the effort to create a plugin for their favorite engine.

The only observation we have to offer (and this indeed is some recommendation):

Relying on Tesseract alone will probably give you dissatisfying overall results. Not in every case, but often enough to be annoying. The problems are usually around absurd text bounding boxes identified by Tesseract. Often enough also a too eager recognition of artifacts as text.

RapidOCR has shown a steadily improving overall results trend. I won’t repeat my reservations above against it.
Generally, its text bbox detection seems to be very good - much better than Tesseract.
Its text recognition (inside those bboxes) seems to be improving but still has lots of head room.

So, I would recommend having both, RapidOCR and Tesseract on board.

Regarding your observation:
I understood that we do not accept 3.10.x as a valid RapidOCR version? Thanks for the hint, I will check this.

7

Just checked it out:
We do not look at RapidOCR versions at all. The only differentiation is RapidOCR versus RapidOCR-ONNXRuntime.

I read your comment more carefully now and I understood that RapidOCR has obviously changed the API … again!!!
These guys are a real pain in the neck. I can’t remember a comparable lack of professionalism in my long career.

Anyway, thanks for the observation; I need to change the RapidOCR plugins again, so they work with the new version.

8

@PurpleOrange @HaraldLieder I’ve updated the docs again here: OCR - PyMuPDF documentation
Probably it will need another update again soon, :slight_smile:

1 Like
9

Right Jamie, I am looking at options to make us as independent as possible from RapidOCR’s unforeseeable changes.

One thing is already clear:
Version 3.10.0 is more flexible again; its 3 internal phases det, cls and rec can be activated independently from each other (because we must activate to avail them).
Especially det (detection) is available separately again - without incurring the effort / response time of the cls and rec phases.
Therefore, the combination RapidOCR+Tesseract is an attractive option again for RapidOCR 3.10.0.