Layout mode silently drops the text of blocks classified as "formula" (to_markdown / to_text)

PyMuPDF
1

Environment

  • pymupdf4llm 1.28.0, PyMuPDF 1.28.0, pymupdf-layout 1.28.0
  • Python 3.11, Linux container on arm64
  • Also checked against the 1.28.2 wheels: same behaviour

Summary

In layout mode, blocks classified as formula lose their text entirely in the output. Blocks classified as picture keep their text, but formula blocks do not, even with force_text=True (the default). When write_images and embed_images are both False (the defaults), no image is produced either, so the block contributes nothing at all.

Combined with a classification issue, this silently removed 50 paragraphs of body text from one of our documents.

What we observe

Our documents are Korean legal texts (statutes and insurance policy terms). The layout analyzer classifies the “supplementary provisions” sections as formula.

import pymupdf, pymupdf4llm

doc = pymupdf.open("statute.pdf")
page = doc[58]
page.get_layout(return_raw=True)

for b in page.layout_information:
    if b["class_name"] == "formula":
        print(repr(page.get_textbox(pymupdf.Rect(b["group_bbox"])).strip()))

Output:

'부   칙  <2010. 5. 7.>'
'제1조(시행일) 이 규정은 2010년 5월 7일부터 시행한다. 부  칙 <2015. 7. 16.>'
'제1조(시행일) 이 규정은 세종특별자치시가 조합에 가입한 날부터 효력을 발생한다. ...'
'제1조(시행일) 이 규정은 조합회의 의결이 있는 날부터 시행한다.'

This is ordinary body text. It states the effective dates of the regulation. None of it appears in the result of pymupdf4llm.to_markdown(doc).

Where it happens

In pymupdf4llm/helpers/document_layout.py, the markdown emitter (1.28.0 around line 814, 1.28.2 around line 1007):

if btype in ("picture", "formula"):
    ...                                   # write or embed the image
    # output text in image if requested
    if box.textlines:
        if btype == "picture":            # formula is not included here
            md_string += picture_text_to_md(...)
    string_lengths.append(len(md_string))
    continue

box.textlines is populated for formula blocks as well, but it is only emitted fo path has the same shape (1.28.2 around line 1138).

Since we do not write or embed images, the branch above appends only "\n\n" and tho warning or log entry.

Impact

In a 179 page document with 138 text pages, 50 paragraphs were missing from the marked because we compare the output against the raw text layer. In legal and insurancedocuments, the affected sections carry effective dates and transitional rules, so losing them is not acceptable for us.

For comparison, running the same document in legacy mode (use_layout(False)) does not lose these paragraphs.

Questions

  1. Is it intentional that formula blocks do not emit their text while picture blocks do? If so, what is the reasoning, given that force_text defaults to True?
  2. Is there any option to have the text of formula blocks included in the output?
  3. Is it known that this kind of layout, a short heading line followed by one or two short provisions, tends to be classified as formula? We see it consistently across several Korean statute documents.

We are happy to provide a small sample PDF that reproduces this if that helps.

2

Hi @lsj6924 Welcome to the forum - yes please , if you could please share a PDF that would be perfect and then we can investigate further! CC’ing @HaraldLieder

3

As an initial reaction from a maintainer:
Yes, we intentionally handle “formula” regions differently than “picture” ones.
This was based on our observation that real formulas reliably generate nonsense output under plain text extraction (which is effectively what happens for pictures and force_text=True).

We may have to re-consider this, so an example file would indeed be valuable.