xPlore indexing for large documents
The Index Agent decides whether a file is even sent to xPlore. That decision is the file size on the content object, not the size of the extracted text. Past the limit, the document is skipped and the rest of the batch still runs.
contentSizeLimit
In indexagent.xml (Index Agent WAR, WEB-INF/classes), the exporter section has contentSizeLimit. Default is 20000000 bytes (about 19 MiB). Compare it to r_full_content_size / dmr_content.full_content_size, not to the extracted text. If the file is larger, Index Agent does not convert it. You get a size-exceeded message in the Index Agent log and on the matching dmi_queue_item.
$JBOSS_HOME/server/DctmServer_Indexagent/deployments/IndexAgent.war/WEB-INF/classes/indexagent.xml
Path varies with the JBoss/WildFly install name (DctmServer_Indexagent_<docbase> when you have more than one agent). Stop the Index Agent before you edit the XML; start it after.
select r_object_id, object_name, r_full_content_size, a_content_type
from dm_document
where r_full_content_size > 20000000
Use the double field. See r_content_size vs r_full_content_size.
Timeouts next to the limit
Raising the byte limit without raising timeouts just moves the failure from “skipped” to “timed out.” Same file, indexagent.xml:
| Parameter | Typical default | Notes |
|---|---|---|
request_time_out | 600 s | Whole request to CPS / xPlore. |
text_extraction_time_out | 300 s (range 60–300) | Text extraction of one file. |
linguistic_processing_time_out | 360 s (range 60–360) | Linguistic processing after extract. |
content_clean_interval | 1200000 ms (20 min) | How long the exported DFTXML stays in local_content_area. Too small and CPS fetches a file the agent already deleted. |
Heap on the Index Agent JVM has to match the new sizes. Huge binaries (zip, video, already-scanned PDF with no text) are often better excluded from full-text than forced through CPS.
What a skip looks like
Inspect Index Agent logs first. Failed or skipped work also shows up as dmi_queue_item rows for the full-text indexer. Do not copy production object ids into tickets; examples use 09xxxxxxx.
select r_object_id, item_id, event, message, date_sent
from dmi_queue_item
where item_id = '09xxxxxxx'