chore: sync upstream (205 Commits) #17
No reviewers
Labels
No labels
herkunft/alpha-feedback
status
blockiert
status
zu-testen
typ
bug
typ
docs
typ
enhancement
typ
feature
typ
idea
No milestone
No project
No assignees
1 participant
Notifications
Due date
No due date set.
Dependencies
No dependencies set.
Reference
iris/pdfcraft!17
Loading…
Add table
Add a link
Reference in a new issue
No description provided.
Delete branch "chore/sync-upstream-20261105"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
Upstream-Sync von
upstream/main(205 Commits). Fast-Forward.250c2623Tests: pin the file identifier formula, not a whole rewritten file (#6346)60f60fc0Give a redacted or sanitized copy a new file identifier (#3734)f16b5d93M13 batch: test that results hold no planted secret, and record network-crate reachability (#6297)ee39c7e2Issue #5081: assembly at revision 3 and later needs bit 11; high-quality print, accessibility and form fill follow the revision (#6309)bb9da1c1Convert replaces a removed forbidden annotation, so it is not reported again (#6306)b3a90f07Parity: titles and the PDF/X row say only what is implemented (#6300)f9c150c1Add IBAN matching to Search and Redact patterns (#4332)ae893c15M13 batch: correct parity status and scope the no-replace sentence (#6299)0c744357M13 batch: an action step that fails now fails its file (#6307)9a95273cFuzz targets for PDF/A verify and convert (#6308)c15c9097Fuzz target for redaction apply (#6305)a2c6e63dFuzz target for the JavaScript engine's script runner (#6304)0d7ae1b0Tests: a signed encrypted document's byte ranges cover its earlier revision (#6303)9ff881e5The PDF/A panel reports its own checks, not conformance (#6302)b1c1dbdaOne known-answer case per implemented clause (#6301)A new `model_add` automation tool and `Edit::AddModel3d` embed a U3D or PRC file as a 3D annotation (ISO 32000-2 13.6.2) in a chosen rectangle: - The model goes into a Flate-compressed 3D stream (13.6.3) with /Type /3D and /Subtype U3D or PRC, read from the file's header (`U3D` and a zero byte, or `PRC`); anything else is refused before the document changes. Models over 256 MiB (MAX_MODEL_BYTES) are refused. model_add checks the file's size (from the file system) and its header before reading the rest, so a large file that is not a model is never loaded; the path goes through the read root, and `read_model` joins the helpers the file-access test knows take a resolved path. - /3DD references the stream, so the annotation has its own instance. - /3DA states /A: click (XA, the standard's default and the tool's), page_open (PO) or page_visible (PV). - /AP /N is the poster readers print and show before activation: a chosen image or PDF page fitted inside the box keeping its shape and centred (the custom-stamp picture path: `mark_source`, `fit_picture`, and the stamp's placement matrix now shared as `appearance::picture_place`), or, with no picture, a light frame labelled "3D model". It is drawn in user space and not counter-rotated, so on a page with /Rotate it turns with the page, as the model does. - /Contents carries the name (the file stem by default) as the alternate description (12.5.2). - A document older than the model needs gets /Version in its catalog (7.7.2): 1.6 for U3D, 1.7 for PRC, as the optimizer does for JBIG2. Adobe's extension level for PRC in 1.7 is not written. - The 3D rectangle has its own MAX_MODEL_SIDE (14400 pt). The edit is undoable ("Add 3D model"), needs the comments permission and rebuilds only the comments view. A new parity entry, misc.3d-add, is partial: no poster rendered from the model, no views (/VA, /3DV), no replacing the poster, no UI command, no /Extensions. misc.3d-view is unchanged. ROADMAP.md progress log line. Adapted from pdfcer (MIT, same author), commit 1ca5ece, rewritten to this repository's types and rules. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01SqmHYx8FiWM5PcsoiErgp99ce6d89f) into feat/prc-reader 43b95919bb* A comment's text, border or line, and fill each at an opacity of its own Comment Properties showed one opacity for a whole comment, so a text box's see-through background took its text and border with it, and a rectangle marking an area on a plan couldn't have an opaque line over a faint fill. Now a text box, typewriter text or callout has a Text, Border and Fill opacity, a rectangle, oval, polygon or cloud a Line and Fill opacity (a part not drawn is greyed); everything else keeps one Opacity. The line's stays /CA, the fill's is /ca (PDF 2.0's opacity for what is filled) and the text's /PCTextOpacity, each only where it differs from the one it would follow, and the appearance draws them with /GS0 (strokes at the line's, fills at the fill's) and /GS1 for the text, so every viewer shows them. The quick bar's Opacity and StyleAnnotation's set the whole comment's. Engine: Style::{fill_opacity, text_opacity}, Edit::PartOpacity, annot::{set_part_opacity, appearance::part_opacity}, Props::{fill_opacity, text_opacity}. Make Properties Default and the settings carry the parts. Headless: comment_add / comment_edit fill_opacity, text_opacity; comment_list reports opacity, fill_opacity, text_opacity. Four labels in every catalog (AI-assisted translations from English, noted in ATTRIBUTION.toml). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Review: the text's opacity slider shows what the text is drawn at; comment_list skips replies' properties; ui-parity's change record A text box without fill and border whose text kept an opacity of its own showed the border's in its Text opacity slider; it now shows the text's, and a change sets the whole comment's. The opacity sliders are named by their labels. comment_list reads a comment's properties only for comments, not for every reply. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Rebase: comment_list tells replies by is_reply, as main now threads them main threads replies by their /IRT reference (#4061); the opacity fields skip a reply the same way. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Review: a text-only text box made default gives the tool its text's opacity With neither a border nor a fill, a text box's text is the whole comment: Comment Properties shows its drawn opacity (the text's own, when set), and a change there sets the whole comment's. Make Properties Default, and the context menu's Make Current Properties Default, stored /CA instead, so a text box whose text was at 50 % gave the tool 100 %. The default is now the text's drawn opacity in that case. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Rebase: the part opacities in main's annot layout (Style's fields, the text box drawing's parts, set_part_opacity exported) Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Rebase: main's new comment test builds a Style with the part opacities Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> Co-authored-by: FredApps <FredApps@users.noreply.github.com> Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>Comments copy and paste, within and between documents. ⌘C with comments selected (and no text selected) copies them, each with its replies, as XFDF: into the app's clipboard (every tab and window) and as text onto the system clipboard. ⌘V, Edit ▸ Paste comments or the page's context menu pastes them onto the current page at their own positions, as new comments under names of their own, so nothing is replaced and pasting twice gives two sets; the copies are selected. XFDF with comments from the system clipboard pastes too, so comments travel between processes. pdfcraft_xfdf::export_xfdf_comments writes a selection with its replies; import_comments takes a target page and a name prefix and imports copies (the usual import still replaces same names). Engine: Edit::PasteComments { xfdf, page } (one undo step, comment permission) and Session::export_comments. Tools: comments_copy (read-only) and comments_paste. Several comments select at once: ⌘-click (Ctrl on Windows and Linux) or ⇧-click adds one or takes it out, on the page and in the Comments panel; the one clicked last has the handles. Delete removes them all as one step, a drag moves them together, the context menu copies them. Connected lines, polygons and clouds finish from the right-click menu's Complete (with Undo last point and Cancel drawing), as other editors do, besides Enter and a double-click. Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: FredApps <FredApps@users.noreply.github.com>* Add IBAN matching to Search and Redact patterns The Pattern/PATTERNS/from_id interface is consumed generically by redact_mark and the UI pattern combobox through the engine re-exports, so one new variant and matcher exposes IBAN in Search and Redact everywhere. A unit test shows valid compact and spaced IBANs are marked while wrong check digits, short runs, unknown countries, letter check digits and token-glued IBANs are not. Add an Iban variant to Pattern, list it in PATTERNS, handle it in label, id and find, and add an iban matcher with a country-code length table, compact or group-of-four-spaced acceptance, alphanumeric start and end boundaries and ISO 7064 mod-97 check-digit validation. Test: iban_matches_compact_spaced_and_rejects_near_matches failed before the change and passes after. * Address review for pull request 4332 * Address review for pull request 4332 * review: address CodeRabbit findings - patterns.rs: fix SWIFT registry IBAN lengths EG 27->29, LY 27->25, SD 24->18 and add NI 28, BI 27, DJ 27, OM 23 (crates/redact/src/patterns.rs:545,549,551,558-561) - patterns.rs: state table source as SWIFT IBAN registry without claiming full ISO 13616 coverage (crates/redact/src/patterns.rs:475) - patterns.rs: regression test with valid registry examples for EG, LY, SD, NI, BI, DJ, OM and a wrong-length EG rejection (crates/redact/src/patterns.rs:892) * Complete the IBAN country table and accept spaces only between groups The country table now lists all 89 countries of the SWIFT IBAN registry (release 99) as one sorted constant. Honduras, Russia, Yemen, Falkland Islands, Mongolia and Somalia were missing. A space is accepted only after a full group of four characters, so a code followed by ordinary words ("BA37 leaves at noon") ends the run instead of being tried against the check digits. Tests: the registry example of every country matches compact and printed, changes to one digit are rejected, and the table and the examples agree. * Advertise the IBAN pattern in the redact_mark tool schema The pattern enum in the advertised schema was kept by hand and still listed phone, email, credit-card, ssn and date, so clients that obey the schema could not select iban, which the runtime already accepted. The enum is now built from the engine's pattern registry, so it cannot drift from it again, and the description names iban. Tests: tools/list advertises exactly the registry's pattern ids and names each in the description, and an iban mark made through the tool covers the spaced German sample and nothing else. --------- Co-authored-by: eliets2 <287273517+eliets2@users.noreply.github.com> Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>* cos: give a redacted or sanitized full save a new file identifier A full save that the document asked for (Document::require_full_save_with_new_id) now writes a fresh /ID pair, so a redacted or sanitized copy does not claim to be the file it came from (ISO 32000-2 section 14.4). Ordinary saves are unchanged. An encrypted file keeps the first string of /ID, because the file key is derived from it, and gets a new second string. An encrypted file whose /ID is not a pair of strings is left as it was, as before. SaveOptions::id_entropy lets a caller or a test supply entropy; with it the new identifier depends only on that entropy and the written bytes. Without it the identifier mixes the standard library's random hash keys and a process-wide counter, the same facility the engine uses for new encryption keys. Checked: the new tests in crates/cos/tests/file_identifier.rs fail on the clean base (compile errors for the missing API), fail on assertions when only the API is in place, and fail again when each pinned line is reverted (the regeneration call, the first-string keep, the flag, the entropy input, and the ordinary-save guard). cargo test -p pdfcraft-cos passes, and clippy with warnings denied is clean. A probe that wrote 16 ordinary full and incremental saves gave byte-identical files with and without this change. * redact: give a redacted or sanitized copy a new file identifier Applying redactions, sanitizing and removing hidden information now ask the cos layer for a new file identifier (Document::require_full_save_with_new_id), so the saved copy no longer carries the /ID of its source. Before this change the previous revision was dropped but the identifier still linked the copy to the original. The request sits in remove_hidden itself, because its per-category callers (the remove-hidden edit and the optimizer's discard option) call it directly, not through sanitize(). sanitize() asks only on its nothing-hidden path, where remove_hidden returned before asking: the rewrite is a new file either way. Checked: the new tests a_redacted_or_sanitized_copy_gets_a_new_identifier and a_remove_hidden_copy_gets_a_new_identifier fail before this change (the request is missing after apply, and after remove_hidden) and fail again when the apply call, the nothing-hidden branch or the remove_hidden request is reverted. cargo test -p pdfcraft-redact passes with the tests included (152 passed, 1 ignored); cargo clippy with warnings denied and cargo fmt --check are clean. * docs(cos): say when a full save regenerates the file identifier The writing section of the crate README described full saves without saying what happens to /ID. It now says that an ordinary save keeps the identifier, that a document which asks for a new one (redaction and sanitizing) gets both strings regenerated, that SaveOptions::id_entropy supplies entropy for reproducible output, and that an encrypted file keeps its first string because the key depends on it. Checked: the text matches the code added in e1f761cc (regenerate_id, new_id_bytes, require_full_save_with_new_id) and the tests in crates/cos/tests/file_identifier.rs. No code changed in this commit. * cos: hash the whole rewritten file into the new identifier The identifier of a requested rewrite hashed only the first 64 KiB of the written file. With caller-supplied id_entropy the identifier is deterministic, so two rewrites of documents with the same length that differ only after byte 65,536 received the same /ID, and a later edit of a large document could keep its previous /ID. Every written byte is now hashed. The output is read once: one loop over 4 KiB chunks feeds both halves of the 16-byte identifier, so the extra cost is one linear pass over the output. Measured with a probe in the workspace dev profile (opt-level 1): hashing 64 MiB took 0.024 s, about 24 ms per 64 MiB of output. A release build was not measured. Ordinary saves never reach this code, and their output is unchanged. Checked: crates/cos/tests/file_identifier.rs gains the_identifier_depends_on_the_whole_rewritten_file. It fails on the branch head before this change ("content past 64 KiB must change the identifier", both identifiers equal) and passes after it; reverting only the chunk loop to the 64 KiB prefix makes it fail again. The six existing tests in that file pass, and so do the full cos and redact test suites. * cos: keep the key valid when an encrypted rewrite has no usable /ID A requested rewrite of an encrypted file whose /ID was absent, had one entry, or had a first entry that is not a string wrote no identifier at all, although the document had asked for a new one. The reader takes the empty string as the key input for these shapes (Document::first_id), so the rewrite now writes the empty string (<>) as the first entry and a new second entry. The key stays valid. A file whose first entry is a string keeps that string, as before. Ordinary saves and unencrypted rewrites are unchanged. Checked: an_encrypted_rewrite_without_a_usable_identifier_still_gets_a_new_one in crates/cos/tests/file_identifier.rs covers the three shapes for RC4-128, AES-128 and AES-256. After the rewrite each file opens with its password, the first string is empty and the second is new. On the branch head the test fails with "the rewrite writes a pair" (left 0, right 2). Reverting only the guard to the head's early return makes it fail again. Isolated, the one-entry shape fails with left 1, right 2, and the non-string shape with left Some(0), right Some(16). The other seven tests in the file pass. * cos: derive the identifier from caller entropy with a fixed hash With SaveOptions::id_entropy the identifier was built with std's DefaultHasher, whose algorithm the standard library does not specify, and with write_usize (4 bytes on wasm32, 8 on 64-bit targets) and native-endian integers. The same entropy and the same file could therefore give different identifiers on different platforms or toolchains, although the documentation promised the bytes depend only on the entropy and the written file. The identifier is now FNV-1a (64 bit, the published constants) over fixed-width big-endian fields and the whole written file: the salt, the lane, id_seed and the length, then the bytes. With caller entropy the salt is the FNV-1a hash of that entropy. Without entropy it is still the standard library's random hash of a process counter, so only the salt differs between the two paths. Every written byte is hashed, in one pass that feeds both lanes. Cost: this pass is slower than the SipHash it replaces. A temporary probe in the workspace dev profile (opt-level 1) hashed 64 MiB through both lanes in 0.129 s (about 500 MiB/s), against 0.024 s before. A release build was not measured. The cost is paid once per requested rewrite; ordinary saves do not reach this code. Checked: the golden test fixed_entropy_rewrite in crates/cos/tests/file_identifier.rs pins the 16 bytes for a fixed entropy and the standard fixture. On the previous hash it fails (left [64, 145, ...], right [50, 125, ...]). An independent re-implementation of the same formula in Python, run over the bytes the writer hashed (everything before startxref), gives the same 16 bytes. The unit test fnv1a_is_the_published_64_bit_algorithm checks the constants against the published vectors. FNV is not collision-resistant against chosen content; that matters only if someone could choose a file's identifier, and the identifier only has to be unique. * Repair review branch integration * cos: take fresh identifier entropy from the browser on wasm Without SaveOptions::id_entropy the salt of a rewritten file's identifier was RandomState and a process-wide counter. On wasm32-unknown-unknown those hash keys are fixed at compile time, so two fresh web apps that followed the same allocation and call sequence reproduced the same salt and the same identifier for identical output, and a redacted copy stayed linkable to its source. The default salt now comes from the browser first: crypto.getRandomValues (Web Crypto, through js_sys, sixteen bytes folded with the same FNV-1a the identifier uses), or Math.random where that API is missing, and only when neither is reachable the previous RandomState salt. js-sys and wasm-bindgen join pdfcraft-cos as wasm32-only dependencies; both are already in the dependency tree of the web shell, and native builds compile neither. Caller entropy is untouched: with id_entropy the identifier is still a pure function of that entropy, the seed and the written bytes. Checked: separate_fresh_processes_get_different_identifiers re-runs this test binary as two fresh child processes over the same fixture and fails when the default salt is made constant (so does the in-process divergence test); stand_in_for_a_fresh_process is its no-op child half. Both pass with the real salt. On WebAssembly no browser test harness exists, so the browser path is covered by the wasm32 cargo check and by construction, not by a test. cargo check --target wasm32-unknown-unknown -p pdfcraft-cos passes, cargo test -p pdfcraft-cos and -p pdfcraft-redact pass, and clippy with warnings denied is clean on both. * Extract full-save trailer assembly Assisted-by: OpenAI Codex --------- Co-authored-by: eliets2 <287273517+eliets2@users.noreply.github.com> Co-authored-by: FredApps <FredApps@users.noreply.github.com> Co-authored-by: FredApps <153173854+FredApps@users.noreply.github.com>