---
title: "Smart features"
output: rmarkdown::html_vignette
vignette: >
  %\VignetteIndexEntry{Smart features}
  %\VignetteEngine{knitr::rmarkdown}
  %\VignetteEncoding{UTF-8}
---

```{r setup, include = FALSE}
knitr::opts_chunk$set(collapse = TRUE, comment = "#>")
library(ksCompare)
```

`ksCompare` adds several conveniences on top of plain `PROC COMPARE`-style
diffing.

## ULP tolerance

`ks_tol(ulp = N)` accepts pairs whose IEEE-754 binary64 representations
are within `N` units in the last place.

```{r}
x <- 1.0
y <- x + .Machine$double.eps
ks_compare(
  data.frame(id = 1, x = x),
  data.frame(id = 1, x = y),
  by = "id",
  tolerance = ks_tol(ulp = 4)
)
```

## Auto-key inference

```{r}
df <- data.frame(study = "S1", id = 1:3, x = 1:3)
suppressMessages(ks_compare(df, df, by = "auto"))
```

`by = "auto"` searches for the smallest combination of shared columns
whose values are unique on both sides.

## Diff triage helpers

`ks_cause_summary()` groups value diffs by likely cause, while
`ks_row_diff_summary()` shows which matched observations changed the
most.

```{r}
a <- data.frame(id = 1:4, x = c("A", "b ", "c", "d"), y = c(1, 2, 3, 4))
b <- data.frame(id = 1:4, x = c("a", "b",  "c", "d"), y = c(1, 9, 3, 9))
cmp <- ks_compare(a, b, by = "id")
ks_cause_summary(cmp)
ks_row_diff_summary(cmp)
```

## First / last unequal observations

For SAS-style review tables, `ks_compare()` also stores the first and
last differing observations per matched column.

```{r}
ks_compare(a, b, by = "id", n_first_last = 1)$first_last_unequal
```

## Duplicate-key strategies

```{r}
a <- data.frame(id = c(1, 1, 2), x = c(10, 99, 20))
b <- data.frame(id = c(1, 2),    x = c(10, 20))

ks_compare(a, b, by = "id", dup_keys = "first")
ks_compare(a, b, by = "id", dup_keys = "last")
ks_compare(a, b, by = "id", dup_keys = "all_pairs")
```

`dup_keys = "keep_all"` pairs duplicate rows positionally within each
key group (1<->1, 2<->2, ...). Leftover rows on the longer side
become base- or compare-only, and `cmp$row_diff` carries a
`pair_rank` / `pair_total` column so you can tell which pair each
diff belongs to. The HTML report shows a *Pair* column when this
strategy is active.

## SAS format normalisation

Format comparison is trailing-dot- and case-tolerant, so `haven`
import quirks do not surface as false schema diffs:

```{r, eval = FALSE}
a <- data.frame(d = as.Date("2024-01-01"))
attr(a$d, "format.sas") <- "DATE9."
b <- a
attr(b$d, "format.sas") <- "date9"   # no trailing dot, lowercase

ks_compare(a, b)$schema_diff   # format_match == TRUE
```

## Encoding-safe string comparison

Strings imported from mixed Latin1 / UTF-8 sources are converted to
UTF-8 before equality testing, so the same character in two encodings
does not register as a diff. Combine with `str_norm = "NFC"` for
fully Unicode-normalised comparison.

```{r, eval = FALSE}
ks_compare(
  base, comp, by = "id",
  options = ks_comp_options(str_norm = "NFC")
)
```

## Pattern detection

Pattern detection is opt-in because the detectors scan every column
with at least one diff. When many cells differ in the same way
(constant offset, sign flip, trim-only string differences, etc.), set
`find_patterns = TRUE` to populate `cmp$pattern_summary`:

```{r}
a <- data.frame(id = 1:5, x = c(1, 2, 3, 4, 5))
b <- data.frame(id = 1:5, x = c(2, 3, 4, 5, 6))
ks_compare(a, b, by = "id", find_patterns = TRUE)$pattern_summary
```

Common labels include `constant_offset`, `constant_scale`,
`sign_flip`, `integer_round`, `trim_only`, `case_only`,
`epoch_swap`, and `factor_recoded`, depending on the column type.