TL;DR

 1
 2
 3
 4
 5
 6
 7
 8
 9
10

apt install convmv                                   # dnf install convmv

# 1. See what would be renamed (without --notest, convmv only simulates)
convmv -f utf-8 -t utf-8 -r --nfc /var/www/nextcloud/data/alice/files

# 2. Apply
convmv -f utf-8 -t utf-8 -r --notest --nfc /var/www/nextcloud/data/alice/files

# 3. Have Nextcloud index again
sudo -u www-data php occ files:scan --all

If the names come from an old Windows or Samba and are not even UTF-8 (convmv flags them as not UTF-8): convmv -f iso-8859-1 -t utf-8 -r --notest <path>.

On Docker, occ runs inside the container: docker exec -u www-data nextcloud php occ files:scan --all (on the linuxserver.io image, sudo -u abc php /config/www/nextcloud/console.php files:scan --all).

The error

1

Entry "Ñandú y Ñoño.pdf" will not be accessible due to incompatible encoding

occ files:scan prints it. The files are on disk, ls shows them, but they do not appear on the web, do not sync, and do not count towards quota. It shows up almost always after copying data straight into the data directory (migrations, rsync from a Mac, an SMB share mounted as external storage) instead of uploading through the client.

Why it happens

Unicode allows writing “ñ” two ways: precomposed (NFC, a single code point) or decomposed (NFD, an “n” plus a combining tilde). They look identical and are different bytes. macOS has historically stored names in NFD (HFS+ forced it; APFS and Finder still produce them when copying over SMB or rsync); Linux and Nextcloud work in NFC. Nextcloud normalizes every name to NFC and discards the ones it cannot match with what is on disk, hence the warning.

The other case is older: ISO-8859-1 (Latin-1) names inherited from Windows or from a Samba without unix charset = UTF-8. They are not valid UTF-8 and Nextcloud does not accept them either.

The fix, calmly

  1. Maintenance mode so no client syncs halfway: occ maintenance:mode --on.
  2. Simulate: convmv without --notest lists every rename it would do. If files show up as not UTF-8, those go with -f iso-8859-1.
  3. Apply with --notest. If the NFC twin already exists (the same name uploaded twice), convmv skips it and warns; --replace overwrites, so decide beforehand which one is the good one.
  4. occ files:scan --all (or --path=/alice/files/Photos for one user or folder). The warning goes away and the files enter the index.
  5. occ maintenance:mode --off.

convmv renames with rename(): it keeps owner, permissions, and dates, and it is instant even with hundreds of thousands of files.

Keeping it from coming back

  • From macOS, upload with the Nextcloud client, which normalizes to NFC. If you copy with rsync, --iconv=utf-8-mac,utf-8 converts the names in flight.
  • Samba as a bridge: vfs objects = fruit with fruit:encoding = native does the conversion for Mac clients; and unix charset = UTF-8 always.
  • External storage SMB/local: fix the names at the source; Nextcloud does not rename what it cannot index.

Traps

  • --notest is what applies. The opposite of what one expects: by default convmv is a dry run.
  • Renaming changes the fileid. For Nextcloud, the NFD file disappears and an NFC one is born: shared links, comments, and versions of those files are lost. Warn the affected users first.
  • ownCloud behaves the same and has the same occ.
  • MySQL databases with 3-byte utf8 give a similar but different error (emoji and CJK): there it is occ db:convert-mysql-charset, not convmv.

See also: Seafile: reclaim space with seaf-gc and rsync: resume large transfers (the --iconv). Reference: thread on help.nextcloud.com.