# seafile — GCID table loss, 6-month CPU burn (fixed 2026-08-22)

`seaf-server` in the `seafile` container sat at 40–47 % CPU for months
while appearing to do nothing: negligible client traffic on `eth0`
(344 MB sent over 102 days) but 82 GB and 385 M packets over loopback,
and ~11 **days** of accumulated CPU time.

## Root cause

The `seafile-db` MariaDB database was missing the `GCID` and
`LastGCID` tables. Since Seafile 11 every client sync operation checks
a per-repo GC id (`SELECT gc_id FROM GCID WHERE repo_id = ? FOR
UPDATE`); with the table gone the query failed, the client retried,
and the cycle repeated forever — no sync ever completed:

- 6 of seaf-server's fileserver worker threads spinning in pure
  user-space, ~3 new MySQL connections/s (167 loopback `TIME_WAIT`
  to :3306 at any moment),
- `seafile.log` grew to **1.8 GB** of the single warning
  `Failed to prepare sql SELECT gc_id FROM GCID … Table
  'seafile-db.GCID' doesn't exist`, several times per second.

First occurrence **2026-03-01 22:17** — under 11.0.9, i.e. *before*
both the 2026-03-16 restart and the 2026-05-11 upgrade to 12.0.14.
`GCID` is an old core table (the 6.3.0 upgrade SQL already ALTERs
it), so it existed and was lost. **What dropped it around 2026-03-01
was never identified** — if tables vanish again, start there.

## Diagnosis trail (what worked without container root)

From the host as `julien` (sudo grant covers read-only `incus`
subcommands only):

```sh
sudo incus info seafile                    # PID 2410, 1005 GiB, 2 GiB RAM
ps -eo pid,%cpu,args --sort=-%cpu          # seaf-server pid 10217 top hog
# per-thread CPU from /proc (world-readable, no root needed):
for t in /proc/10217/task/*; do awk '{print $1, $14+$15}' $t/stat; done
# utime vs stime split → pure user-space compute, ~0 syscalls
# container netns TCP table via the process:
awk 'NR>1 && $4=="06"' /proc/10217/net/tcp | wc -l   # 167 TIME_WAIT → :3306
```

Reading `/opt/seafile/logs/seafile.log` needed a temporary sudoers
extension (`/usr/bin/incus exec *`, `/usr/bin/incus file pull *` in a
drop-in — **removed again after the fix**). The log line then made the
cause obvious.

## Fix

The exact DDL ships inside the server binary:

```sh
strings /opt/seafile/seafile-server-12.0.14/seafile/bin/seaf-server \
  | grep 'CREATE TABLE IF NOT EXISTS.*GCID'
```

Applied (root in the container; additive, no restart needed):

```sql
CREATE TABLE IF NOT EXISTS GCID (
  id BIGINT NOT NULL PRIMARY KEY AUTO_INCREMENT,
  repo_id CHAR(36), gc_id CHAR(36),
  UNIQUE INDEX(repo_id)) ENGINE=INNODB;
CREATE TABLE IF NOT EXISTS LastGCID (
  id BIGINT NOT NULL PRIMARY KEY AUTO_INCREMENT,
  repo_id CHAR(36), client_id VARCHAR(128), gc_id CHAR(36),
  UNIQUE INDEX(repo_id, client_id)) ENGINE=INNODB;
```

The warning storm stopped the same second, seaf-server dropped to
0 % CPU, host load fell from ~8–12 toward normal. `seafile.log` was
truncated (`truncate -s 0`).

## Log rotation (added 2026-08-22)

Seafile ships **no rotation** — that's how one warning reached 1.8 GB
unnoticed. `/etc/logrotate.d/seafile` in the container:

```
/opt/seafile/logs/*.log {
	weekly
	maxsize 100M
	rotate 8
	missingok
	notifempty
	compress
	delaycompress
	copytruncate
}
```

`copytruncate` avoids pid/SIGUSR1 handling for the various daemons
(seaf-server, seahub, seafevents…) at the cost of possibly losing a
few lines at rotation — fine here. `maxsize 100M` caps a future spam
storm at the next daily logrotate run instead of next week.

## Related cleanup / leftovers

- Legacy 4 AM cron on ks4 rsyncing seafile's rootfs to old **ks2**
  (`164.132.173.57`) removed 2026-08-22 — redundant with the
  [incus-copy](incus-copy.md) legs and it burned another ~47 % CPU
  per run.
- Still missing from `seafile-db` (harmless, nothing queries them
  unless per-folder permissions are used): `FolderUserPerm`,
  `FolderGroupPerm`, `FolderPermTimestamp`. Same `strings`/upgrade-SQL
  trick provides the DDL if ever needed.
- Host swap (1 GB) was 100 % full during the incident; unrelated but
  worth keeping an eye on.