mirror of
https://codeberg.org/forgejo/forgejo
synced 2024-11-29 21:26:10 +01:00
b272224918
Backport #28935 by @silverwind
The `ToUTF8*` functions were stripping BOM, while BOM is actually valid
in UTF8, so the stripping must be optional depending on use case. This
does:
- Add a options struct to all `ToUTF8*` functions, that by default will
strip BOM to preserve existing behaviour
- Remove `ToUTF8` function, it was dead code
- Rename `ToUTF8WithErr` to `ToUTF8`
- Preserve BOM in Monaco Editor
- Remove a unnecessary newline in the textarea value. Browsers did
ignore it, it seems but it's better not to rely on this behaviour.
Fixes: https://github.com/go-gitea/gitea/issues/28743
Related: https://github.com/go-gitea/gitea/issues/6716 which seems to
have once introduced a mechanism that strips and re-adds the BOM, but
from what I can tell, this mechanism was removed at some point after
that PR.
Co-authored-by: silverwind <me@silverwind.io>
(cherry picked from commit
|
||
|---|---|---|
| .. | ||
| forgejo-v1.19.0.mysql.sql.gz | ||
| forgejo-v1.19.0.postgres.sql.gz | ||
| forgejo-v1.19.0.sqlite3.sql.gz | ||
| gitea-v1.6.4.mssql.sql.gz | ||
| gitea-v1.6.4.mysql.sql.gz | ||
| gitea-v1.6.4.postgres.sql.gz | ||
| gitea-v1.6.4.sqlite3.sql.gz | ||
| gitea-v1.7.0.mssql.sql.gz | ||
| gitea-v1.7.0.mysql.sql.gz | ||
| gitea-v1.7.0.postgres.sql.gz | ||
| gitea-v1.7.0.sqlite3.sql.gz | ||
| migration_test.go | ||