Template
1
0
Fork 0
mirror of https://codeberg.org/forgejo/forgejo synced 2024-11-27 04:06:10 +01:00
forgejo/cmd/serv.go

361 lines
12 KiB
Go
Raw Normal View History

2014-04-10 20:20:58 +02:00
// Copyright 2014 The Gogs Authors. All rights reserved.
// Copyright 2016 The Gitea Authors. All rights reserved.
// SPDX-License-Identifier: MIT
2014-04-10 20:20:58 +02:00
2014-05-02 03:21:46 +02:00
package cmd
2014-04-10 20:20:58 +02:00
import (
"context"
2014-04-10 20:20:58 +02:00
"fmt"
"net/url"
2014-04-10 20:20:58 +02:00
"os"
"os/exec"
"path/filepath"
"regexp"
"strconv"
2014-04-10 20:20:58 +02:00
"strings"
2014-08-10 00:40:10 +02:00
"time"
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
"unicode"
2014-04-10 20:20:58 +02:00
asymkey_model "code.gitea.io/gitea/models/asymkey"
git_model "code.gitea.io/gitea/models/git"
"code.gitea.io/gitea/models/perm"
"code.gitea.io/gitea/modules/git"
"code.gitea.io/gitea/modules/json"
"code.gitea.io/gitea/modules/log"
"code.gitea.io/gitea/modules/pprof"
"code.gitea.io/gitea/modules/private"
"code.gitea.io/gitea/modules/process"
repo_module "code.gitea.io/gitea/modules/repository"
"code.gitea.io/gitea/modules/setting"
Add LFS Migration and Mirror (#14726) * Implemented LFS client. * Implemented scanning for pointer files. * Implemented downloading of lfs files. * Moved model-dependent code into services. * Removed models dependency. Added TryReadPointerFromBuffer. * Migrated code from service to module. * Centralised storage creation. * Removed dependency from models. * Moved ContentStore into modules. * Share structs between server and client. * Moved method to services. * Implemented lfs download on clone. * Implemented LFS sync on clone and mirror update. * Added form fields. * Updated templates. * Fixed condition. * Use alternate endpoint. * Added missing methods. * Fixed typo and make linter happy. * Detached pointer parser from gogit dependency. * Fixed TestGetLFSRange test. * Added context to support cancellation. * Use ReadFull to probably read more data. * Removed duplicated code from models. * Moved scan implementation into pointer_scanner_nogogit. * Changed method name. * Added comments. * Added more/specific log/error messages. * Embedded lfs.Pointer into models.LFSMetaObject. * Moved code from models to module. * Moved code from models to module. * Moved code from models to module. * Reduced pointer usage. * Embedded type. * Use promoted fields. * Fixed unexpected eof. * Added unit tests. * Implemented migration of local file paths. * Show an error on invalid LFS endpoints. * Hide settings if not used. * Added LFS info to mirror struct. * Fixed comment. * Check LFS endpoint. * Manage LFS settings from mirror page. * Fixed selector. * Adjusted selector. * Added more tests. * Added local filesystem migration test. * Fixed typo. * Reset settings. * Added special windows path handling. * Added unit test for HTTPClient. * Added unit test for BasicTransferAdapter. * Moved into util package. * Test if LFS endpoint is allowed. * Added support for git:// * Just use a static placeholder as the displayed url may be invalid. * Reverted to original code. * Added "Advanced Settings". * Updated wording. * Added discovery info link. * Implemented suggestion. * Fixed missing format parameter. * Added Pointer.IsValid(). * Always remove model on error. * Added suggestions. * Use channel instead of array. * Update routers/repo/migrate.go * fmt Signed-off-by: Andrew Thornton <art27@cantab.net> Co-authored-by: zeripath <art27@cantab.net>
2021-04-09 00:25:57 +02:00
"code.gitea.io/gitea/services/lfs"
"github.com/golang-jwt/jwt/v5"
"github.com/kballard/go-shellquote"
"github.com/urfave/cli/v2"
2014-04-10 20:20:58 +02:00
)
2015-02-16 15:38:01 +01:00
const (
Git LFS support v2 (#122) * Import github.com/git-lfs/lfs-test-server as lfs module base Imported commit is 3968aac269a77b73924649b9412ae03f7ccd3198 Removed: Dockerfile CONTRIBUTING.md mgmt* script/ vendor/ kvlogger.go .dockerignore .gitignore README.md * Remove config, add JWT support from github.com/mgit-at/lfs-test-server Imported commit f0cdcc5a01599c5a955dc1bbf683bb4acecdba83 * Add LFS settings * Add LFS meta object model * Add LFS routes and initialization * Import github.com/dgrijalva/jwt-go into vendor/ * Adapt LFS module: handlers, routing, meta store * Move LFS routes to /user/repo/info/lfs/* * Add request header checks to LFS BatchHandler / PostHandler * Implement LFS basic authentication * Rework JWT secret generation / load * Implement LFS SSH token authentication with JWT Specification: https://github.com/github/git-lfs/tree/master/docs/api * Integrate LFS settings into install process * Remove LFS objects when repository is deleted Only removes objects from content store when deleted repo is the only referencing repository * Make LFS module stateless Fixes bug where LFS would not work after installation without restarting Gitea * Change 500 'Internal Server Error' to 400 'Bad Request' * Change sql query to xorm call * Remove unneeded type from LFS module * Change internal imports to code.gitea.io/gitea/ * Add Gitea authors copyright * Change basic auth realm to "gitea-lfs" * Add unique indexes to LFS model * Use xorm count function in LFS check on repository delete * Return io.ReadCloser from content store and close after usage * Add LFS info to runWeb() * Export LFS content store base path * LFS file download from UI * Work around git-lfs client issue with unauthenticated requests Returning a dummy Authorization header for unauthenticated requests lets git-lfs client skip asking for auth credentials See: https://github.com/github/git-lfs/issues/1088 * Fix unauthenticated UI downloads from public repositories * Authentication check order, Finish LFS file view logic * Ignore LFS hooks if installed for current OS user Fixes Gitea UI actions for repositories tracking LFS files. Checks for minimum needed git version by parsing the semantic version string. * Hide LFS metafile diff from commit view, marking as binary * Show LFS notice if file in commit view is tracked * Add notbefore/nbf JWT claim * Correct lint suggestions - comments for structs and functions - Add comments to LFS model - Function comment for GetRandomBytesAsBase64 - LFS server function comments and lint variable suggestion * Move secret generation code out of conditional Ensures no LFS code may run with an empty secret * Do not hand out JWT tokens if LFS server support is disabled
2016-12-26 02:16:37 +01:00
lfsAuthenticateVerb = "git-lfs-authenticate"
2015-02-16 15:38:01 +01:00
)
2016-11-04 12:42:18 +01:00
// CmdServ represents the available serv sub-command.
var CmdServ = &cli.Command{
2014-05-05 06:55:17 +02:00
Name: "serv",
Usage: "This command should only be called by SSH shell",
Description: "Serv provides access auth for repositories",
Before: PrepareConsoleLoggerLevel(log.FATAL),
2014-05-05 06:55:17 +02:00
Action: runServ,
Flags: []cli.Flag{
&cli.BoolFlag{
Name: "enable-pprof",
},
&cli.BoolFlag{
Name: "debug",
},
},
2014-04-10 20:20:58 +02:00
}
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
func setup(ctx context.Context, debug bool) {
if debug {
Rewrite logger system (#24726) ## ⚠️ Breaking The `log.<mode>.<logger>` style config has been dropped. If you used it, please check the new config manual & app.example.ini to make your instance output logs as expected. Although many legacy options still work, it's encouraged to upgrade to the new options. The SMTP logger is deleted because SMTP is not suitable to collect logs. If you have manually configured Gitea log options, please confirm the logger system works as expected after upgrading. ## Description Close #12082 and maybe more log-related issues, resolve some related FIXMEs in old code (which seems unfixable before) Just like rewriting queue #24505 : make code maintainable, clear legacy bugs, and add the ability to support more writers (eg: JSON, structured log) There is a new document (with examples): `logging-config.en-us.md` This PR is safer than the queue rewriting, because it's just for logging, it won't break other logic. ## The old problems The logging system is quite old and difficult to maintain: * Unclear concepts: Logger, NamedLogger, MultiChannelledLogger, SubLogger, EventLogger, WriterLogger etc * Some code is diffuclt to konw whether it is right: `log.DelNamedLogger("console")` vs `log.DelNamedLogger(log.DEFAULT)` vs `log.DelLogger("console")` * The old system heavily depends on ini config system, it's difficult to create new logger for different purpose, and it's very fragile. * The "color" trick is difficult to use and read, many colors are unnecessary, and in the future structured log could help * It's difficult to add other log formats, eg: JSON format * The log outputer doesn't have full control of its goroutine, it's difficult to make outputer have advanced behaviors * The logs could be lost in some cases: eg: no Fatal error when using CLI. * Config options are passed by JSON, which is quite fragile. * INI package makes the KEY in `[log]` section visible in `[log.sub1]` and `[log.sub1.subA]`, this behavior is quite fragile and would cause more unclear problems, and there is no strong requirement to support `log.<mode>.<logger>` syntax. ## The new design See `logger.go` for documents. ## Screenshot <details> ![image](https://github.com/go-gitea/gitea/assets/2114189/4462d713-ba39-41f5-bb08-de912e67e1ff) ![image](https://github.com/go-gitea/gitea/assets/2114189/b188035e-f691-428b-8b2d-ff7b2199b2f9) ![image](https://github.com/go-gitea/gitea/assets/2114189/132e9745-1c3b-4e00-9e0d-15eaea495dee) </details> ## TODO * [x] add some new tests * [x] fix some tests * [x] test some sub-commands (manually ....) --------- Co-authored-by: Jason Song <i@wolfogre.com> Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-22 00:35:11 +02:00
setupConsoleLogger(log.TRACE, false, os.Stderr)
} else {
Rewrite logger system (#24726) ## ⚠️ Breaking The `log.<mode>.<logger>` style config has been dropped. If you used it, please check the new config manual & app.example.ini to make your instance output logs as expected. Although many legacy options still work, it's encouraged to upgrade to the new options. The SMTP logger is deleted because SMTP is not suitable to collect logs. If you have manually configured Gitea log options, please confirm the logger system works as expected after upgrading. ## Description Close #12082 and maybe more log-related issues, resolve some related FIXMEs in old code (which seems unfixable before) Just like rewriting queue #24505 : make code maintainable, clear legacy bugs, and add the ability to support more writers (eg: JSON, structured log) There is a new document (with examples): `logging-config.en-us.md` This PR is safer than the queue rewriting, because it's just for logging, it won't break other logic. ## The old problems The logging system is quite old and difficult to maintain: * Unclear concepts: Logger, NamedLogger, MultiChannelledLogger, SubLogger, EventLogger, WriterLogger etc * Some code is diffuclt to konw whether it is right: `log.DelNamedLogger("console")` vs `log.DelNamedLogger(log.DEFAULT)` vs `log.DelLogger("console")` * The old system heavily depends on ini config system, it's difficult to create new logger for different purpose, and it's very fragile. * The "color" trick is difficult to use and read, many colors are unnecessary, and in the future structured log could help * It's difficult to add other log formats, eg: JSON format * The log outputer doesn't have full control of its goroutine, it's difficult to make outputer have advanced behaviors * The logs could be lost in some cases: eg: no Fatal error when using CLI. * Config options are passed by JSON, which is quite fragile. * INI package makes the KEY in `[log]` section visible in `[log.sub1]` and `[log.sub1.subA]`, this behavior is quite fragile and would cause more unclear problems, and there is no strong requirement to support `log.<mode>.<logger>` syntax. ## The new design See `logger.go` for documents. ## Screenshot <details> ![image](https://github.com/go-gitea/gitea/assets/2114189/4462d713-ba39-41f5-bb08-de912e67e1ff) ![image](https://github.com/go-gitea/gitea/assets/2114189/b188035e-f691-428b-8b2d-ff7b2199b2f9) ![image](https://github.com/go-gitea/gitea/assets/2114189/132e9745-1c3b-4e00-9e0d-15eaea495dee) </details> ## TODO * [x] add some new tests * [x] fix some tests * [x] test some sub-commands (manually ....) --------- Co-authored-by: Jason Song <i@wolfogre.com> Co-authored-by: delvh <dev.lh@web.de> Co-authored-by: Giteabot <teabot@gitea.io>
2023-05-22 00:35:11 +02:00
setupConsoleLogger(log.FATAL, false, os.Stderr)
}
Refactor path & config system (#25330) # The problem There were many "path tricks": * By default, Gitea uses its program directory as its work path * Gitea tries to use the "work path" to guess its "custom path" and "custom conf (app.ini)" * Users might want to use other directories as work path * The non-default work path should be passed to Gitea by GITEA_WORK_DIR or "--work-path" * But some Gitea processes are started without these values * The "serv" process started by OpenSSH server * The CLI sub-commands started by site admin * The paths are guessed by SetCustomPathAndConf again and again * The default values of "work path / custom path / custom conf" can be changed when compiling # The solution * Use `InitWorkPathAndCommonConfig` to handle these path tricks, and use test code to cover its behaviors. * When Gitea's web server runs, write the WORK_PATH to "app.ini", this value must be the most correct one, because if this value is not right, users would find that the web UI doesn't work and then they should be able to fix it. * Then all other sub-commands can use the WORK_PATH in app.ini to initialize their paths. * By the way, when Gitea starts for git protocol, it shouldn't output any log, otherwise the git protocol gets broken and client blocks forever. The "work path" priority is: WORK_PATH in app.ini > cmd arg --work-path > env var GITEA_WORK_DIR > builtin default The "app.ini" searching order is: cmd arg --config > cmd arg "work path / custom path" > env var "work path / custom path" > builtin default ## ⚠️ BREAKING If your instance's "work path / custom path / custom conf" doesn't meet the requirements (eg: work path must be absolute), Gitea will report a fatal error and exit. You need to set these values according to the error log. ---- Close #24818 Close #24222 Close #21606 Close #21498 Close #25107 Close #24981 Maybe close #24503 Replace #23301 Replace #22754 And maybe more
2023-06-21 07:50:26 +02:00
setting.MustInstalled()
if debug {
setting.RunMode = "dev"
}
// Check if setting.RepoRootPath exists. It could be the case that it doesn't exist, this can happen when
// `[repository]` `ROOT` is a relative path and $GITEA_WORK_DIR isn't passed to the SSH connection.
if _, err := os.Stat(setting.RepoRootPath); err != nil {
if os.IsNotExist(err) {
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
_ = fail(ctx, "Incorrect configuration, no repository directory.", "Directory `[repository].ROOT` %q was not found, please check if $GITEA_WORK_DIR is passed to the SSH connection or make `[repository].ROOT` an absolute value.", setting.RepoRootPath)
} else {
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
_ = fail(ctx, "Incorrect configuration, repository directory is inaccessible", "Directory `[repository].ROOT` %q is inaccessible. err: %v", setting.RepoRootPath, err)
}
return
}
if err := git.InitSimple(context.Background()); err != nil {
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
_ = fail(ctx, "Failed to init git", "Failed to init git, err: %v", err)
}
2014-05-22 03:37:13 +02:00
}
var (
allowedCommands = map[string]perm.AccessMode{
"git-upload-pack": perm.AccessModeRead,
"git-upload-archive": perm.AccessModeRead,
"git-receive-pack": perm.AccessModeWrite,
lfsAuthenticateVerb: perm.AccessModeNone,
2014-05-22 03:37:13 +02:00
}
alphaDashDotPattern = regexp.MustCompile(`[^\w-\.]`)
2014-05-22 03:37:13 +02:00
)
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
// fail prints message to stdout, it's mainly used for git serv and git hook commands.
// The output will be passed to git client and shown to user.
func fail(ctx context.Context, userMessage, logMsgFmt string, args ...any) error {
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
if userMessage == "" {
userMessage = "Internal Server Error (no specific error)"
}
// There appears to be a chance to cause a zombie process and failure to read the Exit status
// if nothing is outputted on stdout.
_, _ = fmt.Fprintln(os.Stdout, "")
_, _ = fmt.Fprintln(os.Stderr, "Gitea:", userMessage)
2015-11-08 20:31:49 +01:00
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
if logMsgFmt != "" {
logMsg := fmt.Sprintf(logMsgFmt, args...)
if !setting.IsProd {
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
_, _ = fmt.Fprintln(os.Stderr, "Gitea:", logMsg)
}
if userMessage != "" {
if unicode.IsPunct(rune(userMessage[len(userMessage)-1])) {
logMsg = userMessage + " " + logMsg
} else {
logMsg = userMessage + ". " + logMsg
}
}
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
_ = private.SSHLog(ctx, true, logMsg)
2015-11-08 20:31:49 +01:00
}
return cli.Exit("", 1)
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
}
2015-11-08 20:31:49 +01:00
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
// handleCliResponseExtra handles the extra response from the cli sub-commands
// If there is a user message it will be printed to stdout
// If the command failed it will return an error (the error will be printed by cli framework)
func handleCliResponseExtra(extra private.ResponseExtra) error {
if extra.UserMsg != "" {
_, _ = fmt.Fprintln(os.Stdout, extra.UserMsg)
}
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
if extra.HasError() {
return cli.Exit(extra.Error, 1)
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
}
return nil
2015-08-06 16:48:11 +02:00
}
func runServ(c *cli.Context) error {
ctx, cancel := installSignals()
defer cancel()
// FIXME: This needs to internationalised
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
setup(ctx, c.Bool("debug"))
2014-04-10 20:20:58 +02:00
if setting.SSH.Disabled {
println("Gitea: SSH has been disabled")
return nil
2016-02-22 03:55:59 +01:00
}
if c.NArg() < 1 {
2019-06-12 21:41:28 +02:00
if err := cli.ShowSubcommandHelp(c); err != nil {
fmt.Printf("error showing subcommand help: %v\n", err)
}
return nil
}
2015-02-16 15:38:01 +01:00
keys := strings.Split(c.Args().First(), "-")
if len(keys) != 2 || keys[0] != "key" {
return fail(ctx, "Key ID format error", "Invalid key argument: %s", c.Args().First())
}
keyID, err := strconv.ParseInt(keys[1], 10, 64)
if err != nil {
return fail(ctx, "Key ID parsing error", "Invalid key argument: %s", c.Args().Get(1))
}
2014-04-10 20:20:58 +02:00
cmd := os.Getenv("SSH_ORIGINAL_COMMAND")
2015-08-05 05:14:17 +02:00
if len(cmd) == 0 {
key, user, err := private.ServNoCommand(ctx, keyID)
if err != nil {
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
return fail(ctx, "Key check failed", "Failed to check provided key: %v", err)
}
Add ssh certificate support (#12281) * Add ssh certificate support * Add ssh certificate support to builtin ssh * Write trusted-user-ca-keys.pem based on configuration * Update app.example.ini * Update templates/user/settings/keys_principal.tmpl Co-authored-by: silverwind <me@silverwind.io> * Remove unused locale string * Update options/locale/locale_en-US.ini Co-authored-by: silverwind <me@silverwind.io> * Update options/locale/locale_en-US.ini Co-authored-by: silverwind <me@silverwind.io> * Update models/ssh_key.go Co-authored-by: silverwind <me@silverwind.io> * Add missing creation of SSH.Rootpath * Update cheatsheet, example and locale strings * Update models/ssh_key.go Co-authored-by: zeripath <art27@cantab.net> * Update models/ssh_key.go Co-authored-by: zeripath <art27@cantab.net> * Update models/ssh_key.go Co-authored-by: zeripath <art27@cantab.net> * Update models/ssh_key.go Co-authored-by: zeripath <art27@cantab.net> * Update models/ssh_key.go * Optimizations based on feedback * Validate CA keys for external sshd * Add filename option and change default filename Add a SSH_TRUSTED_USER_CA_KEYS_FILENAME option which default is RUN_USER/.ssh/gitea-trusted-user-ca-keys.pem Do not write a file when SSH_TRUSTED_USER_CA_KEYS is empty. Add some more documentation. * Remove unneeded principalkey functions * Add blank line * Apply suggestions from code review Co-authored-by: zeripath <art27@cantab.net> * Add SSH_AUTHORIZED_PRINCIPALS_ALLOW option This adds a SSH_AUTHORIZED_PRINCIPALS_ALLOW which is default email,username this means that users only can add the principals that match their email or username. To allow anything the admin need to set the option anything. This allows for a safe default in gitea which protects against malicious users using other user's prinicipals. (before that user could set it). This commit also has some small other fixes from the last code review. * Rewrite principal keys file on user deletion * Use correct rewrite method * Set correct AuthorizedPrincipalsBackup default setting * Rewrite principalsfile when adding principals * Add update authorized_principals option to admin dashboard * Handle non-primary emails Signed-off-by: Andrew Thornton <art27@cantab.net> * Add the command actually to the dashboard template * Update models/ssh_key.go Co-authored-by: silverwind <me@silverwind.io> * By default do not show principal options unless there are CA keys set or they are explicitly set Signed-off-by: Andrew Thornton <art27@cantab.net> * allow settings when enabled * Fix typos in TrustedUserCAKeys path * Allow every CASignatureAlgorithms algorithm As this depends on the content of TrustedUserCAKeys we should allow all signature algorithms as admins can choose the specific algorithm on their signing CA * Update models/ssh_key.go Co-authored-by: Lauris BH <lauris@nix.lv> * Fix linting issue Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: zeripath <art27@cantab.net> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: techknowlogick <matti@mdranta.net> Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-10-11 02:38:09 +02:00
switch key.Type {
case asymkey_model.KeyTypeDeploy:
println("Hi there! You've successfully authenticated with the deploy key named " + key.Name + ", but Gitea does not provide shell access.")
case asymkey_model.KeyTypePrincipal:
Add ssh certificate support (#12281) * Add ssh certificate support * Add ssh certificate support to builtin ssh * Write trusted-user-ca-keys.pem based on configuration * Update app.example.ini * Update templates/user/settings/keys_principal.tmpl Co-authored-by: silverwind <me@silverwind.io> * Remove unused locale string * Update options/locale/locale_en-US.ini Co-authored-by: silverwind <me@silverwind.io> * Update options/locale/locale_en-US.ini Co-authored-by: silverwind <me@silverwind.io> * Update models/ssh_key.go Co-authored-by: silverwind <me@silverwind.io> * Add missing creation of SSH.Rootpath * Update cheatsheet, example and locale strings * Update models/ssh_key.go Co-authored-by: zeripath <art27@cantab.net> * Update models/ssh_key.go Co-authored-by: zeripath <art27@cantab.net> * Update models/ssh_key.go Co-authored-by: zeripath <art27@cantab.net> * Update models/ssh_key.go Co-authored-by: zeripath <art27@cantab.net> * Update models/ssh_key.go * Optimizations based on feedback * Validate CA keys for external sshd * Add filename option and change default filename Add a SSH_TRUSTED_USER_CA_KEYS_FILENAME option which default is RUN_USER/.ssh/gitea-trusted-user-ca-keys.pem Do not write a file when SSH_TRUSTED_USER_CA_KEYS is empty. Add some more documentation. * Remove unneeded principalkey functions * Add blank line * Apply suggestions from code review Co-authored-by: zeripath <art27@cantab.net> * Add SSH_AUTHORIZED_PRINCIPALS_ALLOW option This adds a SSH_AUTHORIZED_PRINCIPALS_ALLOW which is default email,username this means that users only can add the principals that match their email or username. To allow anything the admin need to set the option anything. This allows for a safe default in gitea which protects against malicious users using other user's prinicipals. (before that user could set it). This commit also has some small other fixes from the last code review. * Rewrite principal keys file on user deletion * Use correct rewrite method * Set correct AuthorizedPrincipalsBackup default setting * Rewrite principalsfile when adding principals * Add update authorized_principals option to admin dashboard * Handle non-primary emails Signed-off-by: Andrew Thornton <art27@cantab.net> * Add the command actually to the dashboard template * Update models/ssh_key.go Co-authored-by: silverwind <me@silverwind.io> * By default do not show principal options unless there are CA keys set or they are explicitly set Signed-off-by: Andrew Thornton <art27@cantab.net> * allow settings when enabled * Fix typos in TrustedUserCAKeys path * Allow every CASignatureAlgorithms algorithm As this depends on the content of TrustedUserCAKeys we should allow all signature algorithms as admins can choose the specific algorithm on their signing CA * Update models/ssh_key.go Co-authored-by: Lauris BH <lauris@nix.lv> * Fix linting issue Co-authored-by: silverwind <me@silverwind.io> Co-authored-by: zeripath <art27@cantab.net> Co-authored-by: Lauris BH <lauris@nix.lv> Co-authored-by: techknowlogick <matti@mdranta.net> Co-authored-by: techknowlogick <techknowlogick@gitea.io>
2020-10-11 02:38:09 +02:00
println("Hi there! You've successfully authenticated with the principal " + key.Content + ", but Gitea does not provide shell access.")
default:
println("Hi there, " + user.Name + "! You've successfully authenticated with the key named " + key.Name + ", but Gitea does not provide shell access.")
}
println("If this is unexpected, please log in with password and setup Gitea under another user.")
return nil
} else if c.Bool("debug") {
log.Debug("SSH_ORIGINAL_COMMAND: %s", os.Getenv("SSH_ORIGINAL_COMMAND"))
2014-04-10 20:20:58 +02:00
}
words, err := shellquote.Split(cmd)
if err != nil {
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
return fail(ctx, "Error parsing arguments", "Failed to parse arguments: %v", err)
}
if len(words) < 2 {
if git.CheckGitVersionAtLeast("2.29") == nil {
// for AGit Flow
if cmd == "ssh_info" {
fmt.Print(`{"type":"gitea","version":1}`)
return nil
}
}
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
return fail(ctx, "Too few arguments", "Too few arguments in cmd: %s", cmd)
}
verb := words[0]
repoPath := words[1]
if repoPath[0] == '/' {
repoPath = repoPath[1:]
}
Git LFS support v2 (#122) * Import github.com/git-lfs/lfs-test-server as lfs module base Imported commit is 3968aac269a77b73924649b9412ae03f7ccd3198 Removed: Dockerfile CONTRIBUTING.md mgmt* script/ vendor/ kvlogger.go .dockerignore .gitignore README.md * Remove config, add JWT support from github.com/mgit-at/lfs-test-server Imported commit f0cdcc5a01599c5a955dc1bbf683bb4acecdba83 * Add LFS settings * Add LFS meta object model * Add LFS routes and initialization * Import github.com/dgrijalva/jwt-go into vendor/ * Adapt LFS module: handlers, routing, meta store * Move LFS routes to /user/repo/info/lfs/* * Add request header checks to LFS BatchHandler / PostHandler * Implement LFS basic authentication * Rework JWT secret generation / load * Implement LFS SSH token authentication with JWT Specification: https://github.com/github/git-lfs/tree/master/docs/api * Integrate LFS settings into install process * Remove LFS objects when repository is deleted Only removes objects from content store when deleted repo is the only referencing repository * Make LFS module stateless Fixes bug where LFS would not work after installation without restarting Gitea * Change 500 'Internal Server Error' to 400 'Bad Request' * Change sql query to xorm call * Remove unneeded type from LFS module * Change internal imports to code.gitea.io/gitea/ * Add Gitea authors copyright * Change basic auth realm to "gitea-lfs" * Add unique indexes to LFS model * Use xorm count function in LFS check on repository delete * Return io.ReadCloser from content store and close after usage * Add LFS info to runWeb() * Export LFS content store base path * LFS file download from UI * Work around git-lfs client issue with unauthenticated requests Returning a dummy Authorization header for unauthenticated requests lets git-lfs client skip asking for auth credentials See: https://github.com/github/git-lfs/issues/1088 * Fix unauthenticated UI downloads from public repositories * Authentication check order, Finish LFS file view logic * Ignore LFS hooks if installed for current OS user Fixes Gitea UI actions for repositories tracking LFS files. Checks for minimum needed git version by parsing the semantic version string. * Hide LFS metafile diff from commit view, marking as binary * Show LFS notice if file in commit view is tracked * Add notbefore/nbf JWT claim * Correct lint suggestions - comments for structs and functions - Add comments to LFS model - Function comment for GetRandomBytesAsBase64 - LFS server function comments and lint variable suggestion * Move secret generation code out of conditional Ensures no LFS code may run with an empty secret * Do not hand out JWT tokens if LFS server support is disabled
2016-12-26 02:16:37 +01:00
var lfsVerb string
if verb == lfsAuthenticateVerb {
if !setting.LFS.StartServer {
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
return fail(ctx, "Unknown git command", "LFS authentication request over SSH denied, LFS support is disabled")
Git LFS support v2 (#122) * Import github.com/git-lfs/lfs-test-server as lfs module base Imported commit is 3968aac269a77b73924649b9412ae03f7ccd3198 Removed: Dockerfile CONTRIBUTING.md mgmt* script/ vendor/ kvlogger.go .dockerignore .gitignore README.md * Remove config, add JWT support from github.com/mgit-at/lfs-test-server Imported commit f0cdcc5a01599c5a955dc1bbf683bb4acecdba83 * Add LFS settings * Add LFS meta object model * Add LFS routes and initialization * Import github.com/dgrijalva/jwt-go into vendor/ * Adapt LFS module: handlers, routing, meta store * Move LFS routes to /user/repo/info/lfs/* * Add request header checks to LFS BatchHandler / PostHandler * Implement LFS basic authentication * Rework JWT secret generation / load * Implement LFS SSH token authentication with JWT Specification: https://github.com/github/git-lfs/tree/master/docs/api * Integrate LFS settings into install process * Remove LFS objects when repository is deleted Only removes objects from content store when deleted repo is the only referencing repository * Make LFS module stateless Fixes bug where LFS would not work after installation without restarting Gitea * Change 500 'Internal Server Error' to 400 'Bad Request' * Change sql query to xorm call * Remove unneeded type from LFS module * Change internal imports to code.gitea.io/gitea/ * Add Gitea authors copyright * Change basic auth realm to "gitea-lfs" * Add unique indexes to LFS model * Use xorm count function in LFS check on repository delete * Return io.ReadCloser from content store and close after usage * Add LFS info to runWeb() * Export LFS content store base path * LFS file download from UI * Work around git-lfs client issue with unauthenticated requests Returning a dummy Authorization header for unauthenticated requests lets git-lfs client skip asking for auth credentials See: https://github.com/github/git-lfs/issues/1088 * Fix unauthenticated UI downloads from public repositories * Authentication check order, Finish LFS file view logic * Ignore LFS hooks if installed for current OS user Fixes Gitea UI actions for repositories tracking LFS files. Checks for minimum needed git version by parsing the semantic version string. * Hide LFS metafile diff from commit view, marking as binary * Show LFS notice if file in commit view is tracked * Add notbefore/nbf JWT claim * Correct lint suggestions - comments for structs and functions - Add comments to LFS model - Function comment for GetRandomBytesAsBase64 - LFS server function comments and lint variable suggestion * Move secret generation code out of conditional Ensures no LFS code may run with an empty secret * Do not hand out JWT tokens if LFS server support is disabled
2016-12-26 02:16:37 +01:00
}
if len(words) > 2 {
lfsVerb = words[2]
Git LFS support v2 (#122) * Import github.com/git-lfs/lfs-test-server as lfs module base Imported commit is 3968aac269a77b73924649b9412ae03f7ccd3198 Removed: Dockerfile CONTRIBUTING.md mgmt* script/ vendor/ kvlogger.go .dockerignore .gitignore README.md * Remove config, add JWT support from github.com/mgit-at/lfs-test-server Imported commit f0cdcc5a01599c5a955dc1bbf683bb4acecdba83 * Add LFS settings * Add LFS meta object model * Add LFS routes and initialization * Import github.com/dgrijalva/jwt-go into vendor/ * Adapt LFS module: handlers, routing, meta store * Move LFS routes to /user/repo/info/lfs/* * Add request header checks to LFS BatchHandler / PostHandler * Implement LFS basic authentication * Rework JWT secret generation / load * Implement LFS SSH token authentication with JWT Specification: https://github.com/github/git-lfs/tree/master/docs/api * Integrate LFS settings into install process * Remove LFS objects when repository is deleted Only removes objects from content store when deleted repo is the only referencing repository * Make LFS module stateless Fixes bug where LFS would not work after installation without restarting Gitea * Change 500 'Internal Server Error' to 400 'Bad Request' * Change sql query to xorm call * Remove unneeded type from LFS module * Change internal imports to code.gitea.io/gitea/ * Add Gitea authors copyright * Change basic auth realm to "gitea-lfs" * Add unique indexes to LFS model * Use xorm count function in LFS check on repository delete * Return io.ReadCloser from content store and close after usage * Add LFS info to runWeb() * Export LFS content store base path * LFS file download from UI * Work around git-lfs client issue with unauthenticated requests Returning a dummy Authorization header for unauthenticated requests lets git-lfs client skip asking for auth credentials See: https://github.com/github/git-lfs/issues/1088 * Fix unauthenticated UI downloads from public repositories * Authentication check order, Finish LFS file view logic * Ignore LFS hooks if installed for current OS user Fixes Gitea UI actions for repositories tracking LFS files. Checks for minimum needed git version by parsing the semantic version string. * Hide LFS metafile diff from commit view, marking as binary * Show LFS notice if file in commit view is tracked * Add notbefore/nbf JWT claim * Correct lint suggestions - comments for structs and functions - Add comments to LFS model - Function comment for GetRandomBytesAsBase64 - LFS server function comments and lint variable suggestion * Move secret generation code out of conditional Ensures no LFS code may run with an empty secret * Do not hand out JWT tokens if LFS server support is disabled
2016-12-26 02:16:37 +01:00
}
}
// LowerCase and trim the repoPath as that's how they are stored.
repoPath = strings.ToLower(strings.TrimSpace(repoPath))
2014-04-10 20:20:58 +02:00
rr := strings.SplitN(repoPath, "/", 2)
if len(rr) != 2 {
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
return fail(ctx, "Invalid repository path", "Invalid repository path: %v", repoPath)
2014-04-10 20:20:58 +02:00
}
2015-12-01 02:45:55 +01:00
username := strings.ToLower(rr[0])
reponame := strings.ToLower(strings.TrimSuffix(rr[1], ".git"))
if alphaDashDotPattern.MatchString(reponame) {
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
return fail(ctx, "Invalid repo name", "Invalid repo name: %s", reponame)
}
if c.Bool("enable-pprof") {
if err := os.MkdirAll(setting.PprofDataPath, os.ModePerm); err != nil {
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
return fail(ctx, "Error while trying to create PPROF_DATA_PATH", "Error while trying to create PPROF_DATA_PATH: %v", err)
}
stopCPUProfiler, err := pprof.DumpCPUProfileForUsername(setting.PprofDataPath, username)
if err != nil {
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
return fail(ctx, "Unable to start CPU profiler", "Unable to start CPU profile: %v", err)
}
defer func() {
stopCPUProfiler()
err := pprof.DumpMemProfileForUsername(setting.PprofDataPath, username)
if err != nil {
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
_ = fail(ctx, "Unable to dump Mem profile", "Unable to dump Mem Profile: %v", err)
}
}()
}
requestedMode, has := allowedCommands[verb]
2015-02-16 15:38:01 +01:00
if !has {
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
return fail(ctx, "Unknown git command", "Unknown git command %s", verb)
2015-02-16 15:38:01 +01:00
}
2014-04-10 20:20:58 +02:00
Git LFS support v2 (#122) * Import github.com/git-lfs/lfs-test-server as lfs module base Imported commit is 3968aac269a77b73924649b9412ae03f7ccd3198 Removed: Dockerfile CONTRIBUTING.md mgmt* script/ vendor/ kvlogger.go .dockerignore .gitignore README.md * Remove config, add JWT support from github.com/mgit-at/lfs-test-server Imported commit f0cdcc5a01599c5a955dc1bbf683bb4acecdba83 * Add LFS settings * Add LFS meta object model * Add LFS routes and initialization * Import github.com/dgrijalva/jwt-go into vendor/ * Adapt LFS module: handlers, routing, meta store * Move LFS routes to /user/repo/info/lfs/* * Add request header checks to LFS BatchHandler / PostHandler * Implement LFS basic authentication * Rework JWT secret generation / load * Implement LFS SSH token authentication with JWT Specification: https://github.com/github/git-lfs/tree/master/docs/api * Integrate LFS settings into install process * Remove LFS objects when repository is deleted Only removes objects from content store when deleted repo is the only referencing repository * Make LFS module stateless Fixes bug where LFS would not work after installation without restarting Gitea * Change 500 'Internal Server Error' to 400 'Bad Request' * Change sql query to xorm call * Remove unneeded type from LFS module * Change internal imports to code.gitea.io/gitea/ * Add Gitea authors copyright * Change basic auth realm to "gitea-lfs" * Add unique indexes to LFS model * Use xorm count function in LFS check on repository delete * Return io.ReadCloser from content store and close after usage * Add LFS info to runWeb() * Export LFS content store base path * LFS file download from UI * Work around git-lfs client issue with unauthenticated requests Returning a dummy Authorization header for unauthenticated requests lets git-lfs client skip asking for auth credentials See: https://github.com/github/git-lfs/issues/1088 * Fix unauthenticated UI downloads from public repositories * Authentication check order, Finish LFS file view logic * Ignore LFS hooks if installed for current OS user Fixes Gitea UI actions for repositories tracking LFS files. Checks for minimum needed git version by parsing the semantic version string. * Hide LFS metafile diff from commit view, marking as binary * Show LFS notice if file in commit view is tracked * Add notbefore/nbf JWT claim * Correct lint suggestions - comments for structs and functions - Add comments to LFS model - Function comment for GetRandomBytesAsBase64 - LFS server function comments and lint variable suggestion * Move secret generation code out of conditional Ensures no LFS code may run with an empty secret * Do not hand out JWT tokens if LFS server support is disabled
2016-12-26 02:16:37 +01:00
if verb == lfsAuthenticateVerb {
if lfsVerb == "upload" {
requestedMode = perm.AccessModeWrite
} else if lfsVerb == "download" {
requestedMode = perm.AccessModeRead
} else {
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
return fail(ctx, "Unknown LFS verb", "Unknown lfs verb %s", lfsVerb)
Git LFS support v2 (#122) * Import github.com/git-lfs/lfs-test-server as lfs module base Imported commit is 3968aac269a77b73924649b9412ae03f7ccd3198 Removed: Dockerfile CONTRIBUTING.md mgmt* script/ vendor/ kvlogger.go .dockerignore .gitignore README.md * Remove config, add JWT support from github.com/mgit-at/lfs-test-server Imported commit f0cdcc5a01599c5a955dc1bbf683bb4acecdba83 * Add LFS settings * Add LFS meta object model * Add LFS routes and initialization * Import github.com/dgrijalva/jwt-go into vendor/ * Adapt LFS module: handlers, routing, meta store * Move LFS routes to /user/repo/info/lfs/* * Add request header checks to LFS BatchHandler / PostHandler * Implement LFS basic authentication * Rework JWT secret generation / load * Implement LFS SSH token authentication with JWT Specification: https://github.com/github/git-lfs/tree/master/docs/api * Integrate LFS settings into install process * Remove LFS objects when repository is deleted Only removes objects from content store when deleted repo is the only referencing repository * Make LFS module stateless Fixes bug where LFS would not work after installation without restarting Gitea * Change 500 'Internal Server Error' to 400 'Bad Request' * Change sql query to xorm call * Remove unneeded type from LFS module * Change internal imports to code.gitea.io/gitea/ * Add Gitea authors copyright * Change basic auth realm to "gitea-lfs" * Add unique indexes to LFS model * Use xorm count function in LFS check on repository delete * Return io.ReadCloser from content store and close after usage * Add LFS info to runWeb() * Export LFS content store base path * LFS file download from UI * Work around git-lfs client issue with unauthenticated requests Returning a dummy Authorization header for unauthenticated requests lets git-lfs client skip asking for auth credentials See: https://github.com/github/git-lfs/issues/1088 * Fix unauthenticated UI downloads from public repositories * Authentication check order, Finish LFS file view logic * Ignore LFS hooks if installed for current OS user Fixes Gitea UI actions for repositories tracking LFS files. Checks for minimum needed git version by parsing the semantic version string. * Hide LFS metafile diff from commit view, marking as binary * Show LFS notice if file in commit view is tracked * Add notbefore/nbf JWT claim * Correct lint suggestions - comments for structs and functions - Add comments to LFS model - Function comment for GetRandomBytesAsBase64 - LFS server function comments and lint variable suggestion * Move secret generation code out of conditional Ensures no LFS code may run with an empty secret * Do not hand out JWT tokens if LFS server support is disabled
2016-12-26 02:16:37 +01:00
}
}
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
results, extra := private.ServCommand(ctx, keyID, username, reponame, requestedMode, verb, lfsVerb)
if extra.HasError() {
return fail(ctx, extra.UserMsg, "ServCommand failed: %s", extra.Error)
2014-04-10 20:20:58 +02:00
}
// LFS token authentication
Git LFS support v2 (#122) * Import github.com/git-lfs/lfs-test-server as lfs module base Imported commit is 3968aac269a77b73924649b9412ae03f7ccd3198 Removed: Dockerfile CONTRIBUTING.md mgmt* script/ vendor/ kvlogger.go .dockerignore .gitignore README.md * Remove config, add JWT support from github.com/mgit-at/lfs-test-server Imported commit f0cdcc5a01599c5a955dc1bbf683bb4acecdba83 * Add LFS settings * Add LFS meta object model * Add LFS routes and initialization * Import github.com/dgrijalva/jwt-go into vendor/ * Adapt LFS module: handlers, routing, meta store * Move LFS routes to /user/repo/info/lfs/* * Add request header checks to LFS BatchHandler / PostHandler * Implement LFS basic authentication * Rework JWT secret generation / load * Implement LFS SSH token authentication with JWT Specification: https://github.com/github/git-lfs/tree/master/docs/api * Integrate LFS settings into install process * Remove LFS objects when repository is deleted Only removes objects from content store when deleted repo is the only referencing repository * Make LFS module stateless Fixes bug where LFS would not work after installation without restarting Gitea * Change 500 'Internal Server Error' to 400 'Bad Request' * Change sql query to xorm call * Remove unneeded type from LFS module * Change internal imports to code.gitea.io/gitea/ * Add Gitea authors copyright * Change basic auth realm to "gitea-lfs" * Add unique indexes to LFS model * Use xorm count function in LFS check on repository delete * Return io.ReadCloser from content store and close after usage * Add LFS info to runWeb() * Export LFS content store base path * LFS file download from UI * Work around git-lfs client issue with unauthenticated requests Returning a dummy Authorization header for unauthenticated requests lets git-lfs client skip asking for auth credentials See: https://github.com/github/git-lfs/issues/1088 * Fix unauthenticated UI downloads from public repositories * Authentication check order, Finish LFS file view logic * Ignore LFS hooks if installed for current OS user Fixes Gitea UI actions for repositories tracking LFS files. Checks for minimum needed git version by parsing the semantic version string. * Hide LFS metafile diff from commit view, marking as binary * Show LFS notice if file in commit view is tracked * Add notbefore/nbf JWT claim * Correct lint suggestions - comments for structs and functions - Add comments to LFS model - Function comment for GetRandomBytesAsBase64 - LFS server function comments and lint variable suggestion * Move secret generation code out of conditional Ensures no LFS code may run with an empty secret * Do not hand out JWT tokens if LFS server support is disabled
2016-12-26 02:16:37 +01:00
if verb == lfsAuthenticateVerb {
url := fmt.Sprintf("%s%s/%s.git/info/lfs", setting.AppURL, url.PathEscape(results.OwnerName), url.PathEscape(results.RepoName))
Git LFS support v2 (#122) * Import github.com/git-lfs/lfs-test-server as lfs module base Imported commit is 3968aac269a77b73924649b9412ae03f7ccd3198 Removed: Dockerfile CONTRIBUTING.md mgmt* script/ vendor/ kvlogger.go .dockerignore .gitignore README.md * Remove config, add JWT support from github.com/mgit-at/lfs-test-server Imported commit f0cdcc5a01599c5a955dc1bbf683bb4acecdba83 * Add LFS settings * Add LFS meta object model * Add LFS routes and initialization * Import github.com/dgrijalva/jwt-go into vendor/ * Adapt LFS module: handlers, routing, meta store * Move LFS routes to /user/repo/info/lfs/* * Add request header checks to LFS BatchHandler / PostHandler * Implement LFS basic authentication * Rework JWT secret generation / load * Implement LFS SSH token authentication with JWT Specification: https://github.com/github/git-lfs/tree/master/docs/api * Integrate LFS settings into install process * Remove LFS objects when repository is deleted Only removes objects from content store when deleted repo is the only referencing repository * Make LFS module stateless Fixes bug where LFS would not work after installation without restarting Gitea * Change 500 'Internal Server Error' to 400 'Bad Request' * Change sql query to xorm call * Remove unneeded type from LFS module * Change internal imports to code.gitea.io/gitea/ * Add Gitea authors copyright * Change basic auth realm to "gitea-lfs" * Add unique indexes to LFS model * Use xorm count function in LFS check on repository delete * Return io.ReadCloser from content store and close after usage * Add LFS info to runWeb() * Export LFS content store base path * LFS file download from UI * Work around git-lfs client issue with unauthenticated requests Returning a dummy Authorization header for unauthenticated requests lets git-lfs client skip asking for auth credentials See: https://github.com/github/git-lfs/issues/1088 * Fix unauthenticated UI downloads from public repositories * Authentication check order, Finish LFS file view logic * Ignore LFS hooks if installed for current OS user Fixes Gitea UI actions for repositories tracking LFS files. Checks for minimum needed git version by parsing the semantic version string. * Hide LFS metafile diff from commit view, marking as binary * Show LFS notice if file in commit view is tracked * Add notbefore/nbf JWT claim * Correct lint suggestions - comments for structs and functions - Add comments to LFS model - Function comment for GetRandomBytesAsBase64 - LFS server function comments and lint variable suggestion * Move secret generation code out of conditional Ensures no LFS code may run with an empty secret * Do not hand out JWT tokens if LFS server support is disabled
2016-12-26 02:16:37 +01:00
now := time.Now()
claims := lfs.Claims{
RegisteredClaims: jwt.RegisteredClaims{
ExpiresAt: jwt.NewNumericDate(now.Add(setting.LFS.HTTPAuthExpiry)),
NotBefore: jwt.NewNumericDate(now),
},
RepoID: results.RepoID,
Op: lfsVerb,
UserID: results.UserID,
}
token := jwt.NewWithClaims(jwt.SigningMethodHS256, claims)
Git LFS support v2 (#122) * Import github.com/git-lfs/lfs-test-server as lfs module base Imported commit is 3968aac269a77b73924649b9412ae03f7ccd3198 Removed: Dockerfile CONTRIBUTING.md mgmt* script/ vendor/ kvlogger.go .dockerignore .gitignore README.md * Remove config, add JWT support from github.com/mgit-at/lfs-test-server Imported commit f0cdcc5a01599c5a955dc1bbf683bb4acecdba83 * Add LFS settings * Add LFS meta object model * Add LFS routes and initialization * Import github.com/dgrijalva/jwt-go into vendor/ * Adapt LFS module: handlers, routing, meta store * Move LFS routes to /user/repo/info/lfs/* * Add request header checks to LFS BatchHandler / PostHandler * Implement LFS basic authentication * Rework JWT secret generation / load * Implement LFS SSH token authentication with JWT Specification: https://github.com/github/git-lfs/tree/master/docs/api * Integrate LFS settings into install process * Remove LFS objects when repository is deleted Only removes objects from content store when deleted repo is the only referencing repository * Make LFS module stateless Fixes bug where LFS would not work after installation without restarting Gitea * Change 500 'Internal Server Error' to 400 'Bad Request' * Change sql query to xorm call * Remove unneeded type from LFS module * Change internal imports to code.gitea.io/gitea/ * Add Gitea authors copyright * Change basic auth realm to "gitea-lfs" * Add unique indexes to LFS model * Use xorm count function in LFS check on repository delete * Return io.ReadCloser from content store and close after usage * Add LFS info to runWeb() * Export LFS content store base path * LFS file download from UI * Work around git-lfs client issue with unauthenticated requests Returning a dummy Authorization header for unauthenticated requests lets git-lfs client skip asking for auth credentials See: https://github.com/github/git-lfs/issues/1088 * Fix unauthenticated UI downloads from public repositories * Authentication check order, Finish LFS file view logic * Ignore LFS hooks if installed for current OS user Fixes Gitea UI actions for repositories tracking LFS files. Checks for minimum needed git version by parsing the semantic version string. * Hide LFS metafile diff from commit view, marking as binary * Show LFS notice if file in commit view is tracked * Add notbefore/nbf JWT claim * Correct lint suggestions - comments for structs and functions - Add comments to LFS model - Function comment for GetRandomBytesAsBase64 - LFS server function comments and lint variable suggestion * Move secret generation code out of conditional Ensures no LFS code may run with an empty secret * Do not hand out JWT tokens if LFS server support is disabled
2016-12-26 02:16:37 +01:00
// Sign and get the complete encoded token as a string using the secret
tokenString, err := token.SignedString(setting.LFS.JWTSecretBytes)
if err != nil {
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
return fail(ctx, "Failed to sign JWT Token", "Failed to sign JWT token: %v", err)
Git LFS support v2 (#122) * Import github.com/git-lfs/lfs-test-server as lfs module base Imported commit is 3968aac269a77b73924649b9412ae03f7ccd3198 Removed: Dockerfile CONTRIBUTING.md mgmt* script/ vendor/ kvlogger.go .dockerignore .gitignore README.md * Remove config, add JWT support from github.com/mgit-at/lfs-test-server Imported commit f0cdcc5a01599c5a955dc1bbf683bb4acecdba83 * Add LFS settings * Add LFS meta object model * Add LFS routes and initialization * Import github.com/dgrijalva/jwt-go into vendor/ * Adapt LFS module: handlers, routing, meta store * Move LFS routes to /user/repo/info/lfs/* * Add request header checks to LFS BatchHandler / PostHandler * Implement LFS basic authentication * Rework JWT secret generation / load * Implement LFS SSH token authentication with JWT Specification: https://github.com/github/git-lfs/tree/master/docs/api * Integrate LFS settings into install process * Remove LFS objects when repository is deleted Only removes objects from content store when deleted repo is the only referencing repository * Make LFS module stateless Fixes bug where LFS would not work after installation without restarting Gitea * Change 500 'Internal Server Error' to 400 'Bad Request' * Change sql query to xorm call * Remove unneeded type from LFS module * Change internal imports to code.gitea.io/gitea/ * Add Gitea authors copyright * Change basic auth realm to "gitea-lfs" * Add unique indexes to LFS model * Use xorm count function in LFS check on repository delete * Return io.ReadCloser from content store and close after usage * Add LFS info to runWeb() * Export LFS content store base path * LFS file download from UI * Work around git-lfs client issue with unauthenticated requests Returning a dummy Authorization header for unauthenticated requests lets git-lfs client skip asking for auth credentials See: https://github.com/github/git-lfs/issues/1088 * Fix unauthenticated UI downloads from public repositories * Authentication check order, Finish LFS file view logic * Ignore LFS hooks if installed for current OS user Fixes Gitea UI actions for repositories tracking LFS files. Checks for minimum needed git version by parsing the semantic version string. * Hide LFS metafile diff from commit view, marking as binary * Show LFS notice if file in commit view is tracked * Add notbefore/nbf JWT claim * Correct lint suggestions - comments for structs and functions - Add comments to LFS model - Function comment for GetRandomBytesAsBase64 - LFS server function comments and lint variable suggestion * Move secret generation code out of conditional Ensures no LFS code may run with an empty secret * Do not hand out JWT tokens if LFS server support is disabled
2016-12-26 02:16:37 +01:00
}
tokenAuthentication := &git_model.LFSTokenResponse{
Git LFS support v2 (#122) * Import github.com/git-lfs/lfs-test-server as lfs module base Imported commit is 3968aac269a77b73924649b9412ae03f7ccd3198 Removed: Dockerfile CONTRIBUTING.md mgmt* script/ vendor/ kvlogger.go .dockerignore .gitignore README.md * Remove config, add JWT support from github.com/mgit-at/lfs-test-server Imported commit f0cdcc5a01599c5a955dc1bbf683bb4acecdba83 * Add LFS settings * Add LFS meta object model * Add LFS routes and initialization * Import github.com/dgrijalva/jwt-go into vendor/ * Adapt LFS module: handlers, routing, meta store * Move LFS routes to /user/repo/info/lfs/* * Add request header checks to LFS BatchHandler / PostHandler * Implement LFS basic authentication * Rework JWT secret generation / load * Implement LFS SSH token authentication with JWT Specification: https://github.com/github/git-lfs/tree/master/docs/api * Integrate LFS settings into install process * Remove LFS objects when repository is deleted Only removes objects from content store when deleted repo is the only referencing repository * Make LFS module stateless Fixes bug where LFS would not work after installation without restarting Gitea * Change 500 'Internal Server Error' to 400 'Bad Request' * Change sql query to xorm call * Remove unneeded type from LFS module * Change internal imports to code.gitea.io/gitea/ * Add Gitea authors copyright * Change basic auth realm to "gitea-lfs" * Add unique indexes to LFS model * Use xorm count function in LFS check on repository delete * Return io.ReadCloser from content store and close after usage * Add LFS info to runWeb() * Export LFS content store base path * LFS file download from UI * Work around git-lfs client issue with unauthenticated requests Returning a dummy Authorization header for unauthenticated requests lets git-lfs client skip asking for auth credentials See: https://github.com/github/git-lfs/issues/1088 * Fix unauthenticated UI downloads from public repositories * Authentication check order, Finish LFS file view logic * Ignore LFS hooks if installed for current OS user Fixes Gitea UI actions for repositories tracking LFS files. Checks for minimum needed git version by parsing the semantic version string. * Hide LFS metafile diff from commit view, marking as binary * Show LFS notice if file in commit view is tracked * Add notbefore/nbf JWT claim * Correct lint suggestions - comments for structs and functions - Add comments to LFS model - Function comment for GetRandomBytesAsBase64 - LFS server function comments and lint variable suggestion * Move secret generation code out of conditional Ensures no LFS code may run with an empty secret * Do not hand out JWT tokens if LFS server support is disabled
2016-12-26 02:16:37 +01:00
Header: make(map[string]string),
Href: url,
}
tokenAuthentication.Header["Authorization"] = fmt.Sprintf("Bearer %s", tokenString)
enc := json.NewEncoder(os.Stdout)
err = enc.Encode(tokenAuthentication)
if err != nil {
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
return fail(ctx, "Failed to encode LFS json response", "Failed to encode LFS json response: %v", err)
Git LFS support v2 (#122) * Import github.com/git-lfs/lfs-test-server as lfs module base Imported commit is 3968aac269a77b73924649b9412ae03f7ccd3198 Removed: Dockerfile CONTRIBUTING.md mgmt* script/ vendor/ kvlogger.go .dockerignore .gitignore README.md * Remove config, add JWT support from github.com/mgit-at/lfs-test-server Imported commit f0cdcc5a01599c5a955dc1bbf683bb4acecdba83 * Add LFS settings * Add LFS meta object model * Add LFS routes and initialization * Import github.com/dgrijalva/jwt-go into vendor/ * Adapt LFS module: handlers, routing, meta store * Move LFS routes to /user/repo/info/lfs/* * Add request header checks to LFS BatchHandler / PostHandler * Implement LFS basic authentication * Rework JWT secret generation / load * Implement LFS SSH token authentication with JWT Specification: https://github.com/github/git-lfs/tree/master/docs/api * Integrate LFS settings into install process * Remove LFS objects when repository is deleted Only removes objects from content store when deleted repo is the only referencing repository * Make LFS module stateless Fixes bug where LFS would not work after installation without restarting Gitea * Change 500 'Internal Server Error' to 400 'Bad Request' * Change sql query to xorm call * Remove unneeded type from LFS module * Change internal imports to code.gitea.io/gitea/ * Add Gitea authors copyright * Change basic auth realm to "gitea-lfs" * Add unique indexes to LFS model * Use xorm count function in LFS check on repository delete * Return io.ReadCloser from content store and close after usage * Add LFS info to runWeb() * Export LFS content store base path * LFS file download from UI * Work around git-lfs client issue with unauthenticated requests Returning a dummy Authorization header for unauthenticated requests lets git-lfs client skip asking for auth credentials See: https://github.com/github/git-lfs/issues/1088 * Fix unauthenticated UI downloads from public repositories * Authentication check order, Finish LFS file view logic * Ignore LFS hooks if installed for current OS user Fixes Gitea UI actions for repositories tracking LFS files. Checks for minimum needed git version by parsing the semantic version string. * Hide LFS metafile diff from commit view, marking as binary * Show LFS notice if file in commit view is tracked * Add notbefore/nbf JWT claim * Correct lint suggestions - comments for structs and functions - Add comments to LFS model - Function comment for GetRandomBytesAsBase64 - LFS server function comments and lint variable suggestion * Move secret generation code out of conditional Ensures no LFS code may run with an empty secret * Do not hand out JWT tokens if LFS server support is disabled
2016-12-26 02:16:37 +01:00
}
return nil
}
2014-10-01 13:40:48 +02:00
var gitcmd *exec.Cmd
gitBinPath := filepath.Dir(git.GitExecutable) // e.g. /usr/bin
gitBinVerb := filepath.Join(gitBinPath, verb) // e.g. /usr/bin/git-upload-pack
if _, err := os.Stat(gitBinVerb); err != nil {
// if the command "git-upload-pack" doesn't exist, try to split "git-upload-pack" to use the sub-command with git
// ps: Windows only has "git.exe" in the bin path, so Windows always uses this way
verbFields := strings.SplitN(verb, "-", 2)
if len(verbFields) == 2 {
// use git binary with the sub-command part: "C:\...\bin\git.exe", "upload-pack", ...
gitcmd = exec.CommandContext(ctx, git.GitExecutable, verbFields[1], repoPath)
}
}
if gitcmd == nil {
// by default, use the verb (it has been checked above by allowedCommands)
gitcmd = exec.CommandContext(ctx, gitBinVerb, repoPath)
2014-10-01 13:40:48 +02:00
}
process.SetSysProcAttribute(gitcmd)
2014-05-26 02:11:25 +02:00
gitcmd.Dir = setting.RepoRootPath
2014-04-10 20:20:58 +02:00
gitcmd.Stdout = os.Stdout
gitcmd.Stdin = os.Stdin
gitcmd.Stderr = os.Stderr
gitcmd.Env = append(gitcmd.Env, os.Environ()...)
gitcmd.Env = append(gitcmd.Env,
repo_module.EnvRepoIsWiki+"="+strconv.FormatBool(results.IsWiki),
repo_module.EnvRepoName+"="+results.RepoName,
repo_module.EnvRepoUsername+"="+results.OwnerName,
repo_module.EnvPusherName+"="+results.UserName,
repo_module.EnvPusherEmail+"="+results.UserEmail,
repo_module.EnvPusherID+"="+strconv.FormatInt(results.UserID, 10),
repo_module.EnvRepoID+"="+strconv.FormatInt(results.RepoID, 10),
repo_module.EnvPRID+"="+fmt.Sprintf("%d", 0),
repo_module.EnvDeployKeyID+"="+fmt.Sprintf("%d", results.DeployKeyID),
repo_module.EnvKeyID+"="+fmt.Sprintf("%d", results.KeyID),
repo_module.EnvAppURL+"="+setting.AppURL,
)
// to avoid breaking, here only use the minimal environment variables for the "gitea serv" command.
// it could be re-considered whether to use the same git.CommonGitCmdEnvs() as "git" command later.
gitcmd.Env = append(gitcmd.Env, git.CommonCmdServEnvs()...)
2014-07-26 06:24:27 +02:00
if err = gitcmd.Run(); err != nil {
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
return fail(ctx, "Failed to execute git command", "Failed to execute git command: %v", err)
2014-04-10 20:20:58 +02:00
}
2014-06-28 17:56:41 +02:00
2015-08-06 16:48:11 +02:00
// Update user key activity.
if results.KeyID > 0 {
if err = private.UpdatePublicKeyInRepo(ctx, results.KeyID, results.RepoID); err != nil {
Refactor internal API for git commands, use meaningful messages instead of "Internal Server Error" (#23687) # Why this PR comes At first, I'd like to help users like #23636 (there are a lot) The unclear "Internal Server Error" is quite anonying, scare users, frustrate contributors, nobody knows what happens. So, it's always good to provide meaningful messages to end users (of course, do not leak sensitive information). When I started working on the "response message to end users", I found that the related code has a lot of technical debt. A lot of copy&paste code, unclear fields and usages. So I think it's good to make everything clear. # Tech Backgrounds Gitea has many sub-commands, some are used by admins, some are used by SSH servers or Git Hooks. Many sub-commands use "internal API" to communicate with Gitea web server. Before, Gitea server always use `StatusCode + Json "err" field` to return messages. * The CLI sub-commands: they expect to show all error related messages to site admin * The Serv/Hook sub-commands (for git clients): they could only show safe messages to end users, the error log could only be recorded by "SSHLog" to Gitea web server. In the old design, it assumes that: * If the StatusCode is 500 (in some functions), then the "err" field is error log, shouldn't be exposed to git client. * If the StatusCode is 40x, then the "err" field could be exposed. And some functions always read the "err" no matter what the StatusCode is. The old code is not strict, and it's difficult to distinguish the messages clearly and then output them correctly. # This PR To help to remove duplicate code and make everything clear, this PR introduces `ResponseExtra` and `requestJSONResp`. * `ResponseExtra` is a struct which contains "extra" information of a internal API response, including StatusCode, UserMsg, Error * `requestJSONResp` is a generic function which can be used for all cases to help to simplify the calls. * Remove all `map["err"]`, always use `private.Response{Err}` to construct error messages. * User messages and error messages are separated clearly, the `fail` and `handleCliResponseExtra` will output correct messages. * Replace all `Internal Server Error` messages with meaningful (still safe) messages. This PR saves more than 300 lines, while makes the git client messages more clear. Many gitea-serv/git-hook related essential functions are covered by tests. --------- Co-authored-by: delvh <dev.lh@web.de>
2023-03-29 08:32:26 +02:00
return fail(ctx, "Failed to update public key", "UpdatePublicKeyInRepo: %v", err)
2015-08-05 05:14:17 +02:00
}
2014-08-10 00:40:10 +02:00
}
return nil
2014-04-10 20:20:58 +02:00
}