Girocco/Util.pm: use integer value of $! in noFatalsToBrowser
The default value of "$!" is a string which is not at all
helpful when trying to pass that to the "exit" function.
Force it into a numeric context by using (0+$!) to get the
proper integer to pass to the exit function.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
The default value of "$!" is a string which is not at all
helpful when trying to pass that to the "exit" function.
Force it into a numeric context by using (0+$!) to get the
proper integer to pass to the exit function.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
projtool.pl: add prune command
Provide a prune command that can remove any project names from
the group file that do not correspond to an existing project
directory (aka GIT_DIR).
Require explicit --dry-run or --force to operate.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Provide a prune command that can remove any project names from
the group file that do not correspond to an existing project
directory (aka GIT_DIR).
Require explicit --dry-run or --force to operate.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
config: remove xmllint_readme option
The `xmllint` tool no longer constrains checking of the README
data. It's always checked now courtesy of the Markdown module.
Remove the $Girocco::Config::xmllint_readme option from the
config and remove the check for the `xmllint` tool during
installation if that option has been set to a true value.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
The `xmllint` tool no longer constrains checking of the README
data. It's always checked now courtesy of the Markdown module.
Remove the $Girocco::Config::xmllint_readme option from the
config and remove the check for the `xmllint` tool during
installation if that option has been set to a true value.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
projtool.pl: support new readme formats
Make the behind-the-scenes adoption process grok the new
readme formats and process the readme if necessary during
the adoption process.
Add a new optional `--format=<readmetype>` option to the
`setreadme` command to select the desired type and make it
possible to change the type without changing the readme
data itself (reinterprets the old data as the new type).
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Make the behind-the-scenes adoption process grok the new
readme formats and process the readme if necessary during
the adoption process.
Add a new optional `--format=<readmetype>` option to the
`setreadme` command to select the desired type and make it
possible to change the type without changing the readme
data itself (reinterprets the old data as the new type).
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/Project.pm: add _normalize_rmtype function
Add a few extra aliases for possible incoming values of
the 'girocco.rmtype' config value. Notably various possible
file name extensions as well as content-type names.
Provide a semi-private _normalize_rmtype function that
returns one of the three "normalized" rmtype values for
its single argument or optionally "" if unrecognized type.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Add a few extra aliases for possible incoming values of
the 'girocco.rmtype' config value. Notably various possible
file name extensions as well as content-type names.
Provide a semi-private _normalize_rmtype function that
returns one of the three "normalized" rmtype values for
its single argument or optionally "" if unrecognized type.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
check-perl-modules.pl: do not require any XML modules
Previously either XML::Simple and/or XML::Parser were required
with the intent that would allow the explicit README contents
to always be validated.
As the need for that has been overcome by events (the Markdown
module now does its own internal validation without the need for
either of those modules), remove the checks for the XML Perl
modules since they're no longer of any interest.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Previously either XML::Simple and/or XML::Parser were required
with the intent that would allow the explicit README contents
to always be validated.
As the need for that has been overcome by events (the Markdown
module now does its own internal validation without the need for
either of those modules), remove the checks for the XML Perl
modules since they're no longer of any interest.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
readme: allow non-HTML formats
Provide a "README Format" selection to choose between 'Markdown',
'Plain Text' and 'HTML' for an explicit README on the project page.
As before, leaving the actual readme content blank results in the
"Automatic README" mode.
'Plain Text' simply XML-escapes the content and stuffs it into
a "<pre>...</pre>" section.
'Markdown' passes it through the same Markdown processor as for
an "Automatic README" in Markdown format. This is now the default
format.
'HTML' represents the only previously allowed format and will
automatically be selected for non-empty legacy explict HTML README
contents. However, HTML is now always validated courtesy of the
Markdown module's ProcessRaw function. It's also now sanitized (no
more scripts, style tags [style attributes are still okay], javascript
attributes, etc.). Content that is "almost-XML HTML" will be
automatically "fixed up" rather than erroring out like it used to.
The raw README content data is now stored in the file 'README.dat'
alongside the processed result that's always stored in 'README.html'
(thereby maintaining full gitweb compatibility).
Just a few minor housekeeping updates to go along with this change.
Make projtool show a length for the new READMEDATA value (similarly
to how it handles the README value). Add the new 'README.dat' file
to the list of files included in a project's BOM.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Provide a "README Format" selection to choose between 'Markdown',
'Plain Text' and 'HTML' for an explicit README on the project page.
As before, leaving the actual readme content blank results in the
"Automatic README" mode.
'Plain Text' simply XML-escapes the content and stuffs it into
a "<pre>...</pre>" section.
'Markdown' passes it through the same Markdown processor as for
an "Automatic README" in Markdown format. This is now the default
format.
'HTML' represents the only previously allowed format and will
automatically be selected for non-empty legacy explict HTML README
contents. However, HTML is now always validated courtesy of the
Markdown module's ProcessRaw function. It's also now sanitized (no
more scripts, style tags [style attributes are still okay], javascript
attributes, etc.). Content that is "almost-XML HTML" will be
automatically "fixed up" rather than erroring out like it used to.
The raw README content data is now stored in the file 'README.dat'
alongside the processed result that's always stored in 'README.html'
(thereby maintaining full gitweb compatibility).
Just a few minor housekeeping updates to go along with this change.
Make projtool show a length for the new READMEDATA value (similarly
to how it handles the README value). Add the new 'README.dat' file
to the list of files included in a project's BOM.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
check-perl-modules.pl: use explicit error prefix
The "note" and "warning" messages start with an explicit
prefix. Make the "error" messages do the same.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
The "note" and "warning" messages start with an explicit
prefix. Make the "error" messages do the same.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
install.sh: eliminate spurious test_nc_U "Terminated" messages
During the test to see if the configured "nc" command actually
supports the `-U` option, the background process that was doing
the listening gets explicitly killed.
If it has already exited, no problem, the message is suppressed
about an invalid PID and the error ignored.
If it hasn't, it's killed, but then, at some point, the shell
noticies that and wants to write out a "Terminated" notification
about the background process.
Unless, of course, the shell happens to be running with stderr
output redirected to /dev/null at the point it notices.
The test_nc_U function always runs with stderr output redirected
to /dev/null, but sometimes the terminated process does not
get noticed until later.
To try and address this, move the first "sleep 1" to immediately
after the backgrounding of the listen process to make sure it's
really been "exec'd" after forking.
Then add another "sleep 1" immediately after the "kill" of the
background process as apparently the shell only "notices" that
background processes have died just after it's executed an
external process.
The extra second will not matter to the total time taken for
the install process but hopefully will finally squelch the rare
spurious "Terminated" message.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
During the test to see if the configured "nc" command actually
supports the `-U` option, the background process that was doing
the listening gets explicitly killed.
If it has already exited, no problem, the message is suppressed
about an invalid PID and the error ignored.
If it hasn't, it's killed, but then, at some point, the shell
noticies that and wants to write out a "Terminated" notification
about the background process.
Unless, of course, the shell happens to be running with stderr
output redirected to /dev/null at the point it notices.
The test_nc_U function always runs with stderr output redirected
to /dev/null, but sometimes the terminated process does not
get noticed until later.
To try and address this, move the first "sleep 1" to immediately
after the backgrounding of the listen process to make sure it's
really been "exec'd" after forking.
Then add another "sleep 1" immediately after the "kill" of the
background process as apparently the shell only "notices" that
background processes have died just after it's executed an
external process.
The extra second will not matter to the total time taken for
the install process but hopefully will finally squelch the rare
spurious "Terminated" message.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
install.sh: add installation preconditions check
As the last step before actually starting to do the install,
add a preconditions check phase.
Currently the only check is that the Markdown module can
be used and that it produces the expected output on a very
tiny test case.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
As the last step before actually starting to do the install,
add a preconditions check phase.
Currently the only check is that the Markdown module can
be used and that it produces the expected output on a very
tiny test case.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Markdown.pm: make Markdown module available to use
In order for the "Markdown" module to be available to "use", this
must succeed:
use Markdown;
But it must succeed in two different environments:
1) the pre-install environment
2) the post-install environment
In the first case, that means a symbolic link into the checked out
markdown.git project. In the second case, that means installing a
symbolic link into $Girocco::Config::basedir that points to the
installed copy of Markdown.pl.
Since "use" statements are processed even when just using the `perl
-c` command to check syntax, failing to address the first case would
make it very difficult to check syntax prior to installation or use
any module during installation that referenced the Markdown module
even if the Markdown functions were not actually being called.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
In order for the "Markdown" module to be available to "use", this
must succeed:
use Markdown;
But it must succeed in two different environments:
1) the pre-install environment
2) the post-install environment
In the first case, that means a symbolic link into the checked out
markdown.git project. In the second case, that means installing a
symbolic link into $Girocco::Config::basedir that points to the
installed copy of Markdown.pl.
Since "use" statements are processed even when just using the `perl
-c` command to check syntax, failing to address the first case would
make it very difficult to check syntax prior to installation or use
any module during installation that referenced the Markdown module
even if the Markdown functions were not actually being called.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
markdown.git: update to newer version
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
HashUtil.pm: reverse order of hmac_sha1 arguments
Change the order of the arguments passed to HashUtil::hmac_sha1
to be "text" then "key" instead of "key" then "text".
Originally the hmac_sha1 provided by Girocco::HashUtil used the
sane order of "key" then "text" as that's the order they are
mentioned in RFC 2104. The @#%^^@* Digest::SHA module, on the
other hand (which provides both hmac_sha1 and hmac_sha256), for
some inscrutable reason uses the order of "text" then "key"!
In an effort to avoid needless confusion, make the HashUtil
implementation of hmac_sha1 take the arguments in exactly the
same order as the Digest::SHA module's function of the same
name.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Change the order of the arguments passed to HashUtil::hmac_sha1
to be "text" then "key" instead of "key" then "text".
Originally the hmac_sha1 provided by Girocco::HashUtil used the
sane order of "key" then "text" as that's the order they are
mentioned in RFC 2104. The @#%^^@* Digest::SHA module, on the
other hand (which provides both hmac_sha1 and hmac_sha256), for
some inscrutable reason uses the order of "text" then "key"!
In an effort to avoid needless confusion, make the HashUtil
implementation of hmac_sha1 take the arguments in exactly the
same order as the Digest::SHA module's function of the same
name.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/Validator.pm: sanctify non-default Config.pm
When a non-default Config.pm is used via the GIROCCO_CONF mechanism,
the global $INC{'Girocco/Config.pm'} element never gets set.
If the install process then attempts to use some utility that
directly "use"s Girocco::Config the default config will be included
at that point and overwrite the custom settings pointed at by
GIROCCO_CONF.
Since the only time it's okay to not explicitly use Girocco::Validator
after $GIROCCO_CONF is once the config has been frozen (it's always
frozen under the Girocco::Config name), make Validator.pm sanctify
the use of a non-default Config.pm by magically setting the missing
$INC{'Girocco/Config.pm'} when it does its thing. It does, after
all, actually contain a `package Girocco::Config` line; it's not
unreasonable to think it might set $INC{'Girocco/Config.pm'}.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
When a non-default Config.pm is used via the GIROCCO_CONF mechanism,
the global $INC{'Girocco/Config.pm'} element never gets set.
If the install process then attempts to use some utility that
directly "use"s Girocco::Config the default config will be included
at that point and overwrite the custom settings pointed at by
GIROCCO_CONF.
Since the only time it's okay to not explicitly use Girocco::Validator
after $GIROCCO_CONF is once the config has been frozen (it's always
frozen under the Girocco::Config name), make Validator.pm sanctify
the use of a non-default Config.pm by magically setting the missing
$INC{'Girocco/Config.pm'} when it does its thing. It does, after
all, actually contain a `package Girocco::Config` line; it's not
unreasonable to think it might set $INC{'Girocco/Config.pm'}.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
editproj.cgi: if password validates always enable protected fields
Regardless of whether or not there are any other errors in the
submitted data, if the password was valid, the protected fields
are always enabled.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Regardless of whether or not there are any other errors in the
submitted data, if the password was valid, the protected fields
are always enabled.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/Project.pm: validate new JSON POST url with test POST
If the JSON POST url changes from its previous value (a newly created
project is considered to have "" as its previous value), then before
saving the new JSON POST url into the project, attempt a test POST
to see if it's actually usable.
Refuse to accept the new JSON POST url if it fails the POST test.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
If the JSON POST url changes from its previous value (a newly created
project is considered to have "" as its previous value), then before
saving the new JSON POST url into the project, attempt a test POST
to see if it's actually usable.
Refuse to accept the new JSON POST url if it fails the POST test.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/Notify.pm: add json_test_post function
Rather than blindly accept just any old JSON POST url, the
json_test_post function can be called at the time the url
is first configured to verify that it can be POST'd to
successfully.
This is not unlike the behavior of that other well-known
service that we are emulating except that the intent is
to actually decline to configure the URL if anything other
than a "successful" (aka 2xx) status code comes back.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Rather than blindly accept just any old JSON POST url, the
json_test_post function can be called at the time the url
is first configured to verify that it can be POST'd to
successfully.
This is not unlike the behavior of that other well-known
service that we are emulating except that the intent is
to actually decline to configure the URL if anything other
than a "successful" (aka 2xx) status code comes back.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/Notify.pm: add a few fields to JSON repository hash
We do have some additional information that does correspond to
information being provided by the originator of the specification
we're attempting to emulate.
Add the few extra fields that we have in case they prove useful to
JSON POST clients.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
We do have some additional information that does correspond to
information being provided by the originator of the specification
we're attempting to emulate.
Add the few extra fields that we have in case they prove useful to
JSON POST clients.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/Notify.pm: refactor JSON code for easier reuse
Separate out the parts likely to be reused in the future
into their own little private functions.
No externally visible changes.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Separate out the parts likely to be reused in the future
into their own little private functions.
No externally visible changes.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/Project.pm: make json secret available via web gui
Now that protected fields are available, go ahead and add
the jsonsecret field to the list that's viewable/editable
whenever 'notifyjson' has been enabled. It will only be
viewable/editable once the admin password has been entered.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Now that protected fields are available, go ahead and add
the jsonsecret field to the list that's viewable/editable
whenever 'notifyjson' has been enabled. It will only be
viewable/editable once the admin password has been entered.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
editproj.cgi: suppress "operation aborted" for "view all"
On a "view all" operation there's nothing to abort -- either
the password matched or it didn't, but nothing else happens.
Suppress the misleading and alarming "operation aborted"
message that would otherwise occur on a "View All" if there
were any issues detected. The issues will still show, just
not the ugly "operation aborted" -- but it's still there
for failure of an "Update" operation.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
On a "view all" operation there's nothing to abort -- either
the password matched or it didn't, but nothing else happens.
Suppress the misleading and alarming "operation aborted"
message that would otherwise occur on a "View All" if there
were any issues detected. The issues will still show, just
not the ugly "operation aborted" -- but it's still there
for failure of an "Update" operation.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/Project.pm: add optional silent flag to cgi_fill
In order to add more messages before calling err_check, it's
important that cgi_fill not prematurely dump out the messages.
If the optional second parameter is true, the messages will
be suppressed, but the return value will still be the same.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
In order to add more messages before calling err_check, it's
important that cgi_fill not prematurely dump out the messages.
If the optional second parameter is true, the messages will
be suppressed, but the return value will still be the same.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
regproj.cgi: implement project registration time limit
If project registration has not been completed within a set time
limit, specifically $Girocco::Config::project_edit_timeout, display
an error rather than registering the project.
The behavior of an expired registration timeout significantly
resembles a failure of the "Anti-captcha" check, just with a slightly
different message.
With this change, a project registration page that has been sitting
unsubmitted for some time can no longer just be submitted without
an extra step that will require re-entry of the "Anti-captcha" as
well as the project password (twice).
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
If project registration has not been completed within a set time
limit, specifically $Girocco::Config::project_edit_timeout, display
an error rather than registering the project.
The behavior of an expired registration timeout significantly
resembles a failure of the "Anti-captcha" check, just with a slightly
different message.
With this change, a project registration page that has been sitting
unsubmitted for some time can no longer just be submitted without
an extra step that will require re-entry of the "Anti-captcha" as
well as the project password (twice).
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
editproj.cgi: implement protected edit fields
Some fields on the editproj page may contain sensitive information.
Up until now, all project fields have always been displayed when
the edit project link is selected -- even those fields that are not
otherwise visible anywhere else.
The notification email addresses and JSON POST urls may be considered
sensitive information that should not just be blindly available to
anyone who clicks the "edit" link on a project page.
Solve this problem by making fields that contain sensitive information,
specifically the ones in the $Girocco::Config::protect_fields hash,
not visible or editable until the project's admin password has been
entered.
At the same time, enforce an editing time limit, specifically the
$Girocco::Config::project_edit_timeout value, when editing so that
very stale pages cannot just be submitted even if they contain the
correct password.
Currently the list of users allowed to push is *NOT* considered a
sensitive field. That may need to be revisited at some point.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Some fields on the editproj page may contain sensitive information.
Up until now, all project fields have always been displayed when
the edit project link is selected -- even those fields that are not
otherwise visible anywhere else.
The notification email addresses and JSON POST urls may be considered
sensitive information that should not just be blindly available to
anyone who clicks the "edit" link on a project page.
Solve this problem by making fields that contain sensitive information,
specifically the ones in the $Girocco::Config::protect_fields hash,
not visible or editable until the project's admin password has been
entered.
At the same time, enforce an editing time limit, specifically the
$Girocco::Config::project_edit_timeout value, when editing so that
very stale pages cannot just be submitted even if they contain the
correct password.
Currently the list of users allowed to push is *NOT* considered a
sensitive field. That may need to be revisited at some point.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
various: provide protected field support bits
Enhance print_form_fields to allow passing in a list of
protected fields that need to be handled differently.
Enhance cgi_fill to not fill fields from a non-existent
parameter value (thereby avoiding stepping on the value).
Provide a new check_timed_token utility function that
works like verify_timed_token except it takes a category
name instead of an actual secret.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Enhance print_form_fields to allow passing in a list of
protected fields that need to be handled differently.
Enhance cgi_fill to not fill fields from a non-existent
parameter value (thereby avoiding stepping on the value).
Provide a new check_timed_token utility function that
works like verify_timed_token except it takes a category
name instead of an actual secret.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
projtool/usertool: prohibit pager for interactive commands
Certain commands must never start a pager regardless of any
options that may have been given or the "terminal"ness of
standard output.
Interactive input simply does not work properly when standard
output is a pipe to a pager.
Inhibit use of a pager whenever interactive commands are used.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Certain commands must never start a pager regardless of any
options that may have been given or the "terminal"ness of
standard output.
Interactive input simply does not work properly when standard
output is a pipe to a pager.
Inhibit use of a pager whenever interactive commands are used.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
project: apply minor field alignment adjustments
Apply the "formdatatd" class to a couple of fields on the
project form (regproj.cgi/editproj.cgi) pages.
Just a minor tweak, but it does improve the alignment between
the label and the entry field a bit.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Apply the "formdatatd" class to a couple of fields on the
project form (regproj.cgi/editproj.cgi) pages.
Just a minor tweak, but it does improve the alignment between
the label and the entry field a bit.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
cgi: include timed token in project reg/edit forms
Include a timed token in both the project registration and project
edit forms that has a validity from the time the form is created
and lasting for $Girocco::Config::project_edit_timeout seconds.
As of yet, nothing interesting happens when a form is submitted
but the timed token is no longer valid.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Include a timed token in both the project registration and project
edit forms that has a validity from the time the form is created
and lasting for $Girocco::Config::project_edit_timeout seconds.
As of yet, nothing interesting happens when a form is submitted
but the timed token is no longer valid.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
various: prepare for use of timed tokens
Add a new routine to easily fetch the key for a token category.
Add new config items to indicate which project fields require
a validated password even to just view and how long it's valid.
Create new token keys at install time if they do not exist.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Add a new routine to easily fetch the key for a token category.
Add new config items to indicate which project fields require
a validated password even to just view and how long it's valid.
Create new token keys at install time if they do not exist.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
JSON: include another signature header
If a "secret" has been set, compute and include an additional
signature header with the JSON POST push notification.
The signature is not complicated, it's simply the lowercase
hexadecimal representation of the HMAC as described in RFC 2104
<https://tools.ietf.org/html/rfc2104> where the "secret" provides
the "secret key K" and the body of the POST provides the "data
'text'" for the computation.
With this addition the original SHA1-based HMAC signature continues
to be included but now too an additional SHA256-based HMAC signature.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
If a "secret" has been set, compute and include an additional
signature header with the JSON POST push notification.
The signature is not complicated, it's simply the lowercase
hexadecimal representation of the HMAC as described in RFC 2104
<https://tools.ietf.org/html/rfc2104> where the "secret" provides
the "secret key K" and the body of the POST provides the "data
'text'" for the computation.
With this addition the original SHA1-based HMAC signature continues
to be included but now too an additional SHA256-based HMAC signature.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/TimedToken.pm: add support for timed tokens
A timed token is a cryptographically protected string that can be
included within a page sent to a client (i.e. hidden form field,
cookie, segment of url, etc.).
When the server receives the token, it can verify whether or not
it has expired in a cryptographically secure way.
The tokens are created using HMAC and a "secret" key. As long as
the "secret" key is not available to remote clients, forging a token
would require finding an HMAC collision (much harder than just a
plain hash collision).
Of course, the same information (validity time range) could simply
be included on a page in plain text and then signed with an HMAC
also included on that page, both of which are sent to the server
for verification.
However, these tokens are self-contained and inscrutable without
the "secret" key and a bit more convenient to use since no extra
information need be transmitted, just the token itself. And since
the tokens are url safe (they need neither URL escaping nor HTML
escaping because they use the base64url alphabet excluding '='),
they can easily be embedded most anywhere.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
A timed token is a cryptographically protected string that can be
included within a page sent to a client (i.e. hidden form field,
cookie, segment of url, etc.).
When the server receives the token, it can verify whether or not
it has expired in a cryptographically secure way.
The tokens are created using HMAC and a "secret" key. As long as
the "secret" key is not available to remote clients, forging a token
would require finding an HMAC collision (much harder than just a
plain hash collision).
Of course, the same information (validity time range) could simply
be included on a page in plain text and then signed with an HMAC
also included on that page, both of which are sent to the server
for verification.
However, these tokens are self-contained and inscrutable without
the "secret" key and a bit more convenient to use since no extra
information need be transmitted, just the token itself. And since
the tokens are url safe (they need neither URL escaping nor HTML
escaping because they use the base64url alphabet excluding '='),
they can easily be embedded most anywhere.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
projtool/usertool: automatically page output on terminal
Automatically page the output when it's to a terminal unless
the `--no-pager` option has been given.
Expand the help to better detail the global options.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Automatically page the output when it's to a terminal unless
the `--no-pager` option has been given.
Expand the help to better detail the global options.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/CLIUtil.pm: add setup_pager and setup_pager_stdout
The setup_pager function returns a pipe to a pager as determined
by the PAGER environment variable or what can be found in $PATH.
The setup_pager_stdout redirects STDOUT to a pager depending
on the arguments passed in, by default (with no arguments), it
redirects STDOUT to a pager only if STDOUT is a terminal.
But, if a known pager is already in use (tested courtesy of the new
pager_in_use function), then setup_pager_stdout does nothing.
If setup_pager_stdout activates a pager, also set the appropriate
known environment variables to discourage nested pagers.
The setup_pager_stdout function does some "magic" to make things
just work from the point of view of the caller, but those things
are quite possibly unworkable on a non-unixen system. But then
so too will likely be running Girocco on such a system.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
The setup_pager function returns a pipe to a pager as determined
by the PAGER environment variable or what can be found in $PATH.
The setup_pager_stdout redirects STDOUT to a pager depending
on the arguments passed in, by default (with no arguments), it
redirects STDOUT to a pager only if STDOUT is a terminal.
But, if a known pager is already in use (tested courtesy of the new
pager_in_use function), then setup_pager_stdout does nothing.
If setup_pager_stdout activates a pager, also set the appropriate
known environment variables to discourage nested pagers.
The setup_pager_stdout function does some "magic" to make things
just work from the point of view of the caller, but those things
are quite possibly unworkable on a non-unixen system. But then
so too will likely be running Girocco on such a system.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
toolbox/projtool.pl: do not hide global $quiet
A very ancient version of this utility did not have a global
$quiet variable.
Purge a remnant of that old code and simply rely on the
global $quiet variable everywhere.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
A very ancient version of this utility did not have a global
$quiet variable.
Purge a remnant of that old code and simply rely on the
global $quiet variable everywhere.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
projtool.pl: provide command line jsonsecret manipulation
Add the ability to get/set the new jsonsecret value of
a project from the command line via projtool.pl.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Add the ability to get/set the new jsonsecret value of
a project from the command line via projtool.pl.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
JSON: include a signature header
If a "secret" has been set, compute and include a signature
header with the JSON POST push notification.
The signature is not complicated, it's simply the lowercase
hexadecimal representation of the HMAC as described in
RFC 2104 <https://tools.ietf.org/html/rfc2104> where the
"secret" provides the "secret key K" and the body of the
POST provides the "data 'text'" for the computation.
Currently the "secret" must be manually configured by directly
modifying the project's config file, but that will change in
the future.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
If a "secret" has been set, compute and include a signature
header with the JSON POST push notification.
The signature is not complicated, it's simply the lowercase
hexadecimal representation of the HMAC as described in
RFC 2104 <https://tools.ietf.org/html/rfc2104> where the
"secret" provides the "secret key K" and the body of the
POST provides the "data 'text'" for the computation.
Currently the "secret" must be manually configured by directly
modifying the project's config file, but that will change in
the future.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/Notify.pm: use custom user-agent for JSON POST notifications
In order to facilitate tracking the source of potentially unwanted
JSON push POST notifications, provide a custom user agent string.
Since user agent strings are much more likely to be logged than
random "X-<whatever>" headers, they provide a much more convenient
way to identify the source of the incoming POST requests.
RFC 7231 section 5.5.3 <https://tools.ietf.org/html/rfc7231#page-46>
provides the expected format of a User-Agent value.
Conform to the RFC 7231 standard while providing the extra
helpful information in the "comment"s that are permitted within
the User-Agent string.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
In order to facilitate tracking the source of potentially unwanted
JSON push POST notifications, provide a custom user agent string.
Since user agent strings are much more likely to be logged than
random "X-<whatever>" headers, they provide a much more convenient
way to identify the source of the incoming POST requests.
RFC 7231 section 5.5.3 <https://tools.ietf.org/html/rfc7231#page-46>
provides the expected format of a User-Agent value.
Conform to the RFC 7231 standard while providing the extra
helpful information in the "comment"s that are permitted within
the User-Agent string.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/Notify.pm: perform our own JSON body encoding
Instead of relying on the LWP::UserAgent to magically just
"do the right thing", go ahead and explicitly encode the
body of the JSON POST notification and pass that on to be
posted.
This was already being done for the 'application/json' type
and now we also do it for the 'application/x-www-form-urlencoded'
type as well.
This provides total byte-exact control over what actually
goes out over the wire for a JSON POST notification.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Instead of relying on the LWP::UserAgent to magically just
"do the right thing", go ahead and explicitly encode the
body of the JSON POST notification and pass that on to be
posted.
This was already being done for the 'application/json' type
and now we also do it for the 'application/x-www-form-urlencoded'
type as well.
This provides total byte-exact control over what actually
goes out over the wire for a JSON POST notification.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/CGI.pm: allow field sets in metadata_fields
The $Girocco::Project::metadata_fields variable is a HASH ref
containing the necessary information for Girocco::CGI::print_form_fields
to emit the fields as XHTML markup.
Each key of metadata_fields can be enabled or disabled by including
it or not in the @Girocco::Config::project_fields array.
However, sometimes more than one field must be shown to properly edit
a logical field. For example, the JSON notification mechanism
requires both a URL and a Content-Type. It does not make sense to
show one field without the other.
Change the semantics of metadata_fields so that each entry can be
an ARRAY ref of fields rather than just an ARRAY ref of a single field.
Detect this automatically to minimize changes to the interpretation of
the metadata_fields HASH ref.
Update the print_form_fields function to support these new semantics
thereby removing the ugly kludge for the 'jsontype' field and group
the 'notifyjson' and 'jsontype' fields together as a set under the
'notifyjson' entry in metadata_fields.
Also take this opportunity to enhance the callback used to obtain
the list of possible values by passing it the Girocco::Project
object as its single argument if that's available (it is for
editproj.cgi but it's not for regproj.cgi).
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
The $Girocco::Project::metadata_fields variable is a HASH ref
containing the necessary information for Girocco::CGI::print_form_fields
to emit the fields as XHTML markup.
Each key of metadata_fields can be enabled or disabled by including
it or not in the @Girocco::Config::project_fields array.
However, sometimes more than one field must be shown to properly edit
a logical field. For example, the JSON notification mechanism
requires both a URL and a Content-Type. It does not make sense to
show one field without the other.
Change the semantics of metadata_fields so that each entry can be
an ARRAY ref of fields rather than just an ARRAY ref of a single field.
Detect this automatically to minimize changes to the interpretation of
the metadata_fields HASH ref.
Update the print_form_fields function to support these new semantics
thereby removing the ugly kludge for the 'jsontype' field and group
the 'notifyjson' and 'jsontype' fields together as a set under the
'notifyjson' entry in metadata_fields.
Also take this opportunity to enhance the callback used to obtain
the list of possible values by passing it the Girocco::Project
object as its single argument if that's available (it is for
editproj.cgi but it's not for regproj.cgi).
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
projtool/usertool: make friends with taint mode
All the arguments are validated before use, launder the
taint off of them to avoid problems in taint check mode.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
All the arguments are validated before use, launder the
taint off of them to avoid problems in taint check mode.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/Notify.pm: implement application/json POST type
If "application/json" has been chosen as the JSON POST Content-Type,
then POST a single body of that content-type consisting of the
raw JSON notification.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
If "application/json" has been chosen as the JSON POST Content-Type,
then POST a single body of that content-type consisting of the
raw JSON notification.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
projtool.pl: provide command line jsontype manipulation
Add the ability to get/set the new jsontype value of
a project from the command line via projtool.pl.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Add the ability to get/set the new jsontype value of
a project from the command line via projtool.pl.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
JSON: add a JSON Content-Type selection option
Allow a choice between the old mechanism of application/x-www-form-urlencoded
or the new mechanism of application/json that does not require any extra
URL decoding on receipt.
Always default to the old mechanism for backwards compatibility.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Allow a choice between the old mechanism of application/x-www-form-urlencoded
or the new mechanism of application/json that does not require any extra
URL decoding on receipt.
Always default to the old mechanism for backwards compatibility.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Notify: implement JSON notify forced flag
The computation was already being done (unless explicitly
disabled), it just wasn't available for use inside the JSON
notification code.
If the JSON notification code needs to run, compute the "forced"
value at that time and then later reuse it instead of recomputing.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
The computation was already being done (unless explicitly
disabled), it just wasn't available for use inside the JSON
notification code.
If the JSON notification code needs to run, compute the "forced"
value at that time and then later reuse it instead of recomputing.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
taskd.pl: refactor ref indicator computation
Move the ref_indicator computation function into Girocco::Util
but without the check for the `--no-show-fast-forward-info` option.
Provide a new local private _ref_indicator function for taskd
that makes use of the Girocco::Util::ref_indicator function and
takes into account the `--no-show-fast-forward-info` option in
order to continue to provide the same semantics as before.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Move the ref_indicator computation function into Girocco::Util
but without the check for the `--no-show-fast-forward-info` option.
Provide a new local private _ref_indicator function for taskd
that makes use of the Girocco::Util::ref_indicator function and
takes into account the `--no-show-fast-forward-info` option in
order to continue to provide the same semantics as before.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/Notify.pm: add missing JSON flags
Add created ref flag.
Add deleted ref flag.
Add dummy forced ref flag (always false for now).
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Add created ref flag.
Add deleted ref flag.
Add dummy forced ref flag (always false for now).
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/Notify.pm: correct commit payload
Include committer information.
Include tree information.
Use committer timestamp not author timestamp.
Use strict ISO 8601 format timestamp.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Include committer information.
Include tree information.
Use committer timestamp not author timestamp.
Use strict ISO 8601 format timestamp.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Notify: update JSON reference link and object contents
The links keep changing, the current link to the contents
of the push hook has, once again, changed, the new link is:
https://docs.github.com/developers/webhooks-and-events/webhook-events-and-payloads#push
There are also a few extra fields in there that ought to be
added. Add a couple of the easy ones right now.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
The links keep changing, the current link to the contents
of the push hook has, once again, changed, the new link is:
https://docs.github.com/developers/webhooks-and-events/webhook-events-and-payloads#push
There are also a few extra fields in there that ought to be
added. Add a couple of the easy ones right now.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/User.pm: validate more contents on load
Do some additional basic sanity checking when loading a
user. And, in the process, untaint users that pass
all the checks.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Do some additional basic sanity checking when loading a
user. And, in the process, untaint users that pass
all the checks.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
taskd.pl: launder validated ref-change arguments
Instead of throwing away the match that validates the
arguments, use it to launder the taint off them as well.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Instead of throwing away the match that validates the
arguments, use it to launder the taint off them as well.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
sendmail.pl: update to latest version
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/Project.pm: validate more contents on load
Do some additional basic sanity checking when loading a
project. And, in the process, untaint projects that pass
all the checks.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Do some additional basic sanity checking when loading a
project. And, in the process, untaint projects that pass
all the checks.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/Notify.pm: handle root and missing commits better
Add the disambiguating '--' option to guarantee that the commit
hashes are always interpreted as naming commits even if they
should be missing commits.
Correct a longstanding problem where branch creation failed to
produce the correct initial diff.
The root commit does not have a parent which makes <root>^ fail.
Instead, pass the "--root" option to diff-tree in place of the
parent that does not exist. The "--root" option has been around
almost as long as Git.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Add the disambiguating '--' option to guarantee that the commit
hashes are always interpreted as naming commits even if they
should be missing commits.
Correct a longstanding problem where branch creation failed to
produce the correct initial diff.
The root commit does not have a parent which makes <root>^ fail.
Instead, pass the "--root" option to diff-tree in place of the
parent that does not exist. The "--root" option has been around
almost as long as Git.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/ExecUtil.pm: yet more taint laundering
Make some reasonable checks regarding the program executable
and the value for argv[0] and if they pass, launder off the
taint contamination.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Make some reasonable checks regarding the program executable
and the value for argv[0] and if they pass, launder off the
taint contamination.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/Util.pm: launder taint off util_path result
Now that a frozen config has become available, it's much easier
to get to the var_git_exec_path value. Just use that if it's
set and non-empty. Otherwise, if necessary to go through the
whole rigmarole, verify the result is an absolute path and
launder the taint right off it.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Now that a frozen config has become available, it's much easier
to get to the var_git_exec_path value. Just use that if it's
set and non-empty. Otherwise, if necessary to go through the
whole rigmarole, verify the result is an absolute path and
launder the taint right off it.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
jailsetup.sh: validate paths used when preprocessing scripts
Add an initial check to validate that all the paths used to identify
scripts being preprocessed are valid paths. This also allows
them to be laundered of any taint at the same time.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Add an initial check to validate that all the paths used to identify
scripts being preprocessed are valid paths. This also allows
them to be laundered of any taint at the same time.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/Util.pm: add from_json function
With the recent addition of the to_json function, might as well
have a matching inverse from_json function.
Go ahead and add a from_json function that's the inverse of to_json.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
With the recent addition of the to_json function, might as well
have a matching inverse from_json function.
Go ahead and add a from_json function that's the inverse of to_json.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
check-perl-modules.pl: remove a few requirements
Unless CIA.vc mysteriously reappears, there's no point in
requiring RPC::XML or RPC::XML::Client.
Now that we're using our own "to_json" function, there's
also no need to require JSON either.
Remove these requirements and update the message about the
ref-change notification mechanism needs.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Unless CIA.vc mysteriously reappears, there's no point in
requiring RPC::XML or RPC::XML::Client.
Now that we're using our own "to_json" function, there's
also no need to require JSON either.
Remove these requirements and update the message about the
ref-change notification mechanism needs.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/Notify.pm: tidy up use'd modules
CIA has been defunct for a while now.
Since only CIA needed the RPC::XML modules, comment them out.
Also make sure the CIA.vc is defunct only shows up once per push
rather than once per commit.
Remove the enclosing BEGIN from around the `use` lines that
imply their own BEGIN scope -- no need for nested BEGIN scopes.
Switch to the recently added `to_json` function and get rid
of the `use JSON`. The only thing needed to support the POST
of the JSON blob is now LWP::UserAgent.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
CIA has been defunct for a while now.
Since only CIA needed the RPC::XML modules, comment them out.
Also make sure the CIA.vc is defunct only shows up once per push
rather than once per commit.
Remove the enclosing BEGIN from around the `use` lines that
imply their own BEGIN scope -- no need for nested BEGIN scopes.
Switch to the recently added `to_json` function and get rid
of the `use JSON`. The only thing needed to support the POST
of the JSON blob is now LWP::UserAgent.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/Util.pm: add to_json and json_bool functions
Just a simple little version of a JSON encoder that avoids
needing to pull in the whole JSON package.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Just a simple little version of a JSON encoder that avoids
needing to pull in the whole JSON package.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
install.sh: validate paths used when preprocessing scripts
Add an initial check to validate that all the paths used to identify
scripts being preprocessed are absolute paths. This also allows
them to be laundered of any taint at the same time.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Add an initial check to validate that all the paths used to identify
scripts being preprocessed are absolute paths. This also allows
them to be laundered of any taint at the same time.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
install.sh: add more argument checking to quick_move utility func
The quick_move utility function uses perl to access the underlying
OS's "rename" function and quickly move an old directory out while
moving a new directory in.
Add a tiny bit of additional checking in the perl code that makes
the arguments to the rename function call become untainted.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
The quick_move utility function uses perl to access the underlying
OS's "rename" function and quickly move an old directory out while
moving a new directory in.
Add a tiny bit of additional checking in the perl code that makes
the arguments to the rename function call become untainted.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
taskd/taskd.pl: untaint incoming pid
The incoming pid value was already being validated, it just
wasn't being untainted. Make a minor change to correct that
oversight and untaint the incoming pid number after it's been
validated.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
The incoming pid value was already being validated, it just
wasn't being untainted. Make a minor change to correct that
oversight and untaint the incoming pid number after it's been
validated.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/Dumper.pm: validate module names passed to RequireENV
While they are validated by the "require" attempt, that's inside
an eval. Make sure all the module names look sane before trying
the eval just in case. This has the nice side effect of untainting
the passed in module names.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
While they are validated by the "require" attempt, that's inside
an eval. Make sure all the module names look sane before trying
the eval just in case. This has the nice side effect of untainting
the passed in module names.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
update-all-config.pl: run okay in taint mode
Cleanse a couple of values that can become tainted but are
fully validated before use and should not be considered tainted.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Cleanse a couple of values that can become tainted but are
fully validated before use and should not be considered tainted.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
jobd/jobd.pl: avoid tainted load average numbers
The get_load_info function returns an array of the current values
of the three load averages (1 minute, 5 minutes, 15 minutes).
On a Linux system, it reads "/proc/loadavg". On other systems
it parses the output of the `uptime` command.
Surprisingly, the results were only tainted on Linux.
Rearrange the code slightly to take advantage of the same pattern
match that was untainting the `uptime` results and have that
take care of untainting the "/proc/loadavg" values too.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
The get_load_info function returns an array of the current values
of the three load averages (1 minute, 5 minutes, 15 minutes).
On a Linux system, it reads "/proc/loadavg". On other systems
it parses the output of the `uptime` command.
Surprisingly, the results were only tainted on Linux.
Rearrange the code slightly to take advantage of the same pattern
match that was untainting the `uptime` results and have that
take care of untainting the "/proc/loadavg" values too.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/Util.pm: make online_cpus return untainted value
Since online_cpus ends up executing the getconf program, that
result ends up being tainted even though it's only digits.
Purify the result to remove the taint to avoid further
contamination as the return value will likely be used in
many different places.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Since online_cpus ends up executing the getconf program, that
result ends up being tainted even though it's only digits.
Purify the result to remove the taint to avoid further
contamination as the return value will likely be used in
many different places.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
install.sh: perform Perl taint checks on configured PATH
Now that a value for PATH always gets set by the installed
Config.pm, check to make sure that value will pass Perl's
taint checks by actually running a small check using Perl
itself running in taint mode (-T) at install time.
Since it's not too terribly difficult to pass the checks
(all directories in the PATH must be absolute paths and
none of them can be writable by 'other'), it's not really
a burden, but it provides a little bit of extra security.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Now that a value for PATH always gets set by the installed
Config.pm, check to make sure that value will pass Perl's
taint checks by actually running a small check using Perl
itself running in taint mode (-T) at install time.
Since it's not too terribly difficult to pass the checks
(all directories in the PATH must be absolute paths and
none of them can be writable by 'other'), it's not really
a burden, but it provides a little bit of extra security.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
shlib.sh: exit immediately with failure on missing Config
When not using shlib_vars.sh to set the config shell variables such
as during the initial installation, if the $GIROCCO_CONF (which
defaults to Girocco::Config) module cannot be loaded to get the
configuration settings, exit immediately with an error.
While not normally an issue, if a custom configuration is being
used (now very easy by using a config.sh file) but the configuration
name has been set incorrectly, it's much saner to exit immediately
as soon as that's detected rather than accidentally continuing on.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
When not using shlib_vars.sh to set the config shell variables such
as during the initial installation, if the $GIROCCO_CONF (which
defaults to Girocco::Config) module cannot be loaded to get the
configuration settings, exit immediately with an error.
While not normally an issue, if a custom configuration is being
used (now very easy by using a config.sh file) but the configuration
name has been set incorrectly, it's much saner to exit immediately
as soon as that's detected rather than accidentally continuing on.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
config: always explicitly set PATH to something
In preparation for tightening up everything so that the
Perl scripts can run successfully under taint mode, always
explicitly configure a value for PATH.
Update the comments in Config.pm to reflect this (along with
more instructions showing how to keep the PATH in effect at
install time).
Update the FreezeConfig function to always provide a value
for PATH if one was not set and to always remove the other
environment variables that can cause taint failures (i.e.
IFS, CDPATH, ENV, BASH_ENV).
Update the get_girocco_config_var_list function in shlib.sh
to ensure it also has the new PATH semantics for cfg_path.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
In preparation for tightening up everything so that the
Perl scripts can run successfully under taint mode, always
explicitly configure a value for PATH.
Update the comments in Config.pm to reflect this (along with
more instructions showing how to keep the PATH in effect at
install time).
Update the FreezeConfig function to always provide a value
for PATH if one was not set and to always remove the other
environment variables that can cause taint failures (i.e.
IFS, CDPATH, ENV, BASH_ENV).
Update the get_girocco_config_var_list function in shlib.sh
to ensure it also has the new PATH semantics for cfg_path.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
chroot: set PATH when inside the chroot
The only possible value for PATH inside the chroot is "/bin" because
that's where all the binaries are installed.
Sure, /usr/bin/... works, but that's only because of the symlink
"/usr -> .". Anything else left over in the PATH from outside the
chroot is just not valid.
Although non-existent directories in PATH do get ignored, and at
least one of "/bin" or "/usr/bin" ought to be in there already,
don't depend on that and just always set PATH to /bin when running
inside the chroot.
The only scripts that can run inside the chroot are git-shell-verify
and the pre-receive, update and post-receive hooks. None of the
others can ever run inside the chroot -- at least not the chroot that
Girocco sets up for ssh pushing.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
The only possible value for PATH inside the chroot is "/bin" because
that's where all the binaries are installed.
Sure, /usr/bin/... works, but that's only because of the symlink
"/usr -> .". Anything else left over in the PATH from outside the
chroot is just not valid.
Although non-existent directories in PATH do get ignored, and at
least one of "/bin" or "/usr/bin" ought to be in there already,
don't depend on that and just always set PATH to /bin when running
inside the chroot.
The only scripts that can run inside the chroot are git-shell-verify
and the pre-receive, update and post-receive hooks. None of the
others can ever run inside the chroot -- at least not the chroot that
Girocco sets up for ssh pushing.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
lighttpd.conf.in: provide sample lighttpd configuration
Just about everything supported by the apache.conf.in file can be
supported by lighttpd and is by this configuration.
Except that it probably never will have .no_blob_plain support without
using mod_magnet although that's certainly possible at some point.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Just about everything supported by the apache.conf.in file can be
supported by lighttpd and is by this configuration.
Except that it probably never will have .no_blob_plain support without
using mod_magnet although that's certainly possible at some point.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
lighttpd: add lighttpd configuration file support
Add the basic bits to process lighttpd.conf.in into lighttpd.conf
and hook that up to the make process along with a suitable ignore.
For this commit, the lighttpd.conf.in file remains empty.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Add the basic bits to process lighttpd.conf.in into lighttpd.conf
and hook that up to the make process along with a suitable ignore.
For this commit, the lighttpd.conf.in file remains empty.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
make-apache-conf.sh: allow non-Apache conditionals to nest
Recently the apache.conf.in processor gained a new conditional
processing mode that's independent of Apache.
The syntax used being @@if(<test>)@@...@@endif@@ to trigger it.
While the new @@if(<test>)@@...@@endif@@ substitution does not
depend on any Apache-specific processing, it hasn't been possible
to nest multiple levels.
Enhance the syntax to allow the <test> part to be duplicated with
the @@endif@@ part like so:
@@if(some_test)@@
# lines here only active if some_test is true
@@endif(some_test)@@
The prior syntax continues to be supported like so:
@@if(some_test)@@
# lines here only active if some_test is true
@@endif@@
With this change, the matching closing @@endif will be the closest
one that either has no <test> part at all or has an exactly matching
test part.
Now it's possible to do this:
@@if(some_outer_test)@@
@@if(!some_inner_test)@@
# lines here only active if both some_outer_test and !some_inner_test
@@endif(!some_inner_test)@@
@@endif(some_outer_test)@@
The rule to make it work being that all but the outermost conditional
must include the exactly matching <test> in its @@endif. It's optional
for the outermost, but with nested conditionals it's helpful to include
it there anyway for documentation purposes.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Recently the apache.conf.in processor gained a new conditional
processing mode that's independent of Apache.
The syntax used being @@if(<test>)@@...@@endif@@ to trigger it.
While the new @@if(<test>)@@...@@endif@@ substitution does not
depend on any Apache-specific processing, it hasn't been possible
to nest multiple levels.
Enhance the syntax to allow the <test> part to be duplicated with
the @@endif@@ part like so:
@@if(some_test)@@
# lines here only active if some_test is true
@@endif(some_test)@@
The prior syntax continues to be supported like so:
@@if(some_test)@@
# lines here only active if some_test is true
@@endif@@
With this change, the matching closing @@endif will be the closest
one that either has no <test> part at all or has an exactly matching
test part.
Now it's possible to do this:
@@if(some_outer_test)@@
@@if(!some_inner_test)@@
# lines here only active if both some_outer_test and !some_inner_test
@@endif(!some_inner_test)@@
@@endif(some_outer_test)@@
The rule to make it work being that all but the outermost conditional
must include the exactly matching <test> in its @@endif. It's optional
for the outermost, but with nested conditionals it's helpful to include
it there anyway for documentation purposes.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
apache.conf.in: update warning text about TLS cert files
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
hooks/post-receive: check to see if we've lost our HEAD
After a receive, check to see if HEAD is valid and if not, make
sure the headok flag has been cleared.
Do not, however, immediately attempt to find a new HEAD.
When gc runs again, eventually, it will do so (and just in
time for bundle creation).
If the end user removes a ref and then shortly thereafter sets
it again there's no reason for unnecessary churn.
While it's true that if gc happens to run during that interval,
a new HEAD will be attached provided one is available, the
likelihood of hitting that window remains small if the intent
of the end user was to leave HEAD invalid for only a brief
period.
Waiting before trying to find a new HEAD helps to avoid that
case and yet always doing it eventually avoids the case where
the repository ends up with a permanently invalid HEAD which
suppresses bundle creation.
It's a compromise between unintentially changing HEAD prematurely
when the end user intends to remedy the problem shortly and
failing to create a bundle because there's no valid HEAD.
Since generate_auto_gc_update fills in for the behavior of
post-receive when manufacturing an artificial update, make
the same check there too.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
After a receive, check to see if HEAD is valid and if not, make
sure the headok flag has been cleared.
Do not, however, immediately attempt to find a new HEAD.
When gc runs again, eventually, it will do so (and just in
time for bundle creation).
If the end user removes a ref and then shortly thereafter sets
it again there's no reason for unnecessary churn.
While it's true that if gc happens to run during that interval,
a new HEAD will be attached provided one is available, the
likelihood of hitting that window remains small if the intent
of the end user was to leave HEAD invalid for only a brief
period.
Waiting before trying to find a new HEAD helps to avoid that
case and yet always doing it eventually avoids the case where
the repository ends up with a permanently invalid HEAD which
suppresses bundle creation.
It's a compromise between unintentially changing HEAD prematurely
when the end user intends to remedy the problem shortly and
failing to create a bundle because there's no valid HEAD.
Since generate_auto_gc_update fills in for the behavior of
post-receive when manufacturing an artificial update, make
the same check there too.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
jobd/update.sh: check to see if we've lost our HEAD
With everything else that's going on during a fetch, it's just
one more trivial check to make sure that HEAD remains valid.
If the remote removed the ref that we've been using as a HEAD
symbolic ref and we've not suppressed ref pruning (the default),
then we will have lost our HEAD after the fetch.
Check for this and try to find a reasonable HEAD to replace the
one we've lost if it happens.
The bundle creation machinery will refuse to create a bundle for
a repository with an inconsonant HEAD which makes HEAD reconciliation
highly desirable.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
With everything else that's going on during a fetch, it's just
one more trivial check to make sure that HEAD remains valid.
If the remote removed the ref that we've been using as a HEAD
symbolic ref and we've not suppressed ref pruning (the default),
then we will have lost our HEAD after the fetch.
Check for this and try to find a reasonable HEAD to replace the
one we've lost if it happens.
The bundle creation machinery will refuse to create a bundle for
a repository with an inconsonant HEAD which makes HEAD reconciliation
highly desirable.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
shlib.sh: try main in check_and_set_head
Check for refs/heads/main after refs/heads/master when attempting
to set a symbolic HEAD ref.
This alternate ref name reflects newer Git release behavior.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Check for refs/heads/main after refs/heads/master when attempting
to set a symbolic HEAD ref.
This alternate ref name reflects newer Git release behavior.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
config: move configuration and validation into separate module
Now that the actual configuration gets frozen during install into
an installed "frozen" configuration module, it's no longer necessary
to retain the defaults, validation and sanity checks inside the
Config module itself.
While it's true they could have been moved out earlier by simply
using a "do" or "require" at the end of the Config module, it's
better late than never.
Move all the configuration defaults, validation and sanity checks
into a new Validator.pm module that is not installed (i.e. it's
only used during the installation process).
Just in case, go ahead and have the Config module "require" it
at the end.
However, attempt to make sure that regardless of whether or not
the Config module "require"s it, it's always used during the install
process to make sure the configuration is valid and that it
participates during creation of the frozen Config.pm file to make
sure all the defaults are set properly.
Mostly this change just tidies up the source Config.pm by removing
from it anything that not really a configuration setting.
However, it does also attempt to ensure that the defaults and
validation checks needed for the configuration are not accidentally
bypassed when customizing the Config.pm file since they now happen
in another module entirely.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Now that the actual configuration gets frozen during install into
an installed "frozen" configuration module, it's no longer necessary
to retain the defaults, validation and sanity checks inside the
Config module itself.
While it's true they could have been moved out earlier by simply
using a "do" or "require" at the end of the Config module, it's
better late than never.
Move all the configuration defaults, validation and sanity checks
into a new Validator.pm module that is not installed (i.e. it's
only used during the installation process).
Just in case, go ahead and have the Config module "require" it
at the end.
However, attempt to make sure that regardless of whether or not
the Config module "require"s it, it's always used during the install
process to make sure the configuration is valid and that it
participates during creation of the frozen Config.pm file to make
sure all the defaults are set properly.
Mostly this change just tidies up the source Config.pm by removing
from it anything that not really a configuration setting.
However, it does also attempt to ensure that the defaults and
validation checks needed for the configuration are not accidentally
bypassed when customizing the Config.pm file since they now happen
in another module entirely.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/Dumper.pm: support multiple use modules in RequireENV
In case it's necessary to `use` multiple modules in order to fully
define the variables in the module whose config is being frozen,
allow multiple modules to be passed in via an 'ARRAY' ref.
For backwards compatibility, if a scalar is passed in, it's silently
converted to an 'ARRAY' ref with a single element.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
In case it's necessary to `use` multiple modules in order to fully
define the variables in the module whose config is being frozen,
allow multiple modules to be passed in via an 'ARRAY' ref.
For backwards compatibility, if a scalar is passed in, it's silently
converted to an 'ARRAY' ref with a single element.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
gitweb/gitbrowser_config.perl: use Girocco::Config
Although the current git-browser configuratino does not actually
need access to the Girocco::Config variables at this time, that's
not the only thing use'ing Girocco::Config does.
The Girocco::Config module also sets up the GIT_* environment
variables to ensure that Git runs in a well-known and reproducible
environment which includes the global git defaults configured for
the Girocco environment.
The gitweb/gitweb_config.perl file already ensures that gitweb runs
in this environment.
Add the necessary "use Girocco::Config" to make sure that git-browser
does too.
Also bring over the extra bits to make sure that replace refs are
enabled for git-browser just like they now are for gitweb and go
ahead and make sure the last expression is a true value, just in case.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Although the current git-browser configuratino does not actually
need access to the Girocco::Config variables at this time, that's
not the only thing use'ing Girocco::Config does.
The Girocco::Config module also sets up the GIT_* environment
variables to ensure that Git runs in a well-known and reproducible
environment which includes the global git defaults configured for
the Girocco environment.
The gitweb/gitweb_config.perl file already ensures that gitweb runs
in this environment.
Add the necessary "use Girocco::Config" to make sure that git-browser
does too.
Also bring over the extra bits to make sure that replace refs are
enabled for git-browser just like they now are for gitweb and go
ahead and make sure the last expression is a true value, just in case.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
make-apache-conf.sh: enhance and tidy up
Improve the apache.conf.in processor in myriad tiny ways.
* Correct the line number in the #line directive to match the script
* Make the IfDefine substitutions understand 0 and non-0 literal numbers
* Make @@var_online_cpus@@ available for substitution
* Simplify the @@<varname>@@ replacement pattern
* Introduce a new @@if(<test>)@@...@@endif@@ conditional substitution
Up until now, conditional parts of the configuration have been handled
by [ab]using the Apache <IfDefine ...> mechanism and inserting a variable
name that's somewhat descriptive yet unlikely to actually exist in the
environment.
While this works just fine, it's a bit of a kludge and can be a bit
perplexing to immediately understand the semantics when looking at the
processed output. However, it does have the advantage of being usable
to any nesting depth since Apache itself handles the nested parsing.
With the new @@if(<test>)@@...@@endif@@ substitution, if the test
fails (the test may optionally start with a '!' to negate it and then
it's either a Girocco::Config variable name or a 0 or non-0 literal number),
the contained lines are simply commented out in the processed output.
In either case, the test itself always passes through to the output but
as commented-out lines.
The @@if(<test>)@@...@@endif@@ substitution works somewhat like the
Apache <IfDefine ...>...</IfDefine> directive in that it's only recognized
if the "@@if(<test>)@@" part appears at the beginning of the line (optionally
preceeded by white space). The matching "@@endif@@" part must also
appear at the beginning of a line (optionally preceeded by white space) to
be recognized. Furthermore, the only thing that may appear on the same
line following the "@@(<test>)@@" or "@@endif@@" is optional white space
and an optional '#'-introduced comment.
While the new @@if(<test>)@@...@@endif@@ substitution does not depend on any
Apache-specific processing, it's not currently possible to nest multiple
levels.
Since the new @@if(<test>)@@...@@endif@@ gets processed before the old
but still supported <IfDefine @@<var>@@> mechanism, it can be used to
disable those lines.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Improve the apache.conf.in processor in myriad tiny ways.
* Correct the line number in the #line directive to match the script
* Make the IfDefine substitutions understand 0 and non-0 literal numbers
* Make @@var_online_cpus@@ available for substitution
* Simplify the @@<varname>@@ replacement pattern
* Introduce a new @@if(<test>)@@...@@endif@@ conditional substitution
Up until now, conditional parts of the configuration have been handled
by [ab]using the Apache <IfDefine ...> mechanism and inserting a variable
name that's somewhat descriptive yet unlikely to actually exist in the
environment.
While this works just fine, it's a bit of a kludge and can be a bit
perplexing to immediately understand the semantics when looking at the
processed output. However, it does have the advantage of being usable
to any nesting depth since Apache itself handles the nested parsing.
With the new @@if(<test>)@@...@@endif@@ substitution, if the test
fails (the test may optionally start with a '!' to negate it and then
it's either a Girocco::Config variable name or a 0 or non-0 literal number),
the contained lines are simply commented out in the processed output.
In either case, the test itself always passes through to the output but
as commented-out lines.
The @@if(<test>)@@...@@endif@@ substitution works somewhat like the
Apache <IfDefine ...>...</IfDefine> directive in that it's only recognized
if the "@@if(<test>)@@" part appears at the beginning of the line (optionally
preceeded by white space). The matching "@@endif@@" part must also
appear at the beginning of a line (optionally preceeded by white space) to
be recognized. Furthermore, the only thing that may appear on the same
line following the "@@(<test>)@@" or "@@endif@@" is optional white space
and an optional '#'-introduced comment.
While the new @@if(<test>)@@...@@endif@@ substitution does not depend on any
Apache-specific processing, it's not currently possible to nest multiple
levels.
Since the new @@if(<test>)@@...@@endif@@ gets processed before the old
but still supported <IfDefine @@<var>@@> mechanism, it can be used to
disable those lines.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
gitweb/gitweb_config.perl: allow replace refs in gitweb
Although the rest of Girocco tries very carefully to avoid being
misled by any "replace" refs by making sure GIT_NO_REPLACE_OBJECTS
has been set to "1" and exported into the environment, that's not
really the best user experience when browsing the repository via
gitweb.
Since gitweb picks up the standard Girocco environment configuration
via Girocco::Config and that *does* always set GIT_NO_REPLACE_REFS=1,
gitweb has been running in a mode where it ignores all replace refs
(if any are present).
Remedy this by adding a little bit extra to gitweb/gitweb_config.perl
to remove the GIT_NO_REPLACE_REFS=1 setting from the environment.
With this change, gitweb now takes into account any replace refs
(refs/replace/*) that may be present in the repository.
Note that since git-browser never uses Girocco::Config, it has always
been showing the graph with the effect of any replace refs present.
Now gitweb's display will match that of git-browser's and be less of
a surprise when replace refs are present in a repository.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Although the rest of Girocco tries very carefully to avoid being
misled by any "replace" refs by making sure GIT_NO_REPLACE_OBJECTS
has been set to "1" and exported into the environment, that's not
really the best user experience when browsing the repository via
gitweb.
Since gitweb picks up the standard Girocco environment configuration
via Girocco::Config and that *does* always set GIT_NO_REPLACE_REFS=1,
gitweb has been running in a mode where it ignores all replace refs
(if any are present).
Remedy this by adding a little bit extra to gitweb/gitweb_config.perl
to remove the GIT_NO_REPLACE_REFS=1 setting from the environment.
With this change, gitweb now takes into account any replace refs
(refs/replace/*) that may be present in the repository.
Note that since git-browser never uses Girocco::Config, it has always
been showing the graph with the effect of any replace refs present.
Now gitweb's display will match that of git-browser's and be less of
a surprise when replace refs are present in a repository.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
gitweb: provide fallback pure perl FastCGI implementation
When running gitweb.cgi, there's a substantial time penalty on each
initial startup of the perl script -- it's a very large script and,
in addition, it then performs some once-at-startup initialization.
In order to deliver decent performance, it's imperative that the
gitweb.cgi perl script be able to run in FastCGI mode if at all possible.
To this end, include an alternative pure perl implementation of
the FastCGI specification that will automatically be used if either
(or both) of the CGI::Fast or FCGI modules are not present.
The pure perl FastCGI implementation may not be quite as fast as
the native FCGI module, but using the pure perl version delivers
far better responsiveness when the native FCGI module is not available
than falling back to a non-FastCGI mode does.
Update the check-perl-modules.pl warning messages to indicate when
this will happen.
No changes are required to gitweb.cgi, the magic happens in the
`gitweb_config.perl` file that it includes and Girocco provides.
When gitweb.cgi attempts to use CGI::Fast/FCGI (which it only does
if it's been configured to run in FastCGI mode), the extra code
that's been added to `gitweb_config.perl` checks to see whether or
not both the CGI::Fast and FCGI modules are present and if not
automatically uses the substitute pure perl FastCGI implementation
instead.
With this support in place, the only factor limiting use of the
FastCGI mode becomes whether or not the web server software itself
supports FastCGI. Most of the web server software available does.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
When running gitweb.cgi, there's a substantial time penalty on each
initial startup of the perl script -- it's a very large script and,
in addition, it then performs some once-at-startup initialization.
In order to deliver decent performance, it's imperative that the
gitweb.cgi perl script be able to run in FastCGI mode if at all possible.
To this end, include an alternative pure perl implementation of
the FastCGI specification that will automatically be used if either
(or both) of the CGI::Fast or FCGI modules are not present.
The pure perl FastCGI implementation may not be quite as fast as
the native FCGI module, but using the pure perl version delivers
far better responsiveness when the native FCGI module is not available
than falling back to a non-FastCGI mode does.
Update the check-perl-modules.pl warning messages to indicate when
this will happen.
No changes are required to gitweb.cgi, the magic happens in the
`gitweb_config.perl` file that it includes and Girocco provides.
When gitweb.cgi attempts to use CGI::Fast/FCGI (which it only does
if it's been configured to run in FastCGI mode), the extra code
that's been added to `gitweb_config.perl` checks to see whether or
not both the CGI::Fast and FCGI modules are present and if not
automatically uses the substitute pure perl FastCGI implementation
instead.
With this support in place, the only factor limiting use of the
FastCGI mode becomes whether or not the web server software itself
supports FastCGI. Most of the web server software available does.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/Config.pm: add a few more reserved_suffixes to list
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
bin/gitweb-gc.sh: add gitweb FastCGI utility script
The FastCGI specification can still be found at:
https://web.archive.org/web/20160310000121/http://www.fastcgi.com/devkit/doc/fcgi-spec.html
or alternatively archived here:
https://fast-cgi.github.io/
Section "7. Errors" states:
> A FastCGI application exits with zero status to indicate that it
> terminated on purpose, e.g. in order to perform a crude form of
> garbage collection. A FastCGI application that exits with nonzero
> status is assumed to have crashed.
The gitweb.cgi application running in FastCGI mode falls into the
category of "application exits with zero status to indicate that
it terminated on purpose, e.g. in order to perform a crude form of
garbage collection."
The standard gitweb.cgi when running in FastCGI mode always exits with
"zero status" after serving 100 requests.
While Apache's mod_fcgid can easily be configured to expect this
behavior (via the `FcgidCmdOptions /gitweb.cgi MaxRequestsPerProcess 100`
configuration line), mod_fcgid handles unexpected "zero status" exits
just fine as well and simply restarts the cgi as needed.
Some other web servers that provide FastCGI support are not nearly as
forgiving (they do not handle the unexpected "zero status" exit) or as
accomodating (they do not have the equivalent of the `MaxRequestsPerProcess`
configuration option).
For example, lighttpd behaves very badly when gitweb exits with the
unexpected "zero status" after 100 requests and then proceeds to drop and
fail any outstanding requests to that FastCGI application before it manages
to restart it.
This causes a bad user experience as every so often requests will fail,
but then succeed when retried.
To accomodate these web servers that poorly implement the FastCGI
specification while allowing gitweb.cgi running in FastCGI mode to work
without intermittent failures, provide an intermediary "helper" script
that conceals the "zero status" (aka "crude form of garbage collection")
exits from the web server and seemlessly restarts a new copy of gitweb.cgi.
This allows use of gitweb.cgi in FastCGI mode reliably with a greater variety
of web server software than would otherwise be possible.
The new script, "gitweb-gc.sh" gets installed in the
"$Girocco::Config::basedir/bin" directory after performing an install of
Girocco.
When configuring a web server that does not properly handle FastCGI
"zero status [...] crude form of garbage collection" exits, instead of
configuring it to execute "$Gircco::Config::cgiroot/gitweb.cgi" directly,
it must be configured to execute "$Girocco::Config::basedir/bin/gitweb-gc.sh"
instead.
This will allow the deficient web server software to provide gitweb
services via FastCGI (much, much, much, much more responsive than spawning
a new gitweb instance for each request) while avoiding sporadic intermittent
failures as gitweb exits after serving 100 requests to perform a "crude form
of garbage collection."
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
The FastCGI specification can still be found at:
https://web.archive.org/web/20160310000121/http://www.fastcgi.com/devkit/doc/fcgi-spec.html
or alternatively archived here:
https://fast-cgi.github.io/
Section "7. Errors" states:
> A FastCGI application exits with zero status to indicate that it
> terminated on purpose, e.g. in order to perform a crude form of
> garbage collection. A FastCGI application that exits with nonzero
> status is assumed to have crashed.
The gitweb.cgi application running in FastCGI mode falls into the
category of "application exits with zero status to indicate that
it terminated on purpose, e.g. in order to perform a crude form of
garbage collection."
The standard gitweb.cgi when running in FastCGI mode always exits with
"zero status" after serving 100 requests.
While Apache's mod_fcgid can easily be configured to expect this
behavior (via the `FcgidCmdOptions /gitweb.cgi MaxRequestsPerProcess 100`
configuration line), mod_fcgid handles unexpected "zero status" exits
just fine as well and simply restarts the cgi as needed.
Some other web servers that provide FastCGI support are not nearly as
forgiving (they do not handle the unexpected "zero status" exit) or as
accomodating (they do not have the equivalent of the `MaxRequestsPerProcess`
configuration option).
For example, lighttpd behaves very badly when gitweb exits with the
unexpected "zero status" after 100 requests and then proceeds to drop and
fail any outstanding requests to that FastCGI application before it manages
to restart it.
This causes a bad user experience as every so often requests will fail,
but then succeed when retried.
To accomodate these web servers that poorly implement the FastCGI
specification while allowing gitweb.cgi running in FastCGI mode to work
without intermittent failures, provide an intermediary "helper" script
that conceals the "zero status" (aka "crude form of garbage collection")
exits from the web server and seemlessly restarts a new copy of gitweb.cgi.
This allows use of gitweb.cgi in FastCGI mode reliably with a greater variety
of web server software than would otherwise be possible.
The new script, "gitweb-gc.sh" gets installed in the
"$Girocco::Config::basedir/bin" directory after performing an install of
Girocco.
When configuring a web server that does not properly handle FastCGI
"zero status [...] crude form of garbage collection" exits, instead of
configuring it to execute "$Gircco::Config::cgiroot/gitweb.cgi" directly,
it must be configured to execute "$Girocco::Config::basedir/bin/gitweb-gc.sh"
instead.
This will allow the deficient web server software to provide gitweb
services via FastCGI (much, much, much, much more responsive than spawning
a new gitweb instance for each request) while avoiding sporadic intermittent
failures as gitweb exits after serving 100 requests to perform a "crude form
of garbage collection."
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
projtool.pl: avoid rare warning message
If --no-owner is used and there are no matching users,
avoid an undefined variable warning when showing the warning
message by using an empty string.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
If --no-owner is used and there are no matching users,
avoid an undefined variable warning when showing the warning
message by using an empty string.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/Config.pm: add a Bitbucket mirror source
Originally Bitbucket was hg only. Then Atlassian acquired Bitbucket.
Not long afterwards support for git was added. And then we have this:
https://bitbucket.org/blog/sunsetting-mercurial-support-in-bitbucket
Nicely commented on by this:
https://www.theregister.com/2019/08/21/bitbucket_mercurial_repositories/
As of 2020-08-26 (according to the first link above):
> All hg repos have now been disabled and cannot be accessed.
Therefore, since Bitbucket is now git only, go ahead and add a new mirror
source option to easily clone from a specified user and project on Bitbucket.
And, at the same time, update the labels for Gitlab to "User" and "Project"
instead of the incorrect leftover old labels from the Gitorious conversion.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Originally Bitbucket was hg only. Then Atlassian acquired Bitbucket.
Not long afterwards support for git was added. And then we have this:
https://bitbucket.org/blog/sunsetting-mercurial-support-in-bitbucket
Nicely commented on by this:
https://www.theregister.com/2019/08/21/bitbucket_mercurial_repositories/
As of 2020-08-26 (according to the first link above):
> All hg repos have now been disabled and cannot be accessed.
Therefore, since Bitbucket is now git only, go ahead and add a new mirror
source option to easily clone from a specified user and project on Bitbucket.
And, at the same time, update the labels for Gitlab to "User" and "Project"
instead of the incorrect leftover old labels from the Gitorious conversion.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
rename: lt256-LICENSE.txt -> lt-LICENSE.txt
The same license applies to all the lt1* files and the lt256* files.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
The same license applies to all the lt1* files and the lt256* files.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
keyinfo: update ssh-keygen info to reflect current versions
Since OpenSSH 6.8p1 (released 2015-03-18), the default fingerprint
shown for public keys has been changed and a new option (-E) added
to select the fingerprint being used.
Go ahead and mention the needed option and argument (-E md5) in
order to make ssh-keygen show the same fingerprint that reguser.cgi
and edituser.cgi do.
While it's certainly possible that clients are using a version of
OpenSSH prior to 6.8p1, don't bother trying to point out that the
`-E` option and argument should be omitted with such older versions.
Older less secure SSH protocols are being phased out and at least
OpenSSH version 7.2p1 is required in order to have support for the
newer "rsa-sha2-256" public key algorithm (see RFC 8332). Since
the `-E` option was added as of OpenSSH version 6.8p1, if support
for the newer more secure protocols is available, then so too will
be support for the `-E` option.
In any case, attempting to use the `-E` option with a pre-6.8p1
version of OpenSSH will conveniently show an "illegal option" message
and a list of allowed options.
The excessive clutter of trying to explain all that concisely in the
various places the "ssh-keygen -l" text appears can therefore be
avoided with very little impact.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Since OpenSSH 6.8p1 (released 2015-03-18), the default fingerprint
shown for public keys has been changed and a new option (-E) added
to select the fingerprint being used.
Go ahead and mention the needed option and argument (-E md5) in
order to make ssh-keygen show the same fingerprint that reguser.cgi
and edituser.cgi do.
While it's certainly possible that clients are using a version of
OpenSSH prior to 6.8p1, don't bother trying to point out that the
`-E` option and argument should be omitted with such older versions.
Older less secure SSH protocols are being phased out and at least
OpenSSH version 7.2p1 is required in order to have support for the
newer "rsa-sha2-256" public key algorithm (see RFC 8332). Since
the `-E` option was added as of OpenSSH version 6.8p1, if support
for the newer more secure protocols is available, then so too will
be support for the `-E` option.
In any case, attempting to use the `-E` option with a pre-6.8p1
version of OpenSSH will conveniently show an "illegal option" message
and a list of allowed options.
The excessive clutter of trying to explain all that concisely in the
various places the "ssh-keygen -l" text appears can therefore be
avoided with very little impact.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
httpspush.html: update with latest OpenSSH information
Starting with OpenSSH version 7.8p1 (2018-08-23), the default format
for OpenSSH private keys has become a new, specific-to-OpenSSH format.
In order to be used for https pushing, the private key must be in
a compatible format that can interoperate with the https software.
Since OpenSSH version v8.1p1 (2019-10-09), the PKCS8 format (see
RFC 5958) can be used. Prior to OpenSSH version v8.1p1, the older
PEM format must be used.
Add instructions detailing how to inspect the RSA private key and
determine whether or not it's in a compatible format and, if not,
how to convert it to a compatible format.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Starting with OpenSSH version 7.8p1 (2018-08-23), the default format
for OpenSSH private keys has become a new, specific-to-OpenSSH format.
In order to be used for https pushing, the private key must be in
a compatible format that can interoperate with the https software.
Since OpenSSH version v8.1p1 (2019-10-09), the PKCS8 format (see
RFC 5958) can be used. Prior to OpenSSH version v8.1p1, the older
PEM format must be used.
Add instructions detailing how to inspect the RSA private key and
determine whether or not it's in a compatible format and, if not,
how to convert it to a compatible format.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
src/lt1: include, build and test ltsha1 utility
Prepare for future use of the LibTomCrypt SHA-1 implementation by
including it along with a test program and script in a similar
fashion to the way the LibTomCrypt SHA-256 implementation has been
included.
Although the standalone ltsha1 utility is not expected to be used
directly (and is not installed by install.sh), it provides a means
to test the implementation to verify that it's working correctly
which install.sh does.
Add the test script to the scripts run by "make test" in the src
subdirectory.
The LibTomCrypt code <https://www.libtom.net/LibTomCrypt/> has a
very liberal license that is fully compatible with GPLv2.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Prepare for future use of the LibTomCrypt SHA-1 implementation by
including it along with a test program and script in a similar
fashion to the way the LibTomCrypt SHA-256 implementation has been
included.
Although the standalone ltsha1 utility is not expected to be used
directly (and is not installed by install.sh), it provides a means
to test the implementation to verify that it's working correctly
which install.sh does.
Add the test script to the scripts run by "make test" in the src
subdirectory.
The LibTomCrypt code <https://www.libtom.net/LibTomCrypt/> has a
very liberal license that is fully compatible with GPLv2.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
gitweb_config.perl: minor text twiddling
Remove extraneous space.
Reword comment that mentions 'use lib "."' as all of those
were removed quite some time ago.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Remove extraneous space.
Reword comment that mentions 'use lib "."' as all of those
were removed quite some time ago.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/HashUtil.pm: increase maximum password rounds
Each time a password is converted for storage in the Girocco
project password file, a random number of iterations will be
chosen (up to a maximum number) and a random password "salt"
will be selected.
Together they help make it computationally much more expensive
to "crack" any of the stored passwords as pre-computing all the
possible "rainbow tables" starts to take up an inordinate amount
of space with the number of possible "salt" values combined with
the number of possible iterations.
With the recent order of magnitude speed up of the computation,
it's now possible to increase the maximum number of iterations
by roughly the same order of magnitude without requiring any
more CPU time than was required before the computation speed up.
Therefore go ahead and increase the maximum number of possible
iterations to make any brute force "crack" attempts just that
little bit more expensive to perform.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Each time a password is converted for storage in the Girocco
project password file, a random number of iterations will be
chosen (up to a maximum number) and a random password "salt"
will be selected.
Together they help make it computationally much more expensive
to "crack" any of the stored passwords as pre-computing all the
possible "rainbow tables" starts to take up an inordinate amount
of space with the number of possible "salt" values combined with
the number of possible iterations.
With the recent order of magnitude speed up of the computation,
it's now possible to increase the maximum number of iterations
by roughly the same order of magnitude without requiring any
more CPU time than was required before the computation speed up.
Therefore go ahead and increase the maximum number of possible
iterations to make any brute force "crack" attempts just that
little bit more expensive to perform.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/HashUtil.pm: speed up by an order of magnitude
The perl transliteration operator (tr/// aka y///) runs much,
much, much more quickly than a global pattern substitution.
Replace the various global substitution patterns with
suitable transliteration operations to vastly speed up the
computation of the result.
Speed is roughly 10x faster with these changes.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
The perl transliteration operator (tr/// aka y///) runs much,
much, much more quickly than a global pattern substitution.
Replace the various global substitution patterns with
suitable transliteration operations to vastly speed up the
computation of the result.
Speed is roughly 10x faster with these changes.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Girocco/CGI.pm: add missing `require Exporter`
Before adding a class to @ISA it's important to make sure it's been
loaded.
Previously this has not been a problem because of `use Girocco::Util`
that executes before modifying @ISA and which has the side effect
of loading the `Exporter` class.
Nevertheless, add the explicit `require Exporter` line to avoid
depending on this happenstance.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Before adding a class to @ISA it's important to make sure it's been
loaded.
Previously this has not been a problem because of `use Girocco::Util`
that executes before modifying @ISA and which has the side effect
of loading the `Exporter` class.
Nevertheless, add the explicit `require Exporter` line to avoid
depending on this happenstance.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
git.git: pick up gitweb GetOptions fix
Fatal errors that occur while attempting to activate fastcgi
mode via the --fastcgi option really must be fatal and not ignored.
Now they are with this fix.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Fatal errors that occur while attempting to activate fastcgi
mode via the --fastcgi option really must be fatal and not ignored.
Now they are with this fix.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
git-http-backend-verify: provide REQUIRE_SSL_CLIENT_VERIFY_SUCCESS knob
If REQUIRE_SSL_CLIENT_VERIFY_SUCCESS has been exported (to any value)
when running the git-http-backend-verify script, then if the request
is a push, `SSL_CLIENT_VERIFY=SUCCESS` will always be required in the
environment.
The sample apache.conf file already checks for this and does not run
the script unless SSL_CLIENT_VERIFY is SUCCESS. As a result it does
not export SSL_CLIENT_VERIFY since that's a minor efficiency penalty
and it's already checked it.
This script does verify that SSL_CLIENT_VERIFY is SUCCESS for push
operations, but only if SSL_CLIENT_VERIFY is actually set.
Nevertheless, for the security paranoid, setting the environment
variable REQUIRE_SSL_CLIENT_VERIFY_SUCCESS (to any value) before
running git-http-backend-verify will always require SSL_CLIENT_VERIFY
to be set to SUCCESS before allowing a push.
This will necessitate adding "+StdEnvVars" to the "SSLOptions"
directive(s) in order to make Apache export SSL_CLIENT_VERIFY when
running the git-http-backend-verify script.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
If REQUIRE_SSL_CLIENT_VERIFY_SUCCESS has been exported (to any value)
when running the git-http-backend-verify script, then if the request
is a push, `SSL_CLIENT_VERIFY=SUCCESS` will always be required in the
environment.
The sample apache.conf file already checks for this and does not run
the script unless SSL_CLIENT_VERIFY is SUCCESS. As a result it does
not export SSL_CLIENT_VERIFY since that's a minor efficiency penalty
and it's already checked it.
This script does verify that SSL_CLIENT_VERIFY is SUCCESS for push
operations, but only if SSL_CLIENT_VERIFY is actually set.
Nevertheless, for the security paranoid, setting the environment
variable REQUIRE_SSL_CLIENT_VERIFY_SUCCESS (to any value) before
running git-http-backend-verify will always require SSL_CLIENT_VERIFY
to be set to SUCCESS before allowing a push.
This will necessitate adding "+StdEnvVars" to the "SSLOptions"
directive(s) in order to make Apache export SSL_CLIENT_VERIFY when
running the git-http-backend-verify script.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
install.sh: make cgiroot available as installation substitution
Add $Girocco::Config::cgiroot to the list of substitutions that
are made when installing various scripts.
Scripts that do not need access to the entire shlib.sh repertoire
can run just that much little bit faster if all they need has
been substituted during installation.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
Add $Girocco::Config::cgiroot to the list of substitutions that
are made when installing various scripts.
Scripts that do not need access to the entire shlib.sh repertoire
can run just that much little bit faster if all they need has
been substituted during installation.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
git-browser.git: pick up release 1.0.1
This version avoids command line argument overrun with
an excessive number of ref names.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>
This version avoids command line argument overrun with
an excessive number of ref names.
Signed-off-by: Kyle J. McKay <mackyle@gmail.com>