Commit Briefs


d735340e74 Stefan Sperling

pass repository hash algorithm to libexec helpers on the command line

For now, this is a purely mechanical non-functional change which ensures that the main process's idea of the hash algorithm is known to all helpers. This will allow us to reduce overhead by not having to send the hash algorithm along via a struct got_object_id in imsg buffers. Instead we could now send raw SHA1 or SHA256 hashes in messages since the hash algorithm is now known by all processes before messages are exchanged.




17e46db594 Stefan Sperling

do not ask got-read-pack to paint commits which are known keeps or drops

Avoids a long loop observed in gotadmin cleanup with enumeration offloaded to got-read-pack.


808577a26f Stefan Sperling

remove loaded commits counter from gotadmin cleanup progress

This counter was now always zero since the looping code which was raising this counter has been removed.


828ad4330c Stefan Sperling

speed up gotadmin cleanup object enumeration

The gotadmin cleanup main process was enumerating objects one-by-one, before creating a pack file, by opening every object in the repository with a round-trip to libexec helpers like got-read-pack, got-read-blob, etc. This enumeration loop was a remnant of an early version of gotadmin cleanup which only supported cleaning up loose objects. We can remove this loop, and rely on the pack file creation code instead where we benefit from offloading of object enumeration to got-read-pack with much improved speed. As a side-effect, fix some issues where some objects where not added to generated pack files: directly referenced trees and blobs, and tags which are only referenced via other tags. These issues were uncovered by the regression test suite of gotadmin cleanup.


1ba8f4363f Stefan Sperling

stop passing seed around where not needed


cd09e88230 Stefan Sperling

speed up object enumeration when offloaded to got-read-pack

Previously, got-read-pack was using standard tree object imsg data structures when reporting enumrated tree objects which include information that is not needed when enumerating objects. For example, tree paths passed back from got-read-pack were run through a hash function by the main process and never used for anything else. It is faster to pass the required 4 byte hash directly instead of passing the full path. Reduce such overhead as much as possible with a dedicated message structure. My test case was running just the enumeration phase on a git.git repo which contains commit history up to Git's 1c00d2d8392f603a6263f11f1a50fde96ae5475e In this debug build, 'gotadmin pack' exits with "feature not implemented" as soon as object enumeration is done (a change not part of this commit). Before: $ time gotadmin pack -a master 80846 commits colored; 399225 objects found; 160376 trees scanned gotadmin: feature not implemented 1m59.86s real 2m11.57s user 0m09.13s system After: $ time gotadmin pack -a master 80846 commits colored; 399225 objects found; 160376 trees scanned gotadmin: feature not implemented 1m29.78s real 1m39.55s user 0m06.81s system


1365c7b749 Stefan Sperling

bump version number


Branches

Tags

Tree

.gitignorecommits | blame
CHANGEScommits | blame
LICENCEcommits | blame
Makefilecommits | blame
Makefile.inccommits | blame
READMEcommits | blame
TODOcommits | blame
cvg/
gitwrapper/
got/
got-dist.txtcommits | blame
got-version.mkcommits | blame
gotadmin/
gotctl/
gotd/
gotsh/
gotsys/
gotsysctl/
gotsysd/
gotwebctl/
gotwebd/
include/
lib/
libexec/
regress/
sha1-sha2.txtcommits | blame
template/
tog/
util/

README

Game of Trees (Got) is a version control system which prioritizes ease
of use and simplicity over flexibility (https://gameoftrees.org)

Got is still under development; it is being developed exclusively
on OpenBSD and its target audience are OpenBSD developers. Got is
ISC-licensed and was designed with pledge(2) and unveil(2) in mind.

Got uses Git repositories to store versioned data. Git can be used
for any functionality which has not yet been implemented in Got.
It will always remain possible to work with both Got and Git on
the same repository.

A Got release tarball will install files under /usr/local by default.
This default can be changed by passing PREFIX=/some/path to make.

A build started in Got's Git repository will install files under ~/bin,
which may have to be added to $PATH and be created first:

 $ mkdir ~/bin

To compile the Got client tool suite on OpenBSD, run:

 $ make obj
 $ make
 $ make install

This will install the following commands:

 got, the command line interface
 tog, an ncurses-based interactive Git repository browser
 several helper programs from the libexec directory
 man pages (only installed if building sources from a Got release tarball)

Tests will pass only after 'make install' because they rely on installed
binaries in $PATH. Any tests written as shell scripts also depend on git(1).
Tests which use the got clone, fetch, and send commands will fail if
'ssh 127.0.0.1' does not succeed non-interactively.

Tests for HTTP protocol support rely on the HTTP::Daemon Perl module.

 $ doas pkg_add git p5-http-daemon
 $ make regress

To test with packed repositories, run:

 $ make regress GOT_TEST_PACK=1

To test with packed repositories using the ref-delta representation for
deltified objects, run:

 $ make regress GOT_TEST_PACK=ref-delta

To test with sha1 object IDs instead of sha256, run:

 $ make regress GOT_TEST_ALGO=sha1

The GOT_TEST_PACK and GOT_TEST_ALGO flags can be combined to test packed
repositories with sha1 object IDs.

Because got unveils the /tmp directory by default using the /tmp directory
for test data can hide bugs. However, /tmp remains the default because
there is no better alternative that works out of the box. In order to
store test data in a directory other than /tmp, such as ~/got-test, run:

 $ mkdir ~/got-test
 $ make regress GOT_TEST_ROOT=~/got-test

The tog automated test suite is also run with 'make regress'.
Like Got, however, individual tests or the entire suite can be run:

 $ cd regress/tog
 $ make		# run all tests
 $ ./log.sh	# run log view tests

Man page files in the Got source tree can be viewed with 'man -l':

 $ man -l got/got.1
 $ man -l got/git-repository.5
 $ man -l got/got-worktree.5
 $ man -l tog/tog.1

EXAMPLES in got.1 contains a quick-start guide for OpenBSD developers.


To compile the Got server tool suite on OpenBSD, run:

 $ make obj
 $ make server
 $ make server-install

This will install the following commands:

 gotd, the repository server program
 gotctl, the server control utility
 gotsh, the login shell for users accessing the server via the network
 gitwrapper, like mailwrapper(8) but for git-upload-pack and git-receive-pack

See the following manual page files for information about server setup:

 $ man -l gotd/gotd.8
 $ man -l gotd/gotd.conf.5
 $ man -l gotctl/gotctl.8
 $ man -l gotsh/gotsh.1
 $ man -l gitwrapper/gitwrapper.1

See regress/gotd/README for information about running the server test suite.


Game of Trees Web Daemon (gotwebd) is a FastCGI program which displays
repository data and is designed to work with httpd(8).

To compile gotwebd on OpenBSD, run:

 $ make webd
 # make webd-install

This will create the following files:
  the daemon program /usr/local/sbin/gotwebd
  css and image files in /var/www/htdocs/gotwebd
  the gotwebd init script in /etc/rc.d
  man pages (only installed if building sources from a Got release tarball)

Documentation is available in manual pages:

 $ man -l gotwebd/gotwebd.8
 $ man -l gotwebd/gotwebd.conf.5

The gotwebd test suite must be started as root in order to start and stop
the gotwebd daemon. With the gotwebd binary installed, the test suite can
be run from the top-level directory:

 $ doas make webd-regress

The test suite switches to non-root users as appropriate.


gotsysd(8) is a daemon which can be used to manage a gotd(8) server by
committing a configuration file to the special "gotsys.git" repository.

To compile gotsysd on OpenBSD, run:

 $ make sysd
 # make sysd-install

This will create the following files:
  the daemon program /usr/local/sbin/gotsysd
  the control program /usr/local/sbin/gotsysctl
  man pages (only installed if building sources from a Got release tarball)

Related documentation is available in manual pages:

 $ man -l gotsys/gotsys.conf.5
 $ man -l gotsys/gotsys.1
 $ man -l gotsysd/gotsysd.8
 $ man -l gotsysd/gotsysd.conf.5
 $ man -l gotsysctl/gotsysctl.8

See regress/gotsysd/README for information about the gotsysd test suite.


Got can be built with profiling enabled to debug performance issues.
Note that profiled builds cannot make use of pledge(2).
Profiling should only be enabled for one program at a time. Otherwise,
multiple programs will attempt to write to the 'gmon.out' file in the
current working directory.

For example, to compile got-read-pack with profiling enabled:

  $ cd libexec/got-read-pack
  $ make clean
  $ make PROFILE=1
  $ make install

Running any Got command which ends up using got-read-pack should now
produce the file 'gmon.out' in the current working directory.
The gprof2dot program can be used to generate a profile graph:

  $ doas pkg_add gprof2dot graphviz
  $ gprof ~/bin/got-read-pack gmon.out | gprof2dot | dot -T png > profile.png


Guidelines for reporting problems:

All problem/bug reports should include a reproduction recipe in form of a
shell script which starts out with an empty repository and runs a series of
Got and/or Git commands to trigger the problem, be it a crash or some other
undesirable behaviour.

The regress/cmdline directory contains plenty of example scripts.
An ideal reproduction recipe is written as an xfail ("expected failure")
regression test. For a real-world example of an xfail test, see commits
4866d0842a2b34812818685aaa31d3e0a966412d and
2b496619daecc1f25b1bc0c53e01685030dc2c74 in Got's history.

Please take this request very seriously; Ask for help with writing your
regression test before asking for your problem to be fixed. Time invested
in writing a regression test saves time wasted on back-and-forth discussion
about how the problem can be reproduced. A regression test will need to be
written in any case to verify a fix and prevent the problem from resurfacing.

It is also possible to write test cases in C. Various examples of this
exist in the regress/ directory. Most such tests are unit tests; it is
unlikely that a problem found during regular usage will require a test
to be written in C.

Please always try to find a way to trigger your problem via the command line
interface before reporting a problem without a written test case included.
If writing an automated test really turns out to be impossible, please
explain in very clear terms how the problem can be reproduced.

Mail problem reports to: gameoftrees@openbsd.org

If you prefer to send a private report (e.g. for high-impact security issues),
send email to: Stefan Sperling, Omar Polo, and Thomas Adam.
You will find our email addresses in the commit logs of this repository.


Guidelines for submitting patches:

Mail patches to: gameoftrees@openbsd.org
Pull requests via any Git hosting sites will likely be overlooked.
Please keep the intended target audience in mind when contributing to Got.

Game of Trees is entirely handwritten and contributions generated by LLMs
will not be accepted. The main author (Stefan Sperling) has successfully
been avoiding LLMs and would prefer to keep it that way, even if/when legal
uncertainties around the copyright status of LLM-generated output settle.

Should you feel a need to ask an LLM questions in order to help you understand
something, try asking us first. If you are truly trying to wrap your head
around something then you will learn from us, and we will learn from you.
We may have some expertise you are lacking, but because we cannot ignore what
we already know, we can be blind to problems you are seeing. Our code and
documentation can likely be improved as a result of communication, even before
you send a first patch.


Subscribing to the gameoftrees@openbsd.org mailing list:

The mailing list is used for patch reviews, bug reports, and user questions.
To subscribe, send mail to majordomo@openbsd.org with a message body of:
subscribe gameoftrees

See https://www.openbsd.org/mail.html for more information.