Merge pull request #46 from drachenfels-de/imports-and-docs

Imports and docs

Author: r10r

Dockerfile

@@ -26,7 +26,7 @@ ENV K8S_URL="https://dl.k8s.io/v1.20.4/kubernetes-server-linux-amd64.tar.gz"
 ENV LXC_GIT_REPO=https://github.com/lxc/lxc.git
 ENV LXC_GIT_VERSION=master
 
-ENV LXCRI_GIT_REPO=https://github.com/drachenfels-de/lxcri.git
+ENV LXCRI_GIT_REPO=https://github.com/lxc/lxcri.git
 ENV LXCRI_GIT_VERSION=main
 
 COPY install.sh /

README.md

@@ -1,28 +1,38 @@
+[![Go Reference](https://pkg.go.dev/badge/github.com/lxc/lxcri.svg)](https://pkg.go.dev/github.com/lxc/lxcri)
+![Build](https://github.com/lxc/lxcri/actions/workflows/build.yml/badge.svg)
+
 # About
 
 `lxcri` is a wrapper around [LXC](https://github.com/lxc/lxc) which can be used as
 a drop-in container runtime replacement for use by [CRI-O](https://github.com/kubernetes-sigs/cri-o).
 
 ### OCI compliance
 
-With liblxc >= https://github.com/lxc/lxc/commit/b5daeddc5afce1cad4915aef3e71fdfe0f428709
+With liblxc starting from [lxc-4.0.0-927-gb5daeddc5](https://github.com/lxc/lxc/commit/b5daeddc5afce1cad4915aef3e71fdfe0f428709)
 it passes all sonobuoy conformance tests.
 
-## Installation
+## Build
+
+You can use the provided [Dockerfile](Dockerfile) to build an</br>
+
+runtime only image (`lxcri` + `lxc`)
+
+`docker build --build-arg installcmd=install_runtime`
+
+or with everything required for a kubernetes node (kubelet, kubeadm, cri-o, lxcri, lxc ...)
+
+`docker build`
 
-For the installation of the runtime see [install.md](doc/install.md)</br>
-For the installation and initialization of a kubernetes cluster see [kubernetes.md](doc/kubernetes.md)
+Note: The images are not pre-configured and you must follow the steps in setup for now.
 
-## Bugs
+## Setup
 
-* cli: --help shows environment values not defaults https://github.com/urfave/cli/issues/1206
+To use `lxcri` as OCI runtime in `cri-o` see [setup.md](doc/setup.md)
 
-## Requirements and restrictions
+## API Usage
 
-* Only cgroupv2 (unified cgroup hierarchy) is supported.
-* A recent kernel >= 5.8 is required for full cgroup support.
+Please have a look at the [runtime tests](runtime_test.go) for now.
 
-### Unimplemented features
+## Notes
 
-* [runtime: Implement POSIX platform hooks](https://github.com/Drachenfels-GmbH/lxcri/issues/10)
-* [runtime: Implement cgroup2 resource limits](https://github.com/Drachenfels-GmbH/lxcri/issues/11)
+* It's currently only tested with cgroups v2.

cmd/lxcri-hook-builtin/main.go

@@ -5,7 +5,7 @@ import (
 	"os"
 	"path/filepath"
 
-	"github.com/drachenfels-de/lxcri/pkg/specki"
+	"github.com/lxc/lxcri/pkg/specki"
 	"github.com/opencontainers/runtime-spec/specs-go"
 	"golang.org/x/sys/unix"
 )

cmd/lxcri-hook/README.md

@@ -8,7 +8,7 @@ import (
 	"path/filepath"
 	"time"
 
-	"github.com/drachenfels-de/lxcri/pkg/specki"
+	"github.com/lxc/lxcri/pkg/specki"
 	"github.com/opencontainers/runtime-spec/specs-go"
 )
 

cmd/lxcri-hook/main.go

@@ -9,7 +9,7 @@ import (
 	"path/filepath"
 	"time"
 
-	"github.com/drachenfels-de/lxcri/pkg/specki"
+	"github.com/lxc/lxcri/pkg/specki"
 	"github.com/opencontainers/runtime-spec/specs-go"
 	"golang.org/x/sys/unix"
 )

cmd/lxcri-init/main.go

@@ -10,9 +10,9 @@ import (
 	"text/template"
 	"time"
 
-	"github.com/drachenfels-de/lxcri"
-	"github.com/drachenfels-de/lxcri/pkg/log"
-	"github.com/drachenfels-de/lxcri/pkg/specki"
+	"github.com/lxc/lxcri"
+	"github.com/lxc/lxcri/pkg/log"
+	"github.com/lxc/lxcri/pkg/specki"
 	"github.com/opencontainers/runtime-spec/specs-go"
 	"github.com/urfave/cli/v2"
 )

cmd/lxcri/cli.go

@@ -8,7 +8,7 @@ import (
 	"strings"
 	"time"
 
-	"github.com/drachenfels-de/lxcri/pkg/specki"
+	"github.com/lxc/lxcri/pkg/specki"
 	"github.com/opencontainers/runtime-spec/specs-go"
 	"github.com/rs/zerolog"
 	"golang.org/x/sys/unix"

container.go

@@ -7,7 +7,7 @@ import (
 	"path/filepath"
 	"strings"
 
-	"github.com/drachenfels-de/lxcri/pkg/specki"
+	"github.com/lxc/lxcri/pkg/specki"
 	"github.com/opencontainers/runtime-spec/specs-go"
 )
 

create.go

@@ -0,0 +1,111 @@
+## Glossary
+
+* `runtime` the lxcri binary and the command set that implement the [OCI runtime spec](https://github.com/opencontainers/runtime-spec/releases/download/v1.0.2/oci-runtime-spec-v1.0.2.html)
+* `container process`  the process that starts and runs the container using liblxc (lxcri-start)
+* `container config` the LXC config file
+* `bundle config` the lxcri container state (bundle path, pidfile ....)
+* `runtime spec` the OCI runtime spec from the bundle
+
+## Setup 
+
+The runtime binary implements flags that are required by the `OCI runtime spec`,</br>
+and flags that are runtime specific (timeouts, hooks, logging ...).
+
+Most of the runtime specific flags have corresponding environment variables. See `lxcri --help`.</br>
+The runtime evaluates the flag value in the following order (lower order takes precedence).
+
+1. cmdline flag from process arguments (overwrites process environment)
+2. process environment variable (overwrites environment file)
+3. environment file (overwrites cmdline flag default)
+4. cmdline flag default
+
+### Environment variables
+
+Currently you have to compile to environment file yourself.</br>
+To list  all available variables:
+
+```
+grep EnvVars cmd/cli.go | grep -o LXCRI_[A-Za-z_]* | xargs -n1 -I'{}' echo "#{}="
+```
+
+###  Environment file
+
+The default path to the environment file is `/etc/defaults/lxcri`.</br>
+It is loaded on every start of the `lxcri` binary, so changes take immediate effect.</br>
+Empty lines and those commented with a leading *#* are ignored.</br>
+
+A malformed environment will let the next runtime call fail.</br>
+In production it's recommended that you replace the environment file atomically.</br>
+
+E.g the environment file `/etc/default/lxcri` could look like this:
+
+```sh
+LXCRI_LOG_LEVEL=debug
+LXCRI_CONTAINER_LOG_LEVEL=debug
+#LXCRI_LOG_FILE=
+#LXCRI_LOG_TIMESTAMP=
+#LXCRI_MONITOR_CGROUP=
+#LXCRI_LIBEXEC=
+#LXCRI_APPARMOR=
+#LXCRI_CAPABILITIES=
+#LXCRI_CGROUP_DEVICES=
+#LXCRI_SECCOMP=
+#LXCRI_CREATE_TIMEOUT=
+#LXCRI_CREATE_HOOK=/usr/local/bin/lxcri-backup.sh
+#LXCRI_CREATE_HOOK_TIMEOUT=
+#LXCRI_START_TIMEOUT=
+#LXCRI_KILL_TIMEOUT=
+#LXCRI_DELETE_TIMEOUT=
+```
+
+### Runtime (security) features
+
+All supported runtime security features are enabled by default.</br>
+The following runtime (security) features can optionally be disabled.</br>
+Details see `lxcri --help`
+
+* apparmor
+* capabilities
+* cgroup-devices
+* seccomp
+
+### Logging
+
+There is only a single log file for runtime and container process log output.</br>
+The log-level for the runtime and the container process can be set independently.
+
+* containers are ephemeral, but the log file should not be
+* a single logfile is easy to rotate and monitor
+* a single logfile is easy to tail (watch for errors / events ...)
+* robust implementation is easy
+
+#### Log Filtering
+
+Runtime log lines are written in JSON using [zerolog](https://github.com/rs/zerolog).</br>
+The log file can be easily filtered with [jq](https://stedolan.github.io/jq/).</br>
+For filtering with  `jq` you must strip the container process logs with `grep -v '^lxc'`</br>
+
+E.g Filter show only errors and warnings for runtime `create` command:
+
+```sh
+ grep -v '^lxc ' /var/log/lxcri.log |\
+  jq -c 'select(.cmd == "create" and ( .l == "error or .l == "warn")'
+```
+
+#### Runtime log fields
+
+Fields that are always present:
+
+* `l` log level
+* `m` log message
+* `c` caller (source file and line number)
+* `cid` container ID
+* `cmd` runtime command
+* `t` timestamp in UTC (format matches container process output)
+
+### Debugging
+
+Apart from the logfile following resources are useful:
+
+* Systemd journal for cri-o and kubelet services
+* `coredumpctl` if runtime or container process segfaults.