This blog post provides a step-by-step guide on how to install the zkg package manager for Zeek, including resolving external module dependencies and common errors encountered during the installation process.
In the Zeek documentation, it is mentioned that the zkg package manager, which is included in the Zeek installation, has two external Python modules as dependencies. These modules are GitPython and semantic-version. If you try to run zkg without installing these Python modules, you will encounter errors. Here is an example of the error message:
zeek-5.2.2 ./bin/zkg -h
error: zkg failed to import one or more dependencies:
* GitPython: https://pypi.org/project/GitPython
* semantic-version: https://pypi.org/project/semantic-version
If you use 'pip', they can be installed like:
pip3 install GitPython semantic-version
Also check the following exception output for possible alternate explanations:
ModuleNotFoundError: No module named 'git'
When you try to install the suggested Python modules, you may encounter another error:
➜ zeek-5.2.2 pip3 install GitPython semantic-version
error: externally-managed-environment
× This environment is externally managed
╰─> To install Python packages system-wide, try apt install
python3-xyz, where xyz is the package you are trying to
install.
If you wish to install a non-Debian-packaged Python package,
create a virtual environment using python3 -m venv path/to/venv.
Then use path/to/venv/bin/python and path/to/venv/bin/pip. Make
sure you have python3-full installed.
If you wish to install a non-Debian packaged Python application,
it may be easiest to use pipx install xyz, which will manage a
virtual environment for you. Make sure you have pipx installed.
See /usr/share/doc/python3.11/README.venv for more information.
note: If you believe this is a mistake, please contact your Python installation or OS distribution provider. You can override this, at the risk of breaking your Python installation or OS, by passing --break-system-packages.
hint: See PEP 668 for the detailed specification.
These errors indicate that the Linux system is preventing the installation of Python packages globally. Instead, it recommends using a virtual environment, which is a best practice for Python development to avoid conflicts between package versions. To install the Python modules in a virtual environment, follow these steps using Zeek version 5.2.2 as an example:
➜ mkdir zeek-5.2.2-env && cd zeek-5.2.2-env
➜ zeek-5.2.2-env python3 -m venv venv
➜ zeek-5.2.2-env source venv/bin/activate
(venv) ➜ zeek-5.2.2-env pip3 install GitPython semantic-version
...
Installing collected packages: smmap, semantic-version, gitdb, GitPython
Successfully installed GitPython-3.1.31 gitdb-4.0.10 semantic-version-2.10.0 smmap-5.0.0
After installing all the required modules, you may encounter an error related to the readline module:
(venv) ➜ zeek-5.2.2-env ~/Workspaces/zeek-5.2.2/bin/zkg -h
Traceback (most recent call last):
File "/home/probe/Workspaces/zeek-5.2.2/bin/zkg", line 88, in <module>
from zeekpkg._util import (
File "/home/probe/Workspaces/zeek-5.2.2/lib/zeek/python/zeekpkg/__init__.py", line 18, in <module>
from .manager import *
File "/home/probe/Workspaces/zeek-5.2.2/lib/zeek/python/zeekpkg/manager.py", line 45, in <module>
from .source import AGGREGATE_DATA_FILE, Source
File "/home/probe/Workspaces/zeek-5.2.2/lib/zeek/python/zeekpkg/source.py", line 15, in <module>
from .package import name_from_path, Package
File "/home/probe/Workspaces/zeek-5.2.2/lib/zeek/python/zeekpkg/package.py", line 11, in <module>
from .uservar import UserVar
File "/home/probe/Workspaces/zeek-5.2.2/lib/zeek/python/zeekpkg/uservar.py", line 8, in <module>
import readline
ModuleNotFoundError: No module named 'readline'
To resolve this issue, you can continue the installation process by running the following command:
(venv) ➜ zeek-5.2.2-env pip3 install readline
...
Successfully installed readline-6.2.4.1
Once all the required modules have been installed, you can successfully run zkg:
(venv) ➜ zeek-5.2.2-env ~/Workspaces/zeek-5.2.2/bin/zkg -h
usage: zkg [-h] [--version] [--configfile FILE | --user] [--verbose] [--extra-source NAME=URL]
{test,install,bundle,unbundle,remove,uninstall,purge,refresh,upgrade,load,unload,pin,unpin,list,search,info,config,autoconfig,env,create,template} ...
A command-line package manager for Zeek.
options:
-h, --help show this help message and exit
--version show program's version number and exit
--configfile FILE Path to Zeek Package Manager config file. Precludes --user.
--user Store all state in user's home directory. Precludes --configfile.
--verbose, -v Increase program output for debugging. Use multiple times for more output (e.g. -vvv).
--extra-source NAME=URL
Add an extra source.
commands:
{test,install,bundle,unbundle,remove,uninstall,purge,refresh,upgrade,load,unload,pin,unpin,list,search,info,config,autoconfig,env,create,template}
See `zkg <command> -h` for per-command usage info.
test Runs unit tests for Zeek packages.
install Installs Zeek packages.
bundle Creates a bundle file containing a collection of Zeek packages.
unbundle Unpacks Zeek packages from a bundle file and installs them.
remove (uninstall) Uninstall a package.
purge Uninstall all packages.
refresh Retrieve updated package metadata.
upgrade Upgrade installed packages to latest versions.
load Register packages to be be auto-loaded by Zeek.
unload Unregister packages to be be auto-loaded by Zeek.
pin Prevent packages from being automatically upgraded.
unpin Allows packages to be automatically upgraded.
list Lists packages.
search Search packages for matching names.
info Display package information.
config Show Zeek Package Manager configuration info.
autoconfig Generate a Zeek Package Manager configuration file.
env Show the value of environment variables that need to be set for Zeek to be able to use installed packages.
create Create a new Zeek package.
template Manage package templates.
Environment Variables:
``ZKG_CONFIG_FILE``: Same as ``--configfile`` option, but has less precedence.
``ZKG_DEFAULT_SOURCE``: The default package source to use (normally https://github.com/zeek/packages).
``ZKG_DEFAULT_TEMPLATE``: The default package template to use (normally https://github.com/zeek/package-template).