History | View | Annotate | Download (18.1 kB)
Merge branch 'devel-2.2'
etags: force Python as a language
etags has cannot properly identify the scripts/gnt-* files as Python, sowe statically force the language to Python.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add template 2.3 design doc
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
makefile: fix TAGS building
“find .” requires that “-path” arguments start with a dot, otherwisethey are not matches. Additionally, we also include the QA files in thetags, for easier search while modifying the QA suite.
Signed-off-by: Iustin Pop <iustin@google.com>...
Adding tool to setup SSH on a remote host
This prepares the remote node to be joined into a cluster
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Introduce git reference/tag tracking for debugging
This patch adds a new vcs-version file that is generated via git (andcan be adapted if VCS is changed) and then embebbded as VCS_VERSION inthe constants module.
This means two things:- local modifications without committing to git (or when using a tar.gz...
Convert RPC client to PycURL
Instead of using our custom HTTP client, using PycURL's multiinterface allows us to get rid of the HTTP client threadpool.The majority of the code is still in the ganeti.http.clientmodule.
A simple per-thread HTTP client pool gives cURL a chance to...
KVM hypervisor: Use utils.ShellWriter for network script
This patch converts hv_kvm to use utils.ShellWriter for writingthe network script. It also adds a few unittests (the firstfor any hypervisor modules).
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
jqueue: Factorize code waiting for job changes
By splitting the _WaitForJobChangesHelper class into multiple smallerclasses, we gain in several places:
- Simpler code, less interaction between functions and variables- Easy to unittest (close to 100% coverage)...
Merge branch 'devel-2.1'
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Manuel Franceschini <livewire@google.com>
Enable from-repository builds on old distributions
… or on distributions which simply have other implementations of man,that do not support '--warnings'.
Introduce lib/netutils.py
This patch moves network utility functions to a dedicated module.
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
BaseDRBD: provide a way to query usermode_helper parameter
Signed-off-by: Luca Bigliardi <shammash@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Silence the check-dirs check
The big shell fragment is just noise, for the common case where itdoesn't fail.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fixing Makefile.am to reflect the document move and adding of cluster merger
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Barebones LXC hypervisor
This needs lots of work, but it can successfully launch an LXC-basedinstance. See the docstring for the limitations/work to be done.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Balazs Lecz <leczb@google.com>
DRBD IPv6 support
Support IPv6 configuration for 'drbdsetup show' parser and add unittestsconcerning that case. Renames some data files to use consistent namesclarifying their usage.
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Add test script for cfgupgrade
This should catch cases where we update the configuration version, butforget to adjust cfgupgrade accordingly.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Makefile: Add support for local Makefile additions
With the recent addition of a check for directories listed in Makefilelocal custom directories are always reported as unlisted. This patchadds support for a “Makefile.local” file, which can adjust settings in...
Makefile: Add check for DIRS consistency
It's easy to forget to add a new directory to DIRS. This check shouldreport such inconsistencies.
Add missing directory from Makefile.am
Add example gnt-debug submit-job json files
These files are being used to test the job queue performance withvarious changes and conditions. Adding them here for posterity.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Add unittest for ganeti-cleaner
Let daemon-utils fix the owners for ganeti-rapi
This is a workaround until we fully switched to user separation and fixes theowners of directories/log files so ganeti-rapi will start flawlessly. This isright now run for every daemon but as it operates on a relatively small subset...
Let ganeti-rapi run under a different user/group
Adding customized user/group as configure flags
import/export daemon: Move command building into separate module
The import/export daemon code is already large. Moving some codeto a separate module will make it smaller and easier to test.
Add asyncnotifier unittests
Add initial mainloop unittests
Add tool to move instances between clusters
This is a first version of the instance move tool and it supports moving1..N instances from one cluster to another. When moving a single instance,the instance can be renamed, allowing for moves within the same cluster...
Fix distcheck broken by check-cert-expired unittest
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
ganeti-cleaner: Remove expired X509 certs
Importing/exporting an instance to a remote machine creates X509certificates which expire after some time. They need to be removed fromthe nodes as they become useless.
Add checks for master IP in cluster verify
This also updates a comment in the unittest for utils.py. We unittestthe new function for two things: correct reporting on real case (forlocalhost), and correct reporting with a mocked-out TcpPing that returns...
Conflicts: daemons/ganeti-noded lib/daemon.py lib/rapi/baserlib.py lib/rapi/rlib2.py lib/utils.py
Signed-off-by: Luca Bigliardi <shammash@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
RAPI: Add new request data format for instance creation
As mentioned in commit d975f482d, the current way of creating aninstance via RAPI is not very flexible. With this patch, a newinstance creation request data format is introduced and documented.Support can be detected by checking the list of features returned...
Introduce Mlockall()
Add Mlockall() utility to lock current process' virtual adress space into RAM.
Add RAPI client utility module with RAPI PollJob function
The RAPI client module shouldn't depend on any Ganeti module, yet it'suseful to have some Ganeti-specific code, like a PollJob function forRAPI.
cmdlib: Add utility for instance data import/export
Interpreting the backend's import/export daemon status is a bit tricky.This utility code keeps track of multiple transfers at the same time.Users can supply callback functions to react to events.
Timeouts are currently hardcoded. Intra-cluster instance moves will likely...
Add separate module for backported language functionality
utils.py, where they were before, is already huge.
Add make commit-check target
distcheck, lint and apidoc. Take three pay only one.
Add ganeti.masterd module
This will be used to keep masterd-specific code.
Conflicts: doc/security.rst trivial lib/cli.py trivial
Signed-off-by: Balazs Lecz <leczb@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Complete check-man changes from 5fa16422
Commit 5fa16422 removed the checks at sed time, but only for %.7. Weneed to do it for %.8 too.
Makefile.am: sort lists, clean *.py[co] from all directories
*.py[co] were not removed from lib/confd. Removing *.py[co]from all directories listed in $DIRS can't hurt.
Add daemon for instance import and export
This backend daemon for instance import and export will be used totransfer instance data to other machines. It is implemented in a genericway to support different ways of data input and output. The third-partyprogram “socat”, which is already used by the KVM hypervisor abstraction,...
Add basic unittests for uid_pool
Signed-off-by: Balazs Lecz <leczb@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Add --uid-pool option to gnt-cluster modify
Forward-port the ganeti 2.0 cfgupgrade
2.1's cfgupgrade doesn't support upgrading from 1.2, only from 2.0.However, it's trivial to forward-port the 2.0 cfgupgrade to work with2.1, thus providing an upgrade path for 1.2 users directly to 2.1,without the intermediate step of installing 2.0....
Add a new tool: sanitize-config
This can be used for two purposes:
- safety copy of the config file, with just the secrets changed- cleanup of the config file (full randomization), so that (e.g.) users could send a broken config file to the devel-list...
Rename the confd_client unittest (to confd.client)
This is to keep same naming across all tests (modules separate with dot,followed by _unittest.py).
Adding RAPI client library.
Signed-off-by: David Knowles <dknowles@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Signed-off-by: Iustin Pop <iustin@google.com>(modified slightly the unittest to account for missing httplib2 library)
Add RPC calls to create and remove X509 certificates
Certificates and keys generated using these functions will be used forinter-cluster instance moves. As per design, the private key should neverleave the node.
Merge remote branch 'origin/devel-2.1'
Fix two issues related to check-man
First, we don't need to check man pages at sed time, because this meanseveryone building the package will do - we only need to check at docbooktime, which is mostly at developer time.
Second, don't force LC_ALL to C, as this breaks newer man-db. I've...
Add unittest for daemon-util
This test doesn't cover everything, but it's better than nothing.
Add support for non-Python unittests
Add unittests for ganeti.errors
utils: Add function to extract X509 cert validity
X509 uses ASN1 GENERALIZEDTIME or UTCTIME to store certificate validity.pyOpenSSL 0.7 and above allow us to retrieve both “notBefore” and“notAfter” as strings. Parsing them turned out to be a challenge since...
Implement disabling of file-based storage
Rationale: the file-based storage backend can add/remove files under acertain directory. However, the master node is also controlling thesetting of the file-based root directory, so basically it means we can'tprevent arbitrary modifications by the master of the node's filesystem....
Make SSH_CONFIG_DIR customizable
This patch adds ability to customize ssh config directory with --with-ssh-config-dir(instead of hardcoded /etc/ssh value). This is useful in Linux distributions withcustom ssh config directories (/etc/openssh in ALTLinux, for example)....
Adding tool for automated cluster-merger
This is the implementation of docs/design-cluster-merger.rst. It allowsthe automatic merging of one or more clusters into the invoking cluster.
While this version is tested and working it still needs some tweaking...
Add make target to generate unittest coverage report
Add unittests for ganeti.opcodes
According to “coverage”, this covers 99% of the code.
Use OpenSSL module instead of binary to generate certs
This saves us one dependency and saves us from complicated handling ofexternal files if we need key and certificate separated from each other.
At the same time, the number of bits used for RSA keys is increased from...
Add function to reset tempfile module after fork
On fork, the tempfile module's pseudo random generator isnot reset. If several processes (e.g. two children or parentand child) try to create a temporary file, they'll conflict.This function can be used to reset the name generator which...
Adding design doc for cluster merger
Signed-off-by: René Nussbaumer <rn@google.com>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>(applied slightly modified patch)
Revert "Workaround man page output for long PREFIX dirs"
This reverts commit 83d9f4366f3aa9ae360e27bfe6619402793e9eb5.
man is still unable to wrap some long lines, so we simply revert this patch(and filter out the specific message in autotools/check-man)....
Workaround man page output for long PREFIX dirs
A long PREFIX variable (to configure) will result in very longLOCALSTATEDIR, which when concatenated with lib/ganeti/ (and even moreitems under it) will go over the 80 char line length we enforce in theman checker....
Check built manpages for errors
One fix is necessary in gnt-cluster.sgml. Also adding “DELETE_ON_ERROR”target to remove output file if an error occurred while building it (inthis case the manpage).
This was reported by Iustin Pop in issue 87 and proposed check method...
Find OpenSSL program at configure time
Otherwise we depend on PATH at runtime.
Makefile.am: Targets used directly should depend on BUILT_SOURCES
Add capability to use syslog for logging
This patch adds a configure-time parameter that will set the defaultsused by all programs, and command-line parameters in the daemons thatallow overriding it.
Syslog 'yes' enables syslog in addition to file-based logging, 'only'...
Add a crude disable for DRBD barriers
Ideally we want to/will have per-device DRBD controls of disk/metadataflushes. In the meantime, we want at least a disable of the barrierfunctionality for cases where one has battery-backed caches.
Background: DRBD has four mechanism of handling ordered disk-writes....
Factorize LUXI parsing and handling code
Also fix a typo in http/__init__.py and add unittestsfor the LUXI parsing and formatting functions.
Don't use hardcoded name for pylint
Look it up at configure time instead.
Partially revert "Makefile.am: Run pylint on all Python code"
This partially reverts commit e4e7c7df3a2356103c2442a91d628ca0470fb267,which made newer pylint versions fail.
Makefile.am: Run pylint on all Python code
That is, all Python code except unittests. In particular,autotool/build-bash-completion was missing from theprevious list.
Provide example default files and install one for development
Allow passing options to pylint in the lint rule
This allows automated builders to override the format, for example.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Olivier Tharan <olive@google.com>
Merge remote branch 'devel-2.1'
Introduce a makefile lint rule
This runs pylint over all the python files. Yes, it takes a long while,but it's the only way to properly analyze the source codes as only inthis way pylint can see all uses of the various modules/classes/etc.
Update the _autoconf.py make rule for pylint
This adds targeted pylint disables, but since _autoconf.py isautogenerated we need to do this in Makefile.am (hence the separatepatch).
Add template Ganeti 2.2 design doc
Signed-off-by: Guido Trotter <ultrotter@google.com>
build-bash-completion: Check for None before comparing
Comparing a number with None is not a good idea:
>>> (0 < None, 0 > None) (False, True)
This patch also adds build-bash-completion to the listof checked Python scripts and wraps one line of more...
sphinx: Treat warnings as errors
This makes it easier to catch warnings.
Include INSTALL in documentation
Fix and simplify socat escape detection
- Program paths should not be --with-… options (see Autoconf docs)- Simplify checks for escape functionality- Make SOCAT_USE_ESCAPE variable a bool
Use “daemon-util” to reload SSH keys
Remove the KVM_MIGRATION_PORT configure.ac param
Since this is easily configurable at run-time, we remove theconfigure-time parameter. If anyone is building custom packages, thenthe default can be tweaked by a one-line patch to constants.py.
Note that this also fixes the type of parameter, the default from...