import-export daemon: Allow changing compression method
For example, exports on the same node shouldn't be compressed.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Merge branch 'devel-2.1'
Make ConfdInotifyEventHandler a library function
Cut&Paste, plus the following changes: - The class is renamed to SingleFileEventHandler - The monitored filename must be passed in and doesn't default to the ganeti cluster config file - A small docstring is added to the class...
Comment on AsyncNotifier upstream's availability
Since we contributed AsyncNotifier back to the upstream pyinotifyproject, we'll be able, one day, to remove the ganeti version of thatcode. For now we still need it to support older distributions, buthaving a note about when we'll be able to remove it is nice....
Remove errors.ConfdFatalError
This exception is caught, but never thrown. It became useless when wemoved confd from on/off to enabled/disabled, but always running on allnodes. Removing its definition and the code catching it can do no harm.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
RAPI changes for instance moves
Two new resources are added:- /2/instances/$name/prepare-export- /2/instances/$name/export
The documentation for the existing resource for creating instances is updatedfor remote imports. The RAPI client is extended for the new resources....
Implement opcode changes for remote-import
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Implement opcode changes for remote-export
Add opcode to prepare export
To prepare a remote export, the X509 key and certificate need to be generated.A handshake value is also returned for an easier check whether both clustersshare the same cluster domain secret.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Conflicts: lib/luxi.py - trivial
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Abstract the LUXI eom into a constant
Currently the EOM terminator is hardcoded on the server side, and iscustomizable in the Transport object (with the default being the same asthe value found in the server), but not in the luxi client.
With this patch we move the value to constants, and remove the "fake"...
KVM: vhost net acceleration support
This will only work on patched or newer (>= 2.6.34) kernels and with apatched version of qemu-kvm.
Add function to read cluster domain secret
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
LUExportInstance: Remove instance only if export was successful
Until now, the instance was always removed (if asked for by theuser). In case of export errors however, it shouldn't.
RAPI client: Handle urllib2.HTTPError and raise GanetiApiError
This allows users of the RAPI client to catch GanetiApiError for all HTTPerrors.
RAPI: /2/{nodes,instances}/$name should return 404 for unknown items
Currently they return a 500 Server Error, not really usefulfor detecting nonexistent items.
Return disk_template from LUQueryInstanceData
Inter-cluster instance moves need the disk template. As they runLUQueryInstanceData to get an instance's details, the disk templatemust be returned.
RAPI client: Rename Get{Node,Instance}Info, add new GetInstanceInfo
GetInstanceInfo should return the resource /2/instances/$name/info,but so far it returns /2/instances/$name. The same applies toGetNodeInfo, which returns /2/nodes/$name. Both names are stripped...
RAPI client: Log request to be made
Add checks for master IP in cluster verify
This also updates a comment in the unittest for utils.py. We unittestthe new function for two things: correct reporting on real case (forlocalhost), and correct reporting with a mocked-out TcpPing that returns...
RetryOnSignal: handle socket error as well
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Remove unused import from daemon.py
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Luca Bigliardi <shammash@google.com>
Add a forgotten comment about overriding a method
AsyncUDPSocket.handle_error
By overriding the default asyncore handle_error (which closes thesocket) with our own version, which logs what happened but tries toproceed, we can get rid of a couple of try/except blocks. The resultingchurn is deindentation of the internal code....
utils.IgnoreSignals
Remove duplicate code between a couple of asyncore related function byhaving a function in charge of handling EINTR errors. Unittests included.
Conflicts: daemons/ganeti-noded lib/daemon.py lib/rapi/baserlib.py lib/rapi/rlib2.py lib/utils.py
Signed-off-by: Luca Bigliardi <shammash@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix some pylint warnings
Disable warnings for:- except Exception,- use of __errno_location,- redeclaration of handleError()
Signed-off-by: Luca Bigliardi <shammash@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix logging string format
Fix this pylint warning:[W6501, Mlockall] Specify string format arguments as logging function parameters
Fix Filehandler / FileHandler typo
Fix typo spotted by pylint:E1101:2095:LogFileHandler.handleError: Module 'logging' has no 'Filehandler' member
RAPI client: Implement instance creation
Currently this only supports the new instance creation request dataformat version 1, but support for the old version can be easilyimplemented.
Most arguments are optional and documented in the RAPI documentation....
RAPI: Add new request data format for instance creation
As mentioned in commit d975f482d, the current way of creating aninstance via RAPI is not very flexible. With this patch, a newinstance creation request data format is introduced and documented.Support can be detected by checking the list of features returned...
Mention RAPI client in documentation
rapi.baserlib: Add function to check variable type
Also add a separate function to retrieve body parameters. Havingit separate (independent of a class instance) will make it easierto unittest users of this function.
Add new /2/features RAPI resource
The /2/features RAPI resource can be used to detect optionalfeatures implemented by the RAPI server. This will be usedto recognize servers implementing a new request format forinstance creation requests.
Daemons conditionally setup console logging
Use LogFileHandler conditionally in SetupLogging
Add a parameter to SetupLogging to use LogFileHandler (default is off)
Introduce LogFileHandler class
Add a log handler class which logs to /dev/console in case of logging errors.
Add /dev/console constant
Lock PowercycleNode child in memory
Introduce Mlockall()
Add Mlockall() utility to lock current process' virtual adress space into RAM.
Fix wrong variable name in commit d975f482
RAPI: Add initial support for instance creation request version
The way the resource /2/instances expects its request data (e.g.instance name, disks, NICs) to be formatted in a dict is notvery extensible. HV and BE parameters are interleaved with allother values. In commit 495cfdf0 a new parameter “mode” was added...
Convert some ReadFile calls to ReadOneLineFile
For passwords we require strict oneliners, we're a bit more lax with pidand uid files.
utils.ReadOneLineFile()
Read the first non-empty file line. When strict is set, abort if morethan one line is non-empty. Some unittests inspired by the reverted onesfrom commit b774bb106cc28d008e790ad2666eb64c76866fa0, and some new ones.
Remove oneline= parameter from utils.ReadFile
This partially reverts commit b774bb106cc28d008e790ad2666eb64c76866fa0.Unittests unrelated to that particular functionality but introduced inthat commit are left untouched. Since the temporary directory is now...
RAPI client: Fix SSL error reporting for real
My previous patch, commit 857705e8, helped in one particularsituation where the exception didn't contain any arguments(pyOpenSSL reporting a WantReadError, which shouldn't occur with ablocking socket anyway). With this patch, more common and uncommon...
RAPI client: Improve SSL error reporting
RAPI client: Don't check node role in client
Only the server knows which node roles can be set via RAPI.Constants are provided for convenience.
RAPI client: Update ReplaceInstanceDisks
- The RAPI client shouldn't check the replacement mode as only the server knows which ones are valid (constants are still provided for convenience)- Disks shouldn't be a list of names, but of indexes
RAPI client: Fix behaviour of “allocatable” storage flag
When modifying a storage unit, the “allocatable” flag should defaultto “no modification”. This replicates the behaviour of the commandline interface.
RAPI client: Encode boolean and None query values
Boolean values must be passed as 0 or 1. None should be an emptyvalue ("").
RAPI client: No longer check storage type locally
Only the RAPI server (actually masterd) knows which storage typesare valid. The exception can no longer be raised as the type isonly checked in the job.
RAPI client: Add constant for RAPI version
This reverts a60e3cb0a partially by moving the RAPI version into a constant.
Add RAPI client utility module with RAPI PollJob function
The RAPI client module shouldn't depend on any Ganeti module, yet it'suseful to have some Ganeti-specific code, like a PollJob function forRAPI.
RAPI client: Don't assume job IDs to be numeric
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: David Knowles <dknowles@google.com>
RAPI client: Include HTTP error code in GanetiApiError exception
Having the HTTP error code allows users of the class to act differently basedon the error code.
RAPI client: Allow waiting for job changes
RAPI: Allow waiting for job changes
RAPI client: Rename “DeleteJob” to “CancelJob”
Jobs can't be deleted, but cancelled (even though the HTTP method“DELETE” is used).
RAPI client: Various code style changes
- Replace hardcoded values with constants- Code formatting- Exception messages without periods and fixed string formatting
RAPI client: Always pass _SendRequest args as positional
RAPI client: Simplify URL construction
RAPI client: Instantiate JSON encoder only once
RAPI client: Always return job ID
Even removing tags returns a job ID.
RAPI client: Hardcode version in URLs
If the version changes, the API is likely to change as well. Nothaving to ask for the version first makes the code simpler.
Remove httplib2 dependency from ganeti.rapi.client
- It's possible to implement all functionality in ganeti.rapi.client using Python's standard modules httplib and urllib2- By doing so, proper SSL certificate verification is implemented- Adjust some of the code to Ganeti's code style (this is not yet...
utils.ReadFile: Add parameter to read only one line
Reading only one line is useful for reading passwords from files. Thisalso adds unittests for utils.ReadFile.
Fix tiny typo in cluster verify
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Balazs Lecz <leczb@google.com>
Fix import/export
63bcea2a5 added file checks for import/export, but unfortunately theywere broken.
cli: Make PollJob generic to support other protocols
By separating the LUXI-specific code and stdio-related codeinto separate classes, we can make cli.PollJob protocol-agnostic, allowing it to be used with RAPI.
This patch also adds unittests for cli.PollJob....
Force ssh to allocate a tty
This is required to avoid the"Pseudo-terminal will not be allocated because stdin is not a terminal." ssh error message in case a Ganeti script is run non-interactively.
Signed-off-by: Balazs Lecz <leczb@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Retry{Again,Timeout}: explain reraising
IsProcessAlive: retry stat() a few times
On multiprocessor dom0 stat() on /proc can sometimes return EINVAL.Before giving up, we try a few times to get a consistent answer.
utils.Retry: pass up timeout arguments
If Retry has to fail with RetryTimeout, it might be useful to pass theRetry argument to RetryTimeout, to help debugging outside the Retrycycle. We also define a RetryTimeout.RaiseInner() helper, to re-raise anexception passed to RetryAgain. All served with a side of unit tests....
Merge branch 'stable-2.1' into devel-2.1
KVM: only export instance tags if present
Currently non-tagged instances fail starting with a TypeError.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: Balazs Lecz <leczb@google.com>
ssh.GetUserFiles: move to EnsureDirs
We also create a generic SECURE_DIR_MODE constant, rather thanhardcoding 0700 in the code.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Balazs Lecz <leczb@google.com>
Hypervisors: use utils.EnsureDirs
Swap a few os.mkdir calls with utils.EnsureDirs
backend: remove a couple of useless mkdir calls
Those directories must exist for the node daemon to run (it's in thenode daemon's list of ensured directories) and those functions are onlycalled by the node daemon, so there's no point in those checks+mkdir...
daemon.GenericMain: fix docstring
The docstring reports a wrong type for the "dirs" argument. Fixing.
jstore: use EnsureDirs, and add more constants
Small fixes for RAPI client
- Always check SSL certificate for validity- Actually JSON-encode HTTP content
Signed-off-by: David Knowles <dknowles@google.com>Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
backend: Check paths and always write CA file for import/export daemon
Once the import/export daemon uses separate users, the node daemon file (whichis used for intra-cluster transfers) might not be readable anymore. Alwayswriting it to a daemon-specific file will make this easier....
Add optional “salt” argument to utils.{,Verify}Sha1Hmac
We're using salted hashes all over the place.
Use utility functions for HMAC
HMAC will be used in more places. Centralizing some parts can't hurt.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
LUExportInstance: Move exporting code to helper class
This will simplify the implementation of intra-cluster instanceexports and reduces the number of local variables inLUExportInstance.Exec.
verify cluster: check /etc/hosts consistency
If we are told to modify /etc/hosts, then verify its consistency duringcluster verify.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Cleanup /etc/hosts during node removal
It seems that commit d8470559 dropped the cleanup of /etc/hosts when anode is removed from the cluster. I don't know for sure, but it seemsaccidental. As long as we add it to /etc/hosts, we should clean it uptoo....
Move cmdlib._VerifyCertificate to utils
This function will also be useful for inter-cluster instancemoves for verifying certificates.
KVM: make tags available in KVM_NET_SCRIPT
Make instance tags available as a space-separated list during the execution ofthe network setup script. This allows tag-based control of things like firewallrules and/or networking setup.
Remove "ssconf.CheckMasterCandidate"
This function is not used anymore, so there's no point in keeping itaround.
This reverts commit 3f71b464ad5cdd1f1b53f2a31a4eef4e2a5550cc, apart froma one empty line conflict in ssconf.py
Add wrapper class for signal.set_wrapper_fd
Managing file descriptors is always a bit tricky. Having this in a separateclass is better.
Conflicts: lib/utils.py: Trivial
masterd: Log PID, UID and GID of connected client
This can be very useful if client programs run as non-root.