ssconf: error out when writing oversized files
Since we impose a maximum limit when reading ssconf files, let's errorout when trying to write them too big, so we don't pretend everything isok, and make mistakes when we actually read partial files.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Move hash functions to the compat module
Since the hash functions' changed their module name between python 2.4and 2.6, and we have to do an try/import/except trick, we'll do it justonce, for both hash functions, and in compat.py. This also fixes a use...
Fix {Ignore, RetryOn}Signals on socket.error
Some confusion arose handling EINTR on those functions: in python 2.6socket.error is an IOError, and thus: - It's an EnvironmentError - It has an .errno member
In 2.4 and 2.5 it's not, and so its errno variable must be extracted...
import/export daemon: Move command building into separate module
The import/export daemon code is already large. Moving some codeto a separate module will make it smaller and easier to test.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
import/export daemon: Move some I/O processing code to module
The code parsing the child process' output is moved to a separateclass in the impexpd module. As more programs are added, it'llbecome more complex and should be separated.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Add KVM chroot feature
This patch adds a new boolean hypervisor parameter to the KVM hypervisor,named 'use_chroot'.If it's turned on for an instance, than KVM is started in "chroot mode":Ganeti creates an empty directory for the instance and passes the path...
Add RemoveDir utility function
Signed-off-by: Balazs Lecz <leczb@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Merge branch 'devel-2.1'
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
SingleFileEventHandler: Remove try/except blocks
Since now we use the SingleFileEventHandler together with an errorhandling asyncore dispatcher, we don't need the internal try/exceptanymore.
ErrorLoggingAsyncNotifier
This mixes AsyncNotifier with GanetiBaseAsyncoreDispatcher to provide anAsyncNotifier which will log errors, rather than bail out.
daemon.GanetiBaseAsyncoreDispatcher
Abstract a few common functionalities between all ganeti asyncoredispatchers: - Handle errors by logging them, and then continue - By default check sockets only for readability
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Balazs Lecz <leczb@google.com>
Mainloop: handle SIGINT as well (and terminate)
This is needed if daemons are in the foreground, and get ctrl+c-ed bythe user. Also add unittests to make sure the correct signals terminatethe mainloop.
SingleFileEventHandler: update comments
The comments in the SingleFileEventHandler are still confd-specific.Update them to make them generic for any single-file monitoring.
Allow control of import/export compression method
For exports to/imports from the same machine, compression willnot be used anymore.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Put common import/export daemon options into object
The X509 key name and CA are passed from cmdlib all the way tothe backend import/export daemon. With the addition of an optionto choose the compression method, another parameter would haveto be passed all the way. By moving these options to a separate...
import-export daemon: Allow changing compression method
For example, exports on the same node shouldn't be compressed.
Make ConfdInotifyEventHandler a library function
Cut&Paste, plus the following changes: - The class is renamed to SingleFileEventHandler - The monitored filename must be passed in and doesn't default to the ganeti cluster config file - A small docstring is added to the class...
Comment on AsyncNotifier upstream's availability
Since we contributed AsyncNotifier back to the upstream pyinotifyproject, we'll be able, one day, to remove the ganeti version of thatcode. For now we still need it to support older distributions, buthaving a note about when we'll be able to remove it is nice....
Remove errors.ConfdFatalError
This exception is caught, but never thrown. It became useless when wemoved confd from on/off to enabled/disabled, but always running on allnodes. Removing its definition and the code catching it can do no harm.
RAPI changes for instance moves
Two new resources are added:- /2/instances/$name/prepare-export- /2/instances/$name/export
The documentation for the existing resource for creating instances is updatedfor remote imports. The RAPI client is extended for the new resources....
Implement opcode changes for remote-import
Implement opcode changes for remote-export
Add opcode to prepare export
To prepare a remote export, the X509 key and certificate need to be generated.A handshake value is also returned for an easier check whether both clustersshare the same cluster domain secret.
Conflicts: lib/luxi.py - trivial
Abstract the LUXI eom into a constant
Currently the EOM terminator is hardcoded on the server side, and iscustomizable in the Transport object (with the default being the same asthe value found in the server), but not in the luxi client.
With this patch we move the value to constants, and remove the "fake"...
KVM: vhost net acceleration support
This will only work on patched or newer (>= 2.6.34) kernels and with apatched version of qemu-kvm.
Add function to read cluster domain secret
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
LUExportInstance: Remove instance only if export was successful
Until now, the instance was always removed (if asked for by theuser). In case of export errors however, it shouldn't.
RAPI client: Handle urllib2.HTTPError and raise GanetiApiError
This allows users of the RAPI client to catch GanetiApiError for all HTTPerrors.
RAPI: /2/{nodes,instances}/$name should return 404 for unknown items
Currently they return a 500 Server Error, not really usefulfor detecting nonexistent items.
Return disk_template from LUQueryInstanceData
Inter-cluster instance moves need the disk template. As they runLUQueryInstanceData to get an instance's details, the disk templatemust be returned.
RAPI client: Rename Get{Node,Instance}Info, add new GetInstanceInfo
GetInstanceInfo should return the resource /2/instances/$name/info,but so far it returns /2/instances/$name. The same applies toGetNodeInfo, which returns /2/nodes/$name. Both names are stripped...
RAPI client: Log request to be made
Add checks for master IP in cluster verify
This also updates a comment in the unittest for utils.py. We unittestthe new function for two things: correct reporting on real case (forlocalhost), and correct reporting with a mocked-out TcpPing that returns...
RetryOnSignal: handle socket error as well
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Remove unused import from daemon.py
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Luca Bigliardi <shammash@google.com>
Add a forgotten comment about overriding a method
AsyncUDPSocket.handle_error
By overriding the default asyncore handle_error (which closes thesocket) with our own version, which logs what happened but tries toproceed, we can get rid of a couple of try/except blocks. The resultingchurn is deindentation of the internal code....
utils.IgnoreSignals
Remove duplicate code between a couple of asyncore related function byhaving a function in charge of handling EINTR errors. Unittests included.
Conflicts: daemons/ganeti-noded lib/daemon.py lib/rapi/baserlib.py lib/rapi/rlib2.py lib/utils.py
Signed-off-by: Luca Bigliardi <shammash@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix some pylint warnings
Disable warnings for:- except Exception,- use of __errno_location,- redeclaration of handleError()
Signed-off-by: Luca Bigliardi <shammash@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix logging string format
Fix this pylint warning:[W6501, Mlockall] Specify string format arguments as logging function parameters
Fix Filehandler / FileHandler typo
Fix typo spotted by pylint:E1101:2095:LogFileHandler.handleError: Module 'logging' has no 'Filehandler' member
RAPI client: Implement instance creation
Currently this only supports the new instance creation request dataformat version 1, but support for the old version can be easilyimplemented.
Most arguments are optional and documented in the RAPI documentation....
RAPI: Add new request data format for instance creation
As mentioned in commit d975f482d, the current way of creating aninstance via RAPI is not very flexible. With this patch, a newinstance creation request data format is introduced and documented.Support can be detected by checking the list of features returned...
Mention RAPI client in documentation
rapi.baserlib: Add function to check variable type
Also add a separate function to retrieve body parameters. Havingit separate (independent of a class instance) will make it easierto unittest users of this function.
Add new /2/features RAPI resource
The /2/features RAPI resource can be used to detect optionalfeatures implemented by the RAPI server. This will be usedto recognize servers implementing a new request format forinstance creation requests.
Daemons conditionally setup console logging
Use LogFileHandler conditionally in SetupLogging
Add a parameter to SetupLogging to use LogFileHandler (default is off)
Introduce LogFileHandler class
Add a log handler class which logs to /dev/console in case of logging errors.
Add /dev/console constant
Lock PowercycleNode child in memory
Introduce Mlockall()
Add Mlockall() utility to lock current process' virtual adress space into RAM.
Fix wrong variable name in commit d975f482
RAPI: Add initial support for instance creation request version
The way the resource /2/instances expects its request data (e.g.instance name, disks, NICs) to be formatted in a dict is notvery extensible. HV and BE parameters are interleaved with allother values. In commit 495cfdf0 a new parameter “mode” was added...
Convert some ReadFile calls to ReadOneLineFile
For passwords we require strict oneliners, we're a bit more lax with pidand uid files.
utils.ReadOneLineFile()
Read the first non-empty file line. When strict is set, abort if morethan one line is non-empty. Some unittests inspired by the reverted onesfrom commit b774bb106cc28d008e790ad2666eb64c76866fa0, and some new ones.
Remove oneline= parameter from utils.ReadFile
This partially reverts commit b774bb106cc28d008e790ad2666eb64c76866fa0.Unittests unrelated to that particular functionality but introduced inthat commit are left untouched. Since the temporary directory is now...
RAPI client: Fix SSL error reporting for real
My previous patch, commit 857705e8, helped in one particularsituation where the exception didn't contain any arguments(pyOpenSSL reporting a WantReadError, which shouldn't occur with ablocking socket anyway). With this patch, more common and uncommon...
RAPI client: Improve SSL error reporting
RAPI client: Don't check node role in client
Only the server knows which node roles can be set via RAPI.Constants are provided for convenience.
RAPI client: Update ReplaceInstanceDisks
- The RAPI client shouldn't check the replacement mode as only the server knows which ones are valid (constants are still provided for convenience)- Disks shouldn't be a list of names, but of indexes
RAPI client: Fix behaviour of “allocatable” storage flag
When modifying a storage unit, the “allocatable” flag should defaultto “no modification”. This replicates the behaviour of the commandline interface.
RAPI client: Encode boolean and None query values
Boolean values must be passed as 0 or 1. None should be an emptyvalue ("").
RAPI client: No longer check storage type locally
Only the RAPI server (actually masterd) knows which storage typesare valid. The exception can no longer be raised as the type isonly checked in the job.
RAPI client: Add constant for RAPI version
This reverts a60e3cb0a partially by moving the RAPI version into a constant.
Add RAPI client utility module with RAPI PollJob function
The RAPI client module shouldn't depend on any Ganeti module, yet it'suseful to have some Ganeti-specific code, like a PollJob function forRAPI.
RAPI client: Don't assume job IDs to be numeric
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: David Knowles <dknowles@google.com>
RAPI client: Include HTTP error code in GanetiApiError exception
Having the HTTP error code allows users of the class to act differently basedon the error code.
RAPI client: Allow waiting for job changes
RAPI: Allow waiting for job changes
RAPI client: Rename “DeleteJob” to “CancelJob”
Jobs can't be deleted, but cancelled (even though the HTTP method“DELETE” is used).
RAPI client: Various code style changes
- Replace hardcoded values with constants- Code formatting- Exception messages without periods and fixed string formatting
RAPI client: Always pass _SendRequest args as positional
RAPI client: Simplify URL construction
RAPI client: Instantiate JSON encoder only once
RAPI client: Always return job ID
Even removing tags returns a job ID.
RAPI client: Hardcode version in URLs
If the version changes, the API is likely to change as well. Nothaving to ask for the version first makes the code simpler.
Remove httplib2 dependency from ganeti.rapi.client
- It's possible to implement all functionality in ganeti.rapi.client using Python's standard modules httplib and urllib2- By doing so, proper SSL certificate verification is implemented- Adjust some of the code to Ganeti's code style (this is not yet...
utils.ReadFile: Add parameter to read only one line
Reading only one line is useful for reading passwords from files. Thisalso adds unittests for utils.ReadFile.
Fix tiny typo in cluster verify
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Balazs Lecz <leczb@google.com>
Fix import/export
63bcea2a5 added file checks for import/export, but unfortunately theywere broken.
cli: Make PollJob generic to support other protocols
By separating the LUXI-specific code and stdio-related codeinto separate classes, we can make cli.PollJob protocol-agnostic, allowing it to be used with RAPI.
This patch also adds unittests for cli.PollJob....
Force ssh to allocate a tty
This is required to avoid the"Pseudo-terminal will not be allocated because stdin is not a terminal." ssh error message in case a Ganeti script is run non-interactively.
Retry{Again,Timeout}: explain reraising
IsProcessAlive: retry stat() a few times
On multiprocessor dom0 stat() on /proc can sometimes return EINVAL.Before giving up, we try a few times to get a consistent answer.
utils.Retry: pass up timeout arguments
If Retry has to fail with RetryTimeout, it might be useful to pass theRetry argument to RetryTimeout, to help debugging outside the Retrycycle. We also define a RetryTimeout.RaiseInner() helper, to re-raise anexception passed to RetryAgain. All served with a side of unit tests....
Merge branch 'stable-2.1' into devel-2.1
KVM: only export instance tags if present
Currently non-tagged instances fail starting with a TypeError.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: Balazs Lecz <leczb@google.com>
ssh.GetUserFiles: move to EnsureDirs
We also create a generic SECURE_DIR_MODE constant, rather thanhardcoding 0700 in the code.
Hypervisors: use utils.EnsureDirs
Swap a few os.mkdir calls with utils.EnsureDirs
backend: remove a couple of useless mkdir calls
Those directories must exist for the node daemon to run (it's in thenode daemon's list of ensured directories) and those functions are onlycalled by the node daemon, so there's no point in those checks+mkdir...