Fix Filehandler / FileHandler typo
Fix typo spotted by pylint:E1101:2095:LogFileHandler.handleError: Module 'logging' has no 'Filehandler' member
Signed-off-by: Luca Bigliardi <shammash@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
RAPI client: Implement instance creation
Currently this only supports the new instance creation request dataformat version 1, but support for the old version can be easilyimplemented.
Most arguments are optional and documented in the RAPI documentation....
RAPI: Add new request data format for instance creation
As mentioned in commit d975f482d, the current way of creating aninstance via RAPI is not very flexible. With this patch, a newinstance creation request data format is introduced and documented.Support can be detected by checking the list of features returned...
Mention RAPI client in documentation
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
rapi.baserlib: Add function to check variable type
Also add a separate function to retrieve body parameters. Havingit separate (independent of a class instance) will make it easierto unittest users of this function.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Add new /2/features RAPI resource
The /2/features RAPI resource can be used to detect optionalfeatures implemented by the RAPI server. This will be usedto recognize servers implementing a new request format forinstance creation requests.
Daemons conditionally setup console logging
Signed-off-by: Luca Bigliardi <shammash@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Use LogFileHandler conditionally in SetupLogging
Add a parameter to SetupLogging to use LogFileHandler (default is off)
Introduce LogFileHandler class
Add a log handler class which logs to /dev/console in case of logging errors.
Add /dev/console constant
Lock PowercycleNode child in memory
Introduce Mlockall()
Add Mlockall() utility to lock current process' virtual adress space into RAM.
Fix wrong variable name in commit d975f482
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
RAPI: Add initial support for instance creation request version
The way the resource /2/instances expects its request data (e.g.instance name, disks, NICs) to be formatted in a dict is notvery extensible. HV and BE parameters are interleaved with allother values. In commit 495cfdf0 a new parameter “mode” was added...
Convert some ReadFile calls to ReadOneLineFile
For passwords we require strict oneliners, we're a bit more lax with pidand uid files.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
utils.ReadOneLineFile()
Read the first non-empty file line. When strict is set, abort if morethan one line is non-empty. Some unittests inspired by the reverted onesfrom commit b774bb106cc28d008e790ad2666eb64c76866fa0, and some new ones.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Remove oneline= parameter from utils.ReadFile
This partially reverts commit b774bb106cc28d008e790ad2666eb64c76866fa0.Unittests unrelated to that particular functionality but introduced inthat commit are left untouched. Since the temporary directory is now...
RAPI client: Fix SSL error reporting for real
My previous patch, commit 857705e8, helped in one particularsituation where the exception didn't contain any arguments(pyOpenSSL reporting a WantReadError, which shouldn't occur with ablocking socket anyway). With this patch, more common and uncommon...
RAPI client: Improve SSL error reporting
RAPI client: Don't check node role in client
Only the server knows which node roles can be set via RAPI.Constants are provided for convenience.
RAPI client: Update ReplaceInstanceDisks
- The RAPI client shouldn't check the replacement mode as only the server knows which ones are valid (constants are still provided for convenience)- Disks shouldn't be a list of names, but of indexes
RAPI client: Fix behaviour of “allocatable” storage flag
When modifying a storage unit, the “allocatable” flag should defaultto “no modification”. This replicates the behaviour of the commandline interface.
RAPI client: Encode boolean and None query values
Boolean values must be passed as 0 or 1. None should be an emptyvalue ("").
RAPI client: No longer check storage type locally
Only the RAPI server (actually masterd) knows which storage typesare valid. The exception can no longer be raised as the type isonly checked in the job.
RAPI client: Add constant for RAPI version
This reverts a60e3cb0a partially by moving the RAPI version into a constant.
Add RAPI client utility module with RAPI PollJob function
The RAPI client module shouldn't depend on any Ganeti module, yet it'suseful to have some Ganeti-specific code, like a PollJob function forRAPI.
RAPI client: Don't assume job IDs to be numeric
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: David Knowles <dknowles@google.com>
RAPI client: Include HTTP error code in GanetiApiError exception
Having the HTTP error code allows users of the class to act differently basedon the error code.
RAPI client: Allow waiting for job changes
RAPI: Allow waiting for job changes
RAPI client: Rename “DeleteJob” to “CancelJob”
Jobs can't be deleted, but cancelled (even though the HTTP method“DELETE” is used).
RAPI client: Various code style changes
- Replace hardcoded values with constants- Code formatting- Exception messages without periods and fixed string formatting
RAPI client: Always pass _SendRequest args as positional
RAPI client: Simplify URL construction
RAPI client: Instantiate JSON encoder only once
RAPI client: Always return job ID
Even removing tags returns a job ID.
RAPI client: Hardcode version in URLs
If the version changes, the API is likely to change as well. Nothaving to ask for the version first makes the code simpler.
Remove httplib2 dependency from ganeti.rapi.client
- It's possible to implement all functionality in ganeti.rapi.client using Python's standard modules httplib and urllib2- By doing so, proper SSL certificate verification is implemented- Adjust some of the code to Ganeti's code style (this is not yet...
utils.ReadFile: Add parameter to read only one line
Reading only one line is useful for reading passwords from files. Thisalso adds unittests for utils.ReadFile.
Fix tiny typo in cluster verify
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Balazs Lecz <leczb@google.com>
cli: Make PollJob generic to support other protocols
By separating the LUXI-specific code and stdio-related codeinto separate classes, we can make cli.PollJob protocol-agnostic, allowing it to be used with RAPI.
This patch also adds unittests for cli.PollJob....
Force ssh to allocate a tty
This is required to avoid the"Pseudo-terminal will not be allocated because stdin is not a terminal." ssh error message in case a Ganeti script is run non-interactively.
Signed-off-by: Balazs Lecz <leczb@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Retry{Again,Timeout}: explain reraising
IsProcessAlive: retry stat() a few times
On multiprocessor dom0 stat() on /proc can sometimes return EINVAL.Before giving up, we try a few times to get a consistent answer.
utils.Retry: pass up timeout arguments
If Retry has to fail with RetryTimeout, it might be useful to pass theRetry argument to RetryTimeout, to help debugging outside the Retrycycle. We also define a RetryTimeout.RaiseInner() helper, to re-raise anexception passed to RetryAgain. All served with a side of unit tests....
Merge branch 'stable-2.1' into devel-2.1
KVM: only export instance tags if present
Currently non-tagged instances fail starting with a TypeError.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Reviewed-by: Balazs Lecz <leczb@google.com>
ssh.GetUserFiles: move to EnsureDirs
We also create a generic SECURE_DIR_MODE constant, rather thanhardcoding 0700 in the code.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Balazs Lecz <leczb@google.com>
Hypervisors: use utils.EnsureDirs
Swap a few os.mkdir calls with utils.EnsureDirs
backend: remove a couple of useless mkdir calls
Those directories must exist for the node daemon to run (it's in thenode daemon's list of ensured directories) and those functions are onlycalled by the node daemon, so there's no point in those checks+mkdir...
daemon.GenericMain: fix docstring
The docstring reports a wrong type for the "dirs" argument. Fixing.
jstore: use EnsureDirs, and add more constants
Small fixes for RAPI client
- Always check SSL certificate for validity- Actually JSON-encode HTTP content
Signed-off-by: David Knowles <dknowles@google.com>Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
verify cluster: check /etc/hosts consistency
If we are told to modify /etc/hosts, then verify its consistency duringcluster verify.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Cleanup /etc/hosts during node removal
It seems that commit d8470559 dropped the cleanup of /etc/hosts when anode is removed from the cluster. I don't know for sure, but it seemsaccidental. As long as we add it to /etc/hosts, we should clean it uptoo....
KVM: make tags available in KVM_NET_SCRIPT
Make instance tags available as a space-separated list during the execution ofthe network setup script. This allows tag-based control of things like firewallrules and/or networking setup.
Remove "ssconf.CheckMasterCandidate"
This function is not used anymore, so there's no point in keeping itaround.
This reverts commit 3f71b464ad5cdd1f1b53f2a31a4eef4e2a5550cc, apart froma one empty line conflict in ssconf.py
masterd: Log PID, UID and GID of connected client
This can be very useful if client programs run as non-root.
Factorize LUExportInstance
Apart from moving parts of the code to separate functions, error handlingis also improved by making sure snapshots are always removed.
Use ints instead of strings to represent user-ids
Signed-off-by: Balazs Lecz <leczb@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Fix broken commit 9e302a8
Commit 9e302a8 split the StopInstance function in two without properlyduplicating the local variables.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: René Nussbaumer <rn@google.com>
ssh: make quiet configurable
Currently both CopyFileToNode and BuildCmd hardcode "-q" in the options.This patch moves this setting as a configurable option to_BuildSshOptions.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
SSH: do not check IPs
Since we use the cluster name for the SSH known_hosts file, ssh willalways detect a changed IP (since we never connect to the cluster mastername, but the node names), and will complain about it/try to update theuser known hosts file (since that is /dev/null, it doesn't matter, but...
Add separate module for backported language functionality
utils.py, where they were before, is already huge.
hv_chroot: move unmount to CleanupInstance()
This allows cleanup to be done properly if the "instance" wasn't runningat all (based on the CleanupInstance framework, instead of the retry inhypervisor).
LUSetClusterParams: initialize needed parameters
… since the opcode doesn't auto-initialize to None.
Add a hypervisor constant for migration support
This variable can be used by other tools to determine in a generic waywhether a given hypervisor supports migration or not.
Fix indentation error
Commit 9cf4321fc39ec36359d9c90b22b36d33b6adc2f4 indented some lines by4 spaces rather than 2, and was git-amed without noticing. Fixing.
utils: Add function for partial application of function arguments
The function's code was mostly copied from Python's documentationand it's equivalent to “functools.partial” in Python 2.5 and above.
Add -usbdevice tablet to KVM when using vnc
When using VNC, it is recommended to use a tablet-styleinput device instead of a mouse. This allows most VNC viewersto send proper mouse coordinates to the virtual machine's desktopresulting in perfectly aligned guest and host mouse pointers....
Only use boot=on on non-ide disks only (KVM)
boot=on implies that KVM boots using extboot. This is only requiredto boot non-IDE disks and has the side-effect that there is at mostone bootable device. This behaviour breaks some operating systems,most notably the windows installer that tries to chainload the hard-disk...
KVM: fix a bug in _TryReadUidFile
If the uid pool is not in use _TryReadUidFile will try to return "uid" even if it hasn't tried to read it at all.
Fix RAPI client methods return values
Ensure that all RAPI client methods that should return job ids actuallyreturn job ids.
Signed-off-by: David Knowles <dknowles@google.com>Reviewed-by: Iustin Pop <iustin@google.com>Signed-off-by: Iustin Pop <iustin@google.com>
KVM: implement the HT_SM_POOL security model
In order not to complicate to much the _ExecuteKVMRuntime function withnested try/except/finally/else constructs we move the actualruncmd+check call in a separate _RunKVMCmd function.
KVM: add an instance uid file concept
If this file exists, for an instance, we release the uid containedinside back to the uid pool, at instance shutdown.
ReleaseUid: accept an integer as uid
We can't guarantee to keep the LockedObject around, when ReleaseUid iscalled (because that might happen, for example, after a ganetishutdown/restart). So we'll support releasing on just an integer uid.
LockedUid: add GetUid() method
uidpool users that get passed a LockedUid by the uidpool need to knowwhich one the uid is. Since it's not nice to make them access a privatemember, and we shouldn't make "uid" public either, we'll add a gettermethod.
KVM: move post-shutdown cleanup operations
Currently putting the cleanup just after the actual shutdown ensuresthat it never gets called, due to how the retry/shutdown cycle works.So we move those cleanups to their new dedicated place.
Add CleanupInstance hypervisor call
Currently some hypervisors (namely kvm) need to do some cleanup aftermaking sure an instance is stopped. With the moving of the retry cyclein backend those cleanups were never done. In order to solve this we adda new optional hypervisor function, CleanupInstance, which gets called...
Fix changing the list of enabled hypervisors
When enabling a new hypervisor, we must ensure that we have itsparameters in the cluster (global) hvparams dictionary.
Furthermore, we add a verify check for this case (this should be betterreplaced with an auto-fix?)....
Fix heading to the one of epydoc
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix a couple of pydoc syntax problems
Signed-off-by: Balazs Lecz <leczb@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Readd node: allow changed primary IP
The primary IP is not used for DRBD disks, only for the job queue andinternal commands. As such, it's trivial (with a quiet job queue) tore-ip the node.
The patch just relaxes the checks in LUAddNode, and manual testing shows...
Add ExecuteWithUnusedUid() to uidpool.py
Manage the assignment of uids from the uid pool
Add uid_pool to ssconf
Add --uid-pool option to gnt-cluster init
Add printing of uid_pool to gnt-cluster info
Add --add-uids/--remove-uids to gnt-cluster modify
Add --uid-pool option to gnt-cluster modify
Add lib/uidpool.py module
Add uid_pool cluster parameter
backend: Consolidate code opening real block device
utils: Add class to split string stream into lines
This will be used by the new import/export daemon to splitoutput by its child processes into lines.
Fix cluster behaviour with disabled file storage
There are a few issues with disabled file storage:- cluster initialization is broken by default, as it uses the 'no' setting which is not a valid path- some other parts of the code require the file storage dir to be a...
Fix cfgupgrade with non-default DATA_DIR
Commit 43575108 added bootstrap.GenerateclusterCrypto and commit7506a7f1 changed cfgupgrade to use it. However, this lost thefunctionality of upgrading in non-default DATA_DIR.
To fix this, we enhance bootstrap.GenerateclusterCrypto to accept custom...
Improving the RAPI documentation
Add a identify-defaults options for import
When importing an instance, all the saved valued will be used asexplicitly specified values, overriding the cluster defaults. This meansexport+import will change the status (from default to explicitlyspecified) of parameters....
Fix create/import verification of hvparams
Currently the instance creation checks the cluster hv defaults + the newparameters for validity, ignoring the os-specific hvparams (this was anoversight during the implementation of the os hvp). This patch uses the...
objects.Cluster: add method to get hv defaults
Currently the FillHV method is the one that does the cluster hvparams +os hvparams merger. However, in some cases we need to do just this,without adding the instance hvparams on top.
This patch adds a function to compute just this (hv + os hvp...