http.auth: Add new function to verify passwords
This new function supports two schemes for passwords:- Old-style cleartext passwords- Hashed passwords according to RFC2617 (H(A1))
Schemes are differentiated by their prefix, a concept alsoused in OpenLDAP. Cleartext passwords can no longer start...
Fix another style issue
For the Nth time, re-fix shadowing of outer-scope variable :)
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Fix an error handling case in TLReplaceDisks
pylint is your friend, since the compiler doesn't exist.
Provide feedback from redistributing configuration
This is particularily useful for “gnt-cluster redist-conf”, butalso for all other cases where the configuration files arerewritten on other nodes.
$ gnt-cluster redist-conf… Copy of file /var/lib/ganeti/config.data to node … failed: Error while...
Fix gnt-node evacuate w. iallocator
Commit 2bb5c911 moved around and changed the _RunAllocator function inthe DiskReplace → TaskLet conversion, but in the process it changed therelocate_from argument from a list of nodes to just the secondary node.This breaks the protocol and current iallocator scripts....
InstanceIpToNodePrimaryIpQuery: use a query dict
In 95b487b we changed InstanceIpToNodePrimaryIpQuery to be able to querymultiple instances at once. We also need to be able to query ipsbelonging to a specific nic link, so what we do is:
1) Move the "query" argument to a dict, containing different fields...
SimpleConfigReader: ips are partitioned by link
We were already half-doing it, but this completes the process.
1) We don't maintain a list of ips or an ip->instance map2) We add a new link,ip->instance map (link->ips list we had)3) We add the link parameter to GetInstanceByIp (making it...
SimpleConfigReader: queries for default nicparams
GetDefaultNicParams returns the default nic parameters.GetDefaultNicLink returns the default nic link.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Import errors in confd init
It's used by some functions defined there.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Olivier Tharan <olive@google.com>
Allow '@' in tag values
This allows using an email address (as is) as part of a tag. The mainproblem that could arise is when parsing tags from a shell script, but(AFAIK) '@' is not a special character when used in values (happy to becorrected if not true)....
cmdlib._AssembleInstanceDisks: Fix case where variable wouldn't be set
The “result” variable may not be set and/or come from the previous loop.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
KVM netscript: add static routes, with no suffix
The /32 suffix is useless, since the kernel already assumes single-host,if no suffix is specified. Moreover we prefer these routes to be"static" so that routing daemons, if present, won't mess with them....
Adding '--no-ssh-init' option to 'gnt-cluster init'.
Allows the initialization of a cluster without the creation or distributionof SSH key pairs. Includes changes for LeaveCluster and RPC.
Signed-off-by: Ken Wehr <ksw@google.com>Signed-off-by: Guido Trotter <ultrotter@google.com>...
KVMHypervisor: implement instance policy routing
Until now we relied on traffic from instances being policy routed via arule based on the instance network. With this change we can enforce iton the instance interfaces. Since the ip rules survive interface...
KVMHypervisor: configure v6 parameters on nic
In routing mode we are tweaking a few parameters on the interface. Withthis patch we'll tweak both the v4 and v6 ones.
confd: query the pnode of multiple instances at once
Signed-off-by: Flavio Silvestrow <flaviops@google.com>Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Try to reduce wrong errors in InstanceShutdown
In backend.InstanceShutdown(), there is a race condition betweenchecking that the instance exists and trying to shut it down whichtranslates sometime in error messages like:
Tue Oct 20 20:08:30 2009 - WARNING: Could not shutdown instance: Failed...
Revert breakage introduced in e4e9b80
Commit e4e9b8064787df01a79846a40f49c8ae06a8eb0e introduced two problemsin backend.InstanceShutdown():
- first, it reduced the check interval significantly (especially for the first few checks); there are very few production VMs that shutdown in...
Xen: Ignore the retry argument in stop instance
Commit 4ad4511 changed the KVM hypervisor to send multiple shutdownrequests to the monitor, but it didn't change this for the Xenhypervisor. We simply remove the return on retry model, since we do wantto send multiple shutdown signals for both Xen and KVM (even if the...
Ensure RpcResult has “payload” attribute
Also add assertions to avoid missing attributes in the future.They won't be included in optimized bytecode.
Introduce checks for /sys and /proc
This patch adds checks for /proc and /sys in cluster verify, sinceGaneti relies on these special filesystems to be mounted.
Fix serializer unittests
Commit d22b29997cd broke the serializer unittests with certainversions of simplejson. This patch removes sort_keys againand implements a slightly more efficient way of detectingsimplejson functionality. The serializer unittests no longer...
bootstrap: Factorize HMAC key generation
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Make bootstrap._GenerateSelfSignedSslCert public
serializer: Sort keys in JSON
mcpu: Use new timeout class for timeout
locking: Convert pipe condition to new timeout class
locking.LockSet: Move timeout calculation to separate class
This class can also be used by mcpu.
locking, mcpu: Ensure timeout is always >= 0.0
locking.LockSet: Improve assertions
locking: Factorize LockSet.acquire
By moving the main code of LockSet.acquire to its own functionwe reduce the code complexity a bit and clarify the exceptionhandling.
This also fixes a case where a lock acquire timeout wasn'thandled correctly, leading to obscure error messages....
mcpu: Make sure added locks are released on errors
opcodes: Add missing shutdown_timeout to OpRemoveInstance
luxi: Pass socket path directly to exception, not in tuple
gnt-* use the correct opcode slot to build opcodes
gnt-* scripts were building wrong opcodes for commands which had theshutdown_timeout slot (due to missing testing after renaming). Fixing.
Also change SHUTDOWN_TIMEOUT_OPT dest field name to "shutdown_timeout":...
rapi: fix tag operations
This patch fixes the tag PUT/DELETE operations, and additionally changesthe Tags* functions to take only positional and not keyword arguments(the defaults do not make any sense at all, and they are always calledwith all arguments)....
cli: add SHUTDOWN_TIMEOUT_OPT
Add timeout options to other LUs
All the LUs that shut down the instance need to be able too pass thetimeout parameter as well.
mcpu: Change lock attempt timeout calculation
With this patch all timeouts are pre-calculated. The interface ofthe _LockTimeoutStrategy class is also changed a bit; NextAttemptnow returns a new instance.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
Code and docstring style fixes
Found using pylint and epydoc.
mcpu: Improve lock reporting with timeouts
mcpu: Implement lock timeouts
The timeout is always between ~0.1 and ~10.0 seconds. A smallvariation of ±5% is added to prevent different jobs fromfighting each other. After 10 attempts to acquire the locks witha timeout, a blocking acquire is made.
Lock status reporting will be improved in a separate patch....
mcpu: Remove unused exclusive_BGL attribute
locking.LockSet: Implement acquire timeouts
The timeout passed to LockSet.acquire() is measured over all lock acquires. IfLockSet.acquire fails to acquire all requested locks within the specifiedamount of time, all locks are released again and the acquire fails....
Accept shutdown timeout from the user
Using the new --timeout option:
- gnt-instance shutdown is changed to accept a timeout- the opcode is changed to hold one- the LU is changed to optionally get one- the rpc is changed to carry one- the backend is changed to take it as a parameter rather than...
ChrootManager: clean StopInstance
Currently it has lots for duplicated code, and internal retries.Clean it up with the following assumptions:
We'll probably be called more than once.It is ok to fail to stop, unless we're called with force=True.If we're called only once, and with force=True it's ok not to run the...
cli: add a timeout option
KVMHypervisor: use the StopInstance retry feature
Since we know StopInstance is going to be called more than once (atleast twice, once with force and once without, but normally quite a lotmore) we don't need our own sleep/loop, and we can just send one monitor...
backend.InstanceShutdown: small cleanup
1) unhardcode the timeout, abstracting it in a constant2) Use time.time() rather than hiding the timeout in a range()3) call hyper.StopInstance multiple times -- currently all hypervisors just ignore all calls but once...
Add default instance shutdown timeout constant
It reflects the "current" two minutes we give to the instance.
Hypervisors: Add retry= to StopInstance
Currently some hypervisors need the stop operations to be retried morethan once, while other ones only do it in one pass. With this changewe'll handle retries outside the hypervisor code, but telling whetherthis is the first try or not....
Get rid of utils.CommaJoin
- We never remember to use it (5 uses vs 21 " ,".join())- It's longer to write than " ,".join()- The added value of the apostrophe in the string is not very much
Match instance and node names case insensitively
Since DNS cannot contain two names with different cases anyway, thisshould be ok.
Add case_sensitive keyword to MatchNameComponent
Now featuring unit testing, and more deterministic results on somecorner cases.
VNC password: move to hv param and use in kvm
Check the OS name for variants
If an OS supports variants, unless --force-variant is specified a validvariant must be passed.
Add force_variant slot to Create/ReinstallInstance
These two opcode need to know whether an unknown variant must be forcedthrough or not.
Allow --force-variant for instance add/reinstall
Passing this option makes an undeclared variant be passed to the os "asis", hoping it'll be able to figure it out (as per the design doc).
Update client os lists to name+variant format
List of OSes are displayed by gnt-os list, rapi, and gnt-instancereinstall --select-os, and checked by burnin. In all of these show thelist with name+variant, if the os has variants.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Add slot and constant for supported OS variants
The slot will contain a list of variants, and the variants file constantcontains the file in the os dir which is supposed to hold the list.
Populate OS variants if an api >= 15 is present
Adding the file name to the os_files dict will fill in the full path andget it checked, if present we also read it and split into lines, one perdeclared variant.
OSEnvironment: populate OS_VARIANT
According to the design on api_version >= 15 the OS variant is the partof the OS name after the "+" sign. If none is found, we just pass in thefirst variant an OS declares (which is bound to exist, as we check forit in _TryOSFromDisk)....
OSFromDisk: handle variants when loading os
When we load an OS from disk, we need _TryOSFromDisk to get the realname, without any variant. This allows any functionality that uses theinstance OS to handle a name with a variant.
Add per-node variants list to OS diagnose output
Add "variants" field to LUDiagnoseOS
If selected this field will contain a list of os variants supported onall nodes.
cli.CalculateOSNames
Given an os and its variants, return a list of "full" os names.
Fix rpc.call_os_get to actually return the OS
Since nobody ever read the actual OS object, this bug was introduced inthe rpc conversion.
Convert os api version file name to a constant
TryOSFromDisk: s/os_scripts/os_files/
We'll be using this dict/loop to check more than just scripts, so we'rerenaming the variables appropriately.
TryOSFromDisk: only check actual os scripts for +x
Currently all checked files in the loop are os scripts, so nothing willchange, but in the future we only want the +x bit on actual os scripts,not necessarily all files.
Add support for using the bootloader in xen-pvm
This patch adds three optional parameters: - 'use_bootloader', whether use or not the bootloader - 'bootloader_path', absolute path to the bootloader - 'bootloader_args', extra arguments to the bootloader...
Replace all xrange() with range()
More locking tests race conditions fixes
There were more race conditions. By adding a notify function toSharedLock.acquire we can prevent them.
LUSetNodeParams: autopromote self when needed
If we're de-offlining or de-draining a node we need to promote it to MCif we have not enough, or the config will be corrupt.
Abstract self-promotion decision
During node add we decide whether to self promote to an MC. Abstractthis decision making to a separate function.
Fix master candidate removal
Currently during a master candidate removal, when it's possible topromote another node, the removal operation fails because of a corruptconfig before it's even possible to do the promotion. Fixing this bydoing the promotion before, excluding the current node....
LUSetNodeParams: Don't break config on mc demotion.
If --force is used to demote an MC, but then there are not enough MCs inthe cluster, the configuration gets corrupted until a node is promoted.
In order to avoid that we only allow demotion with --force if the node...
Master candidate stats, return one more value
Other than returning the current number of candidates, and the number ofdesired and possible candidates, we also return the maximum possiblenumber, even if greater than our desires. All callers for now ignore...
SingleActionPipeCondition =~ s/Action/Notify/
With this patch we simplify usage on the SingleActionCondition (whichwasn't a condition at all) by making it a real condition. This way wecan just wait() on it, or notifyAll() as we would on a normal one. The...
Abstract "base" condition code in a separate class
Each condition has an underlying lock, the acquire and release methods,and a few helper methods to check that it's called in the proper way.
Abstract them to a separate class so we can have more than one without...
locking.SharedLock: Fix bug in delete function
SharedLock.__acquire_unlocked uses keyword parameters. Just passingthe timeout would set the “shared” parameter.
Rename LockSet.acquire parameter “blocking” to “timeout”
Also remove the “blocking” parameter from LockSet.remove andGanetiLockManager.remove. There's no point in implementing timeouts on removalunless we need them.
Change SharedLock to new pipe(2)-based condition
Add _PipeCondition class
_PipeCondition is a condition implemented using pipe(2) and poll(2).It allows the implementation of timeouts without using a busy-wait loopwith time.sleep.
Unlike Python's built-in threading.Condition class and to save filedescriptors and an internal queue, it can only be used to notify...
Add _SingleActionPipeCondition class
This class will be used as a basic block for pipe(2)-basedconditions. Upon initialization it creates a pipe and can benotified once (hence the “single action” in the name). Acallable helper class is used to wait for notifications....
SharedLock: implement timeouts
This patch greatly simplifies the SharedLock code and implementstimeouts for the acquire() and delete() functions. A wrapper aroundPython's threading.Condition class must be used to ensure threadsafety when check whether there are any waiters left....
Extend confd instances ips query
The query now accepts a link parameter.
Signed-off-by: Luca Bigliardi <shammash@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Merge remote branch 'origin/master' into mogu
confd/client: make it possible to update peer list
Until now the peers have to be the same all the time. Adding a newfunction to update the list, and call it from the constructor to avoidduplicating code.
confd/client: pass self to upcalls
It may be handy for upcalls to know which client called them, and callit back. So we create a new "client" field in the upcall target,containing the current client instance
ConfdFilterCallback: fix a bug in expire
The HandleExpire function takes the whole "up" structure, and not justthe salt.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Encode the actual exception raised by LU execution
Currently, the actual exception raised during an LU execution (one ofOpPrereqError, OpExecError, HooksError, etc.) is lost because thejqueue.py code simply sets that to a str(err), and the code in cli.py...
Move the luxi error handling into errors.py
Currently the luxi error handling is hardcoded as special encoding onthe masterd-side and special decoding on the client side. This patchmoves it to errors.py such that other parts of the code can reuse thesame encoding....
Fix the confusing ssh/hostname message in node add
Before, it used to say:
ssh/hostname verification failed node1.example.com -> hostname mismatch, got node2
Now it says for wrong hostnames (maybe too verbose):
ssh/hostname verification failed (checking from node1.example.com): hostname...
Implement ConfdFilterCallback
This callback can be stacked with another one, and will filter duplicateor old results, making handling of results easier.
Remove secrets and kill confd on cluster leave
Confd client: add module level documentation
Populate the docstring with documentation on the client library's usage.
Add uuid on node/instance add and cluster init
This patch does a little bit of cleanup first, since we want to callGenerateUniqueID without reacquiring the lock.
Note that we don't necessarily need to do this for the cluster, since atfirst startup ConfigWriter will do it anyway. But it's better to...
Automatically cleanup _temporary_ids at save
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>(cherry picked from commit 34d657bae4361b9d6fd8c6314dc7cca57b51c773)
Separate the computation of all config IDs
We will need this in another place, so we abstract the 'compute allcurrent IDs' functionality into a separate function. We also change thename of the _ComputeAllLVs to _AllLVs to match the other _All*sfunctions....