Check pycurl module at configure time
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
OpCreateInstance: do not require hv/be/os params
It is perfectly legal to create an instance using only defaults(although beparams will be most times passed in), so let's relax therequirement for these three parameters.
Signed-off-by: Iustin Pop <iustin@google.com>...
Fix ganeti-rapi version string
This was "broken" for almost a year :)
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Silence the check-dirs check
The big shell fragment is just noise, for the common case where itdoesn't fail.
A few more type definitions
This is to simplify the type declarations in the actual LUs.
Make _CheckDiskTemplate a valid type checker
Rework the "type" system
This patch merges the _OP_REQP and _OP_DEFS class attributes into a_OP_PARAMS list, which holds both. The associated unittest checks thatall opcode attributes are declared and checked, and that no LU uses theold fields (could be removed later)....
Remove _CheckBooleanOpField
This is no longer used, and we can remove it.
Merge branch 'stable-2.1'
Bump up version for 2.1.5 release
Also update the release date and the NEWS file.
RapiClient: fix multi-authentication in Python 2.6
In Python 2.6 the urllib2.HTTPBasicAuthHandler has a "retried" count forfailed authentications. The handler fails after 5 of them. To solve thiswe reset the handler's "retried" member variable to 0 after every...
Remove rapi-user and rapi-pass from qa-sample.json
After commit 725ec2f10019c35bafeb1aabfce6f14174bf4f46 they are unused.
utils.OwnIpAddress: Change try/except for Python 2.4
Python 2.4 doesn't support “except” and “finally” in the same block.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
RAPI client: Switch to pycURL
Currently the RAPI client uses the urllib2 and httplib modules fromPython's standard library. They're used with pyOpenSSL in a very fragileway, and there are known issues when receiving large responses from a RAPIserver....
baserlib: Use boolean type for boolean variables
This does not yet fix all issues in the RAPI interface which wereintroduced with the type system. More testing is needed.
qa: fix gnt-instance modify -t drbd
We need to pass the secondary node name, not a dict, which is an invalidvalue.
Fixing Makefile.am to reflect the document move and adding of cluster merger
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Adding a user document for the use of cluster-merge
Signed-off-by: René Nussbaumer <rn@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
Move cluster merger design doc to design-2.1.rst
qa: shutdown instance before trying disk convert
Because we have to. :)
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Fix check in gnt-instance modify -t
Currently when changing the instance disks the instance status (whichmust be down) is only checked if the target status is among the mirroredones. This is incorrect: the instance must be down either way, even ifthe desired final status is non-mirrored....
Modify gnt-instance rename to support --no-check-name
Adding check_name option to the opcode and luxi call for instance rename
This will allow instance rename without dns check as it does for instanceadd.
Enable utils.OwnIpAddress to work with IPv6 addresses
Currently a TcpPing using the target address as source is used todetermine wheter or not an IP address belongs to a node. This does notwork for IPv6 anymore (I'm not sure why) as binding to IPv6 localhost as...
Barebones LXC hypervisor
This needs lots of work, but it can successfully launch an LXC-basedinstance. See the docstring for the limitations/work to be done.
Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Balazs Lecz <leczb@google.com>
Document optional ctypes dependency
Signed-off-by: Luca Bigliardi <shammash@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
DRBD IPv6 support
Support IPv6 configuration for 'drbdsetup show' parser and add unittestsconcerning that case. Renames some data files to use consistent namesclarifying their usage.
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Michael Hanselmann <hansmi@google.com>
IPv6 support for utils.TcpPing()
Add function to retrieve family of an ip address
Signed-off-by: Manuel Franceschini <livewire@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
Introduce utils.IsValidIP{4,6}()
This patch introduces functions to check for valid IPv4 and IPv6addresses and converts IsValidIP() to return True if it is either a IPv4or a IPv6 address.
For now we do not change the functional behavior and replace IsValidIP...
User assertFalse instead of assert_(not ...)
Fix osparams bug when gnt-cluster modify'ing
Commit 625ac1135834c27a2e78fd61bf2df2ce29ad2966 introduced 'osparams'slot in OPSetClusterParams which was not passed when invoked viagnt-cluster modify.
Signed-off-by: Manuel Franceschini <livewire@google.com>...
Introduce IPv6 constants
Replace '0.0.0.0' with constant
Rename some constants to facilitate IPv6 support
Add missing pylint disable for "except:"
Why it's needed here but not a few lines above is a mistery that onlypylint understands.
Also fix an indentation error in another disable, for the same function.
Signed-off-by: Guido Trotter <ultrotter@google.com>...
Adding design-doc for privilege separation work done on Ganeti 2.2
Update NEWS for the 2.1.5 release
Pass force variant option at instance creation
This was supposed to be done in"06073e857e3b518c7195d57306ae01793240c0c2" but by mistake the field wasadded to batch create rather than the normal instance create.
masterd: use AsyncTerminatedMessageStream for luxi
Each luxi connection now creates an asyncore MasterClientHandler (whichis an AsyncTerminatedMessageStream subclass, sending each message to aclient worker). This makes it harder to DOS the master daemon by just...
Fix a few uncommon pylint errors in compat.py
- It's ok if the optional modules functools, roman and hashlib are not there.- It's ok to reference any and all before defining them. We're doing all this exactly because, under python 2.4, they are undefined....
BatchCreate: get force_variant from specs not opts
There is no such option on the command line, and for batch creation it'sbetter to get the option from the json file (with a default of false, aswe fixed it in the previous patch).
AsyncTerminatedMessageStream: send_message
This function adds the ability for a AsyncTerminatedMessageStream tohave a thread-safe message delivery function.
AsyncTerminatedMessageStream: limit message count
Currently the message stream can process any number of messages inparallel (if they get dispatched to different threads or processes).In order to limit their number we only handle messages and read from...
Add test script for cfgupgrade
This should catch cases where we update the configuration version, butforget to adjust cfgupgrade accordingly.
Signed-off-by: Michael Hanselmann <hansmi@google.com>Reviewed-by: Guido Trotter <ultrotter@google.com>
cfgupgrade: Add option to override config verification
This is needed in a new test script for cfgupgrade where it's betterto only use the minimum parts necessary for testing cfgupgrade.Including full configuration data to ensure verification doesn't fail...
cfgupgrade: Fix bug when checking configuration directory
In the condition for checking the configuration directory, one “or” should havebeen an “and”. This bug was in cfgupgrade since commit 95e4a8142 (June 2008).Found thanks to a test script.
Signed-off-by: Michael Hanselmann <hansmi@google.com>...
cfgupgrade: Add support for Ganeti 2.2
Since configuration changes are now usually done in the configurationwriter class, cfgupgrade's only task is to change the config version.This means we can easily support upgrading from Ganeti 2.0 and 2.1 to2.2....
Remove dead code from cfgupgrade
Three constants were no longer user after commit 11c31f5cb5, whichremoved support for upgrading from Ganeti 1.2.
BatchCreate: set a default for force_variant
Currently if the json file doesn't contain a "force_variant" parameterBatchCreate stracktraces. We add a default so passing it explicitely isnot needed.
Signed-off-by: Guido Trotter <ultrotter@google.com>Reviewed-by: Balazs Lecz <leczb@google.com>
Better specify old object slots removal policy
The comments seem to imply we can remove deprecated slots after 2.1 isreleased. This is not quite true, if we still want to support upgradingstraight from 2.0. So let's specify better that they can be removed only...
Fix a type declaration error in LUTestAllocator
Also, improve logging of type errors, to ease debugging.
Fix _UpgradeConfig docstring
jqueue: remove the _big_jqueue_lock module global
By using ssynchronized in the new way, we can remove the module-global_big_jqueue_lock and revert back to an internal _lock inside the jqueue.
ssynchronized: act on a class member
The ssynchronized decorator takes the lock to act on in input.With this change we allow a string to be passed, and if so we assume thefunction it protects is a class method, and we act on the member of theclass itself named as the string we got....
Share the jqueue lock on job-local changes
We can share the jqueue lock when we do per-job updates. These onlyconflict with updates/checks on the same job from another thread (eg.CancelJob, ArchiveJob, which keep the lock unshared, since they are lessfrequent)....
_OpExecCallbacks abstract _AppendFeedback
Move some code to a decorated function rather than explicitelyacquiring/releasing the lock in AppendFeedback.
Remove utils.LockedMethod
All users of this function were converted to another method, hence wecan remove it.
jqueue: convert to a SharedLock()
Remove the jqueue _lock member and convert to a _big_jqueue_locksharedlock. This allows smooth transition from the old single lock to amore granular approach.
MarkUnfinishedOps: update job file on disk
Every time we call MarkUnfinishedOps we do it in a try/finally blockthat updates the job file. With this patch we move the try/finallyinside. CancelJobUnlocked is removed, because it just becomes a wrapperover MarkUnfinishedOps with two constant values....
Remove spurious empty line
Remove pred from compat.any/all
This makes it compatible with the python builtin, and we can even usethe builtin when running under the right version of python. The all andany functions are renamed to _all and _any, so that they can be tested,and (non)existing unittests are updated (translation: there are no unit...
Two more fixes for OS params and opcode defaults
If the OS is not using API v20, the parameter verification should beentirely skipped.
The second change is a simple typo.
Fix breakage due to OS parameters
I was using wrong python installation path (thanks Guido!), so I was notactually testing the new backend.py module. Two immediate things bugsare fixed, and after these burnin passes again…
Introduce constants.DTS_MAY_ADOPT
DTS_MAY_ADOPT include disk templates that may use disk adoption and will beused in all respective checks.
Signed-off-by: Apollon Oikonomopoulos <apollon@noc.grnet.gr>Signed-off-by: Iustin Pop <iustin@google.com>Reviewed-by: Iustin Pop <iustin@google.com>
Convert TestDelay repeat to the type system
Turns out with the current implementation of the type system a variablecan both be required and not required. Interesting (but perhaps worth afurther look).
Fix burnin's export test
Rename some constants and relax pylint rules
For the type system, we want a slightly relaxed rule for constantnaming, so we update the pylint rule. But the old _TPInt and _TNEStringwere not clear enough, so we expand them.
Introduce a micro type system for opcodes
Currently, we have one structual validation for opcode attributes: the_OP_REQP, which checks that a given attribute is not 'None', and therest of the checks are done at runtime. This means our type system hastwo types: None versus Not-None....
LU.CheckPrereq: do not require implementation
Currently, the base class LogicalUnit's CheckPrereq will raiseNotImplementedError, which means that the child LUs have to implementit. However, many LUs don't actually have a need for this function(hence the many "pass" statements as the only body)....
Some more CheckPrereq/CheckArguments cleanup
For a few LUs, a few tests in, or even the whole CheckPrereq, can bemoved to CheckArguments, as they don't touch state and only do a 'type'validation.
Add OS verification support to cluster verify
For this, we needed to extend the NodeImage class with a few extravariables, and we do a trick in the node verification where we pick thefirst node that returned valid OS data as the reference node, and then...
Move opcode attribute defaults to data structures
LUExportInstance had two opcode fields set to default via both_CheckBooleanOpField and getattr(…, False).
Cleanup LU.ExpandNames versus CheckArguments
When LogicalUnit.CheckArguments was introduced, not all code dealingwith static argument checking was moved to it; many of these checks wereleft in ExpandNames. With time, most of them migrated, and this patch...
Abstract export mode validity check
The export mode is checked in two places with the exact same code…
RAPI: switch evacuate node to the new model
This patch removes the last use of the old-style OpEvacuateNode. It alsofixes the dry-run mode for this RAPI resource - the dry-run parameterwas not used at all before.
Remove the obsolete EvacuateNode OpCode/LU
All code has been switched to the new-style LU… time for cleanup.
Add OS parameters to cluster and instance objects
The patch also modifies the instance RPC calls to fill the osparameterscorrectly with the cluster defaults, and exports the OS parameters inthe instance/OS environment.
Show OS parameters in cluster/instance info
Add support for OS parameters during instance add
This is not yet complete, as it lacks proper support for instanceimport.
_GetUpdatedParams: enhance value removal options
This patch adds controls for whether we recognizeconstants.VALUE_DEFAULT or not as a default value, and also addsdash-prefixes as another way for parameter removal.
Add support for modifying cluster OS parameters
We use _GetUpdatedParams in order to support removal too, and thenvalidate the OS parameters if the OS exists.
Add support for modifying instance OS parameters
We move the instance OS rename checks earlier, as we need to run thevalidation against the new OS, if it has changed.
Add support for OS parameters during import/export
Nothing special here, just copy/adjust the beparams code.
LUDiagnoseOS: add more fields, cleanup
This patch exports all the way from backend a new field ‘api_version’which holds the list of support API versions, and exposes the (alreadycomputed) ‘parameters’ field.
The patch also reworks (again) the field calculation in its Exec()...
Add a new gnt-os info command
This can be used to show the actual OS parameters and supportedvariants, in a global manner (rather than per-node as gnt-os diagnose).
Simplify gnt-os diagnose output
Currently, we always list the api/variants, even if these are empty.This patch changes so that we make clear distiction for empty values("[no variants]" versus "[variants: ]"), and we only list variants andparameters when the OS API indicates they should be supported....
Update the 2.2 design doc with OS parameters
Silence a pylint warning
The OS parameters code will bump the number of lines over 10K, and thuswe need to silence this (no, we don't want any other module to becomethis big…, so we use a targeted silence only).
Add os api v20 and related fields to the OS object
Add reading of OS parameters from disk
The patch also modifies the internal methods in LUDiagnoseOS and gnt-osto deal with the format change of call_os_diagnose.
Introduce an RPC call for OS parameters validation
While we only support the 'parameters' check today, the RPC call isgeneric enough that will be able to support other checks in the future.The backend function will both validate the parameters list (so as to...
Remove job object condition
We don't need it anymore, since nobody waits on it.
Parallelize WaitForJobChanges
As for QueryJobs we rely on file updates rather than conditionnotification to acquire job changes. In order to do that we use thepyinotify module to watch files. This might make the client a bit slower(pending planned improvements, such as subscription-based...
Update the job file on feedback
This is needed to convert waitforjobchanges to use inotify and theon-disk version and decouple it from the job queue lock. No replicationto remote nodes is done, to keep the operation fast.
Don't lock on QueryJobs, by using the disk version
We move from querying the in-memory version to loading all jobs from thedisk. Since the jobs are written/deleted on disk in an atomic manner, wedon't need to lock at all. Also, since we're just looking at the...
Add JobQueue.SafeLoadJobFromDisk
This will be used to read a job file without having to deal withexceptions from _LoadJobFromDisk.
jqueue._LoadJobFromDisk: remove safety archival
Currently _LoadJobFromDisk archives job files it finds corrupted. Sincewe want to use it to load files without holding locks, this could causea conflict: we just move the feature to _LoadJobUnlocked which is always...
Add repetition count to the TestDelay opcode
If the repetition count is not passed or is passed as 0 we sleep exactlyone time, otherwise we sleep "repeat" times and log in between.
Merge branch 'devel-2.1'